Welcome, humans.

Every AI model gives you a tiny sliver of what is happening inside it: one answer, one token at a time.

However, some new research proposes that the model may actually be building a much richer internal world first, full of features, circuits, confidence signals, and curved mathematical structures that look more like shapes than words.

In our latest podcast episode, Corey Noles and Grant Harvey sit down with Eric Ho, Cofounder and CEO of Goodfire, to explore what is actually happening inside neural networks, and why understanding those hidden structures could make AI safer, more reliable, and easier to design.

Goodfire is building tools that use AI to interpret AI. Its long-term goal is what Eric calls intentional design: training models less like mystery creatures fed enormous piles of data, and more like software you can inspect, debug, edit, and improve on purpose.

Here are our favorite parts:

• (5:48) The concepts hiding inside models: Goodfire can extract structures representing language, style, users, arithmetic, biology, and even the model’s own uncertainty.

• (12:30) Teaching a model not to hallucinate: Eric explains how researchers found internal hallucination mechanisms, then rewarded the model for avoiding them.

• (13:30) The answer is only the tip of the iceberg: A single token can emerge from an enormous hidden world of knowledge and computation the user never sees.

• (22:29) Models may think in shapes: Goodfire found curved, high-dimensional structures representing concepts such as rabbit ears, horizons, and movement.

• (27:41) The mountain-car test: Push a model in a straight line and the car gets smeared across the mountain. Follow the model’s curved internal geometry and it drives smoothly uphill.

• (33:00) Shape rotators finally get their victory: The models appear to manipulate geometry internally, giving one very online group of thinkers the validation they have been waiting for.

• (39:00) Finding confidence inside the model: A cheaper model could expose when it is uncertain, hedge its answer, or hand the task to a stronger model before making something up.

• (48:51) Designing models like software: Eric explains why reading, steering, and debugging internal representations could change model training from trial-and-error into engineering.

Why watch this? Because interpretability is how we understand the AI systems we are rebuilding our world around. We should probably pay attention to it.

Goodfire is connecting what researchers find inside models to practical outcomes: reducing hallucinations, improving training data, preserving safety guardrails, routing uncertain answers, and helping robotics systems cross the gap between simulation and the real world.

The bottom line: The AI black box may not be as black as we assumed. The more researchers look inside, the more structure they find, and those structures may become the controls we use to build better models.

Watch and/or listen now: YouTube | Spotify | Apple Podcasts

P.S. Eric says only a few hundred full-time industry researchers may be working on interpretability. Jump to (46:30) for why he thinks the field is sitting on an enormous pile of low-hanging fruit.

Additional resources

• Goodfire’s latest neural-geometry research — start here for the new work on block-sparse featurizers, manifolds, geometric representations, and the hidden world inside neural networks.

• Predictive Data Debugging — the research Eric referenced on predicting what a dataset will teach a model before spending the time and compute to train it.

• Features as Rewards — Goodfire’s work using internal model features as reinforcement-learning signals to reduce hallucinations.

• Intentionally Designing the Future of AI — the clearest overview of Goodfire’s larger goal: moving model training from guess-and-check toward systems researchers can inspect, steer, and improve deliberately.

• Explore Silico — Goodfire’s platform for interpreting, debugging, training, and intentionally designing AI models.

• More from the Goodfire blog — company updates, interpretability explainers, research infrastructure, and practical lessons from using AI agents for interpretability research.

THIS EPISODE WAS BROUGHT TO YOU BY…

SAS - AI governance software you can trust

Building trustworthy AI starts with strong governance. SAS delivers AI Governance software that helps organizations manage model risk and work toward compliance across the entire AI life cycle, including generative AI, LLMs, and autonomous agents, without slowing down innovation. Monitor outputs for bias and hallucinations, govern automated decisions, and apply controls across every model.

SAS embeds AI governance directly into your workflows, not as an afterthought.

Scale responsibly, stay audit-ready, and strengthen trust with regulators, customers, and stakeholders.

Learn how SAS helps you govern AI by design at sas.com/aigovernance.

🎙️ In Case You Missed It…

Four recent episodes worth checking out next:

1. Want to see AI leave the browser? Watch: AI That Rides Along With Truck Drivers

TL;DW: Samsara Cofounder and CTO John Bicket explains how AI is moving into fleets, dash cams, dispatching, maintenance, and safety workflows across the physical world.

Why you should watch: This is the episode for anyone wondering what agents look like outside office software, where bad decisions can affect vehicles, workers, equipment, and real-world operations.

• YouTube: Watch Here

• Spotify: Listen Here

• Apple Podcasts: Listen Here

2. Curious what serious AI art workflows look like? Watch: ComfyUI Proves AI Art Is Not Zero Effort

TL;DW: Yannik Marek, cofounder and original creator of ComfyUI, explains how node-based workflows give creators control over models, parameters, memory, hardware, and repeatable visual production.

Why you should watch: If prompt boxes make AI art feel like a slot machine, this shows the other side: visible pipelines, technical choices, iteration, and real creative intent.

• YouTube: Watch Here

• Spotify: Listen Here

• Apple Podcasts: Listen Here

3. Want agents that learn from experts? Watch: Can AI Agents Learn From Expert Corrections?

TL;DW: OpenAI engineers John de Wasseige and Arthur Fernandes Araujo explain how Tax AI turns accountant corrections into structured signals, evals, traces, and scoped product improvements.

Why you should watch: Do not let the word “tax” scare you off. This is an AI engineering masterclass on preserving evidence, handling edge cases, and building agents that improve through expert review.

• YouTube: Watch Here

• Spotify: Listen Here

• Apple Podcasts: Listen Here

4. Want IT that fixes problems before employees complain? Watch: HP Built an AI That Fixes Your Computer Before It Breaks

TL;DW: Larry Meadows from HP shows how Workforce Experience Platform uses AI to predict device problems, recommend fixes, and help IT teams avoid unnecessary hardware refreshes.

Why you should watch: If your company is trying to lower IT costs while adding more AI tools, this connects device health, employee experience, automated remediation, and the growing pressure on enterprise hardware.

• YouTube: Watch Here

• Spotify: Listen Here

• Apple Podcasts: Listen Here

One more thing: We have a goal to hit 50K subscribers by the end of the year, and we’re getting close to 25K away! If you like learning about AI, and already watch some of our videos, do us a favor and click here to subscribe today.

Fun fact: if everyone reading this email clicks this hyperlink, we would hit our goal 10x over. Click the button, we dare you!

Stay curious,

The Neuron Team

P.P.S: Love the newsletter, but don’t want to receive these podcast announcement emails? Don’t unsubscribe — adjust your preferences to opt out of them here instead.