Ryan Carson Teaches Greg Isenberg How to Run an AI Software Factory

We recently spoke with Ryan Carson, where he helped us teach beginners how to learn to code with AI in 2026.

His advice was surprisingly approachable: pick one tool, build something fun, understand one technical layer beneath the agent, and ask it to explain everything you do not understand. You did not need to memorize React, networking, or the finer points of JavaScript before getting started.

You can think of that guide as chapter one.

This new interview Ryan did with Greg Isenberg was like chapter two.

Put simply: our interview walked so Greg's could run.

For example, at (7:40), Carson explains what changed: Devin’s original “AI software engineer” pitch arrived before the models were capable enough. By 2026, the models had caught up to the harness.

Once a real product has customers, bugs, urgent requests, and revenue attached to it, working alongside one AI agent becomes painfully sequential. The advanced problem is coordinating five or ten agents without turning your codebase, budget, and brain into lightly seasoned soup.

Carson’s answer is a software factory: isolated cloud agents, recurring tests, production monitoring, self-improvement loops, and a human operator who spends more time making decisions than writing code.

His thesis can be reduced to one sentence:

The biggest AI coding gains now come from redesigning the development system around agents, rather than finding a smarter prompt.

What changed since our first conversation

During our previous livestream, Carson described keeping three separate checkouts of the same repository. One stayed on the main branch for emergencies, while the other two handled feature work.

That was a pragmatic way to run several local agents without letting their branches collide.

Then Untangle, Carson’s AI platform for family-law firms, reached product-market fit. Real customers started depending on the product, and Carson says he suddenly needed to increase his output by roughly 50 times.

Three folders stopped feeling clever.

They became a ceiling.

At (8:37), Carson describes the goal as scaling himself “horizontally.” Instead of making one agent work harder, he wanted more agents working independently at the same time.

At (12:11), Carson shows the practical result: he commonly runs at least five cloud agents at once, and often ten. Instead of asking each agent to share his laptop, every task receives an isolated development environment.

His slide summarized the change bluntly:

• Local machine: One or two agents before chaos
• Cloud environment: Five to ten agents at once
• Local machine: Branches collide
• Cloud environment: Clean by design
• Local machine: Work happens at the desk
• Cloud environment: Work can continue from anywhere

Or, as the presentation put it:

“The 50x came from leaving the laptop, not from better prompts.”

That line captures Carson’s changed position. The model was already capable. His operating system for using it was too narrow.

Cloud agents vs. Claude Code on your laptop

This distinction confused me at first, too.

Claude Code already reads your repository, edits files, runs tests, uses tools, and works through long tasks. What exactly becomes different when an agent moves “to the cloud”?

Carson starts drawing the distinction at (10:35): “How can you run more agents, get more done?” His answer is to move development into the cloud.

The simplest explanation is that Claude Code and a cloud agent can use similar models, but they work inside different computers.

A local Claude Code session

A standard Claude Code session starts inside a project directory on your machine.

The agent can:

• Read and edit your local files
• Run commands in your terminal
• Start your local development server
• Use the tools and credentials available on your machine
• Consume your laptop’s memory, CPU, and working directories

The model inference still happens remotely through Anthropic, but the development environment being operated is your computer. Anthropic’s setup guide instructs users to install Claude Code and launch it from the local project folder.

One local agent is easy. Ten local agents require ten isolated places to work, plus enough computing power to run their tests and applications.

You can create Git worktrees or duplicate repositories. Carson’s argument is that you are now spending mental energy administering local folders rather than shipping software.

A cloud agent session

At (11:54), Carson explains the mechanism in plain English. The platform creates a virtual machine, or VM, that acts as a fresh development computer for that one agent and task.

A cloud agent gets its own remote computer.

Devin, Carson’s main platform, starts every session inside a Linux virtual machine containing the repository, dependencies, tools, and saved environment configuration. Each session boots from a known snapshot instead of inheriting whatever happened to your laptop yesterday. Devin documents that environment model here.

Amp calls its remote environments “orbs.” Each new orb can contain the code and tools an agent needs, then continue running after the laptop closes. Amp explains agents in orbs here.

The important differences are:

• Every task can receive a fresh environment.
• Several agents can run without touching the same files.
• Long jobs continue while your computer is asleep.
• The agent can test software in its own browser or desktop.
• You can supervise the work through a browser or phone.
• Failed experiments disappear with the environment instead of contaminating your setup.

The cloud agent is closer to a remote employee with a freshly configured laptop. A local agent is closer to someone borrowing yours.

Local work still has a role

Carson’s presentation makes cloud development sound mandatory, but his full answer includes an important caveat.

He still works locally for front-end development that requires rapid visual iteration. Seeing a new interface, steering the agent, and testing small changes can be faster on the machine directly in front of you.

The practical division is:

• Stay local when you need tight, interactive control.
• Move into the cloud when a task can be clearly delegated, tested, and returned as a pull request.
• Use both when the feature begins as a local prototype and becomes a cloud implementation task.

This is also where coding platforms are converging. Codex supports local work and cloud delegation, Cursor now includes cloud agents, Factory offers local and remote Droids, and Amp can launch agents locally or inside remote orbs.

The real choice is no longer “Claude Code or cloud.”

It is which jobs deserve your computer, and which jobs deserve their own.

The advanced agent-management loop

At (1:50), Carson gives Greg Isenberg the premise for the entire conversation: everyone is becoming a manager of agents.

That title comes with a different daily job. At (15:03), Carson says his work now consists largely of making high-stakes decisions. By lunch, he may have made ten to twenty of them.

Greg’s visual framework captures the loop:

1. Delegate the task.
2. Let the agent ship the diff.
3. Review and decide.
4. Repeat until the change ships.

Carson argues that this process makes the manager more technical over time. Reviewing migrations, tests, architecture, and production behavior becomes the education.

1. Delegate one contained task to each agent

Carson’s first rule is one task, one agent, one isolated environment.

A task might be:

• Investigate a failing test
• Add one export function
• Upgrade a dependency
• Reproduce a customer bug
• Improve an onboarding step
• Review one pull request
• Explore one architectural option

He commonly delegates five to ten jobs at once.

The tasks can run simultaneously because each agent gets its own environment. One agent’s broken dependency upgrade does not stop another agent from fixing the signup page.

2. Let the agent return with a reviewable change

The agent works asynchronously, runs tests, and returns with a proposed code change or pull request.

That last part matters.

“Build the feature” is difficult to review. A pull request containing a focused change, test output, screenshots, and an explanation is a manageable unit of work.

This connects directly to our guide on how the Claude Code team uses verification, routines, and agent loops. The agent should finish with evidence, rather than a cheerful claim that everything looks fantastic.

3. Review and decide

Those decisions include:

• Accept the pull request.
• Request a different implementation.
• Ask for stronger tests.
• Redirect the agent toward a new diagnosis.
• Kill a weak idea before it consumes more resources.
• Decide whether the underlying product behavior is correct.

His slide describes the job in three words:

“Decide, don’t code.”

That does not mean the human becomes less technical. Carson says repeatedly reviewing architecture, tests, migrations, queries, and production behavior has made him more technical than he has ever been.

The agent writes the diff. The manager must understand its consequences.

4. Repeat the loop

By (17:37), Carson is showing the output from that loop: 22 to 25 pull requests on an average day, with some days reaching roughly 40.

That output comes from repeating the same loop:

1. Delegate.
2. Let the agent work.
3. Review the evidence.
4. Decide.
5. Merge, redirect, or reject.
6. Start the next job.

The loop turns typing into the cheap part of software development.

Judgment becomes the scarce resource.

How Carson keeps ten agents from eating his brain

More parallel work creates more decisions. Starting at (14:53), Greg asks the obvious question: how do you manage a 50-fold increase in output without going absolutely crazy?

Carson’s answer becomes another six-part framework:

1. Run five to ten agents in parallel.
2. Pin only the threads that matter today.
3. Sweep those threads on a roughly 25-minute cadence.
4. Use the phone to keep agents moving.
5. Spend your time deciding rather than coding.
6. Ship the work that survives review.

As Greg’s slide puts it: “The bottleneck stopped being typing. It’s judgment.”

Pin only the work that matters today

Carson divides agent sessions into two groups:

• Important work that must move today
• Everything else

He pins the critical sessions and lets smaller fixes run without demanding constant attention.

This prevents one minor bug investigation from receiving the same mental weight as a production migration.

Check agents on a cadence

Constant monitoring destroys the benefit of delegation.

Carson checks important threads on an approximate 25-minute cadence.

During each sweep, he:

• Checks progress
• Answers blocking questions
• Redirects weak approaches
• Reviews completed work
• Launches the next tasks

Then he steps away again.

The purpose is to keep agents moving without making yourself watch them move.

Keep the day’s priorities outside the agent interface

Carson still uses a paper to-do list.

At (19:04), he explains that a stream of bugs, alerts, and pull requests can easily consume the whole day. Writing down the few outcomes that matter keeps the system oriented toward product progress.

His eight-screen setup looks futuristic. His prioritization system could have been purchased in 1997.

The agents have changed. Human susceptibility to shiny side quests remains undefeated.

Turn the phone into a control surface

Carson estimates that roughly half his work happens on his phone.

He is not writing functions on a tiny keyboard. He is:

• Reading agent summaries
• Answering questions
• Reviewing pull requests
• Giving feedback
• Approving or rejecting work
• Checking automation results

His argument is that agents can work continuously, but only when their manager remains available to make the decisions blocking them.

You do not need to copy the “work in the shower” part of his routine. Boundaries remain legal in most jurisdictions.

The useful idea is that cloud development separates supervision from the desk.

The three automations doing the busywork

At (24:22), the conversation moves from parallel agents to automations.

Carson recommends looking for anything a team once checked during a daily or weekly meeting, then turning that review into a scheduled agent workflow. His current system includes three:

• An end-to-end signup test that runs three times a week
• A production watchdog that reports every morning
• A self-improvement loop that grades Grace’s conversations and prepares roughly three fixes per day

All three run asynchronously. All three compress their work into a summary Carson can review.

Automation one: the end-to-end signup test

At (24:58), Carson walks through an automation that recreates Untangle’s real customer journey in a browser.

Three times a week, a Devin agent:

1. Opens Untangle in a browser.
2. Creates an account.
3. Starts a legal case.
4. Onboards a client.
5. Sends the client through the discovery flow.
6. Records what happens.
7. Identifies failures.
8. Launches another session to investigate broken steps.

Carson says each run costs roughly $60 in tokens, largely because browser testing requires many actions.

The automation exists to catch failures that normal unit tests miss. A test suite might confirm that the signup function returns the correct response. A browser agent confirms that a customer can actually find the button, complete the form, and arrive at the expected page.

Devin can use a full Linux desktop, interact with graphical applications, take screenshots, and verify changes visually. Devin documents computer use here.

When something fails, the system follows a second loop:

• Detect the failure
• Start a triage agent
• Reproduce the problem
• Attempt a fix
• Run the test again
• Prepare a pull request
• Report the result

This moves the workflow beyond “tell me when something breaks.”

The system begins repairing itself before Carson opens Slack.

Automation two: the production watchdog

At (30:23), Carson introduces his “production watchdog,” which reviews activity from Untangle’s paying customers every morning at 9 a.m.

It examines the previous day’s database events and creates a summary of what customers did, what worked, and what looked unusual.

For important customers, the report links directly into the corresponding production interface. Carson can click from a strange event into the real experience and inspect it himself.

This is valuable because humans remain extremely good at spotting odd behavior.

A dashboard might show that onboarding completion stayed at 81%. A linked production view might reveal that half the page looks wrong, the copy makes no sense, or the agent interpreted a product requirement in a bizarrely literal way.

The watchdog compresses thousands of events into one decision surface:

• What happened?
• Which events deserve attention?
• Where can I inspect them?
• Which issue should become an agent task?

That is much closer to a chief of staff than a traditional log viewer.

Automation three: Grace’s self-improvement loop

At (32:57), Carson explains how Grace grades and improves herself. The system reviews yesterday’s conversations, scores them against a rubric, ranks the failures, and starts agents to address the highest-signal problems.

Grace is Untangle’s AI paralegal. She talks with attorneys, paralegals, and people moving through a divorce.

Every day, another automation:

1. Pulls the previous day’s conversations.
2. Grades each conversation against a rubric.
3. Flags failures.
4. Ranks the highest-signal problems.
5. Starts a child session to address them.
6. Prepares fixes for Carson to review.

Carson says this loop produces roughly three improvements per day.

The strongest part of this system is the rubric.

The automation is evaluating whether each chat met a definition of quality.

That definition might include:

• Did Grace identify the client’s actual question?
• Did she request the necessary information?
• Did she avoid unsupported legal claims?
• Did she explain the next step clearly?
• Did she hand the conversation to a human when needed?
• Did the user reach the intended outcome?

The slide summarizes the compounding mechanism perfectly:

“Tomorrow’s grade includes today’s fix.”

The product improves against the same rubric every day. Small failures become changes while they are still small.

The reusable automation architecture

Carson’s three examples look different, but their machinery is almost identical.

1. Trigger

Start the workflow on a schedule or event.

Examples:

• Every weekday at 9 a.m.
• After a deployment
• When CI fails
• When a support ticket receives a certain label
• After 100 new customer conversations

2. Inspect

Give the agent access to the relevant evidence.

That might include:

• Browser sessions
• Database events
• Application logs
• Customer conversations
• Support tickets
• Pull requests
• Test output

3. Evaluate

Define what good and bad look like.

A useful rubric should be specific enough that the agent can distinguish a true failure from a merely unusual result.

4. Summarize

Produce one short report that a human will actually read.

Carson emphasizes that each automation ends with a summary, rather than another dashboard he must remember to visit.

5. Triage

When something fails, create a child agent to investigate the cause.

6. Repair and verify

The child agent proposes a fix, reruns the relevant test, and returns evidence.

7. Escalate the decision

Send the human:

• The problem
• The likely cause
• The proposed change
• The test evidence
• The pull request
• The remaining uncertainty

That structure can be built in Devin, Claude Code, Codex, Amp, Factory, Cursor, or a custom orchestration system. The product changes. The loop stays recognizable.

Security: keep production power away from the agents

Carson keeps production write credentials in 1Password and does not permanently expose them to his agents.

When an agent truly needs production access, he manually retrieves the credential and provides it within the relevant session. That creates an intentional moment where the human understands what is about to happen.

His rule is simple:

Agents should not quietly inherit the power to alter production.

This matters more as automations begin reacting to external inputs.

A support ticket, GitHub issue, customer chat, or Slack message may contain malicious instructions. An agent with broad credentials can interpret untrusted text as a command.

Use:

• Short-lived credentials
• Least-privilege permissions
• Read-only access where possible
• Separate development and production environments
• Human approval for destructive actions
• Audit logs
• Explicit allowlists
• Isolated agent environments

Carson’s future software factory writes and reviews increasing amounts of code. His present-day system still keeps a human hand near the production keys.

The $20K token bill and the routing lesson

At (34:57), Carson shares the number that forced him to redesign the factory: roughly $20,000 in tokens during one month.

His slide calls it “the wake-up call.” The routing system came after the bill.

By (35:48), Carson has arrived at the rule: route by task, rather than habit.

• Cheap fine-tuned models handle repetitive loop work.
• Independent agent platforms handle long-horizon engineering.
• Subsidized apps handle everyday tasks.
• Premium models remain available for the hardest judgment calls.

His estimate is that serious AI engineering may settle near $5,000 per employee per month, although that is Carson’s operating estimate rather than an established industry average.

The broader lesson is more durable: route each task to the least expensive model capable of performing it reliably.

Carson’s stack separates work into three levels.

Cheap models for high-volume loops

Recurring grading, classification, routine testing, and straightforward fixes happen constantly.

Carson uses Cognition’s less expensive SWE model for much of that work. He estimates some sessions cost around five dollars each.

Expensive models for consequential decisions

Architecture, ambiguous bugs, migrations, and difficult planning receive stronger models with higher reasoning effort.

A premium model can also act as the parent that scopes the project and delegates simpler tasks to cheaper child agents.

Independent platforms for routing

Carson prefers independent agent platforms because they can route work among OpenAI, Anthropic, open models, and their own specialized systems.

His hypothetical stack might use a premium OpenAI model for one difficult task, a cheaper coding model for implementation, and Fable to review the result.

That resembles our broader advice on model selection: the model sets the capability ceiling, while the effort and routing policy determine how much compute you spend reaching it.

What the coding tools cost

Pricing checked July 24, 2026. Agent pricing changes frequently, so use the linked pages as the final source before purchasing.

Devin

Devin is Carson’s primary cloud-agent platform.

Current self-serve pricing:

• Free: limited individual usage
• Pro: $20/month
• Max: $200/month
• Teams: $80/month minimum
• Full team seats: $40/month each
• Flex seats: no fixed seat cost, but they consume shared credits
• Enterprise: custom Agent Compute Unit pricing

Usage beyond included quotas draws from on-demand credits. See Devin’s billing documentation.

Claude Code

Claude Code is included with individual Claude subscriptions:

• Pro: $20/month, or $17 monthly when billed annually
• Max 5x: $100/month
• Max 20x: $200/month
• Team and Enterprise: Claude Code usage is billed separately through the Anthropic Console

Anthropic positions Pro for lighter coding, Max 5x for frequent use, and Max 20x for power users working across larger repositories. See Anthropic’s pricing page.

OpenAI Codex

Codex can work locally through its app, IDE integration, and CLI, or accept delegated cloud tasks.

At publication, Codex is included across eligible ChatGPT plans, with usage varying by plan:

• ChatGPT Plus: $20/month
• ChatGPT Pro: $200/month
• Business and Enterprise: included allowances plus workspace credit options
• Additional Codex credits can be purchased after plan limits

For more detail, see our complete guide to the Codex agent command center.

Cursor

Cursor combines an AI editor with local and cloud agents.

Current pricing begins at:

• Hobby: free
• Individual Pro: $20/month
• Teams: $40 per user each month
• Enterprise: custom pricing
• Additional model and agent usage: usage-based

See Cursor’s pricing page.

Amp

Amp runs in the terminal, editor, browser, phone, local machines, and remote orbs.

Current plans include:

• Megawatt: $20/month
• Gigawatt: $200/month
• Pay as you go: usage-based
• Enterprise: custom terms

See Amp’s manual for current details.

Factory

Factory offers local and cloud “Droids,” model routing, background agents, and managed remote computers.

Current individual plans:

• Pro: $20/month
• Plus: $100/month
• Max: $200/month
• Teams and Enterprise: custom pricing
• Extra usage: prepaid credits

See Factory’s pricing documentation.

Which tool should you actually use?

Carson’s strongest opinion is that a company with product-market fit should eventually use an independent software-factory platform rather than building its entire engineering workflow inside one model provider.

That does not make Devin the automatic answer for everyone.

Use Claude Code or Codex locally when:

• You are learning.
• You are building your first product.
• You need tight control over front-end work.
• One or two agents cover your workload.
• You are still discovering your development process.
• Your existing subscription provides enough usage.

Add Cursor when:

• You want agent features inside a familiar code editor.
• You rely heavily on interactive editing and autocomplete.
• You want local work and cloud agents in one interface.
• Your team needs shared rules and administrative controls.

Consider Devin, Amp, or Factory when:

• Several independent jobs should run in parallel.
• Agents need persistent or isolated compute.
• You want work to continue after the laptop closes.
• Browser testing is central to the product.
• Automations need schedules, events, or remote environments.
• Model routing and cost controls have become engineering concerns.
• Your company’s output is constrained by local coordination.

The expensive mistake would be building Carson’s entire factory before you have enough work to feed it.

Start with the bottleneck you actually have.

The counterargument: more pull requests can produce more review debt

Carson’s daily output sounds extraordinary. Pull-request volume remains an imperfect proxy for product progress.

Twenty weak changes can create more work than one correct decision. Ten parallel agents can generate ten subtly incompatible interpretations of the same architecture.

Research on developer productivity has also produced mixed results.

A 2025 randomized METR study found that experienced open-source developers using early-2025 AI tools took 19% longer on realistic repository tasks, even though they believed AI had made them faster. Read METR’s study.

METR’s 2026 follow-up suggested newer agents were probably producing more speedup, but selection effects made the updated estimate unreliable. Many developers declined to participate because they no longer wanted to work without AI. Read the follow-up.

A broader 2026 meta-analysis found a moderate positive productivity effect across 23 studies, with substantial variation by developer, task, and setting. The gains were smaller in open-source and enterprise environments than in controlled experiments. Read the paper.

All three results can be true.

AI can accelerate a clear greenfield task, slow down an expert inside a mature codebase, and create dramatic leverage when embedded inside a carefully engineered factory.

Carson has spent months building:

• Reproducible environments
• Browser tests
• Rubrics
• Production monitoring
• Notification systems
• Security boundaries
• Model-routing policies
• Review habits
• A codebase agents can navigate

Copying his number of agents without copying that machinery would reproduce the activity, not the outcome.

Carson’s own safeguards make the same point. He withholds production credentials, uses Untangle himself, and admits that even strong agents sometimes produce outcomes that make a human ask, “Why did you think that was okay?”

The software factory removes the typing bottleneck. It does not remove accountability.

Ryan Carson’s four-week advanced AI engineering plan

You do not need ten agents tomorrow.

Build the factory one layer at a time.

Week one: make one environment reproducible

Choose one repository and document:

• How to install dependencies
• How to start the application
• How to run tests
• How to seed development data
• Which credentials are safe
• Which credentials remain unavailable
• What “done” looks like for a code change

Place these instructions in your repository through an AGENTS.md, setup script, playbook, or equivalent configuration.

Your goal is simple: a fresh environment should become productive without you manually repairing it.

Week two: run three isolated jobs

Choose three small tasks that do not overlap.

Assign each task to a separate environment. Require every agent to return:

• A short summary
• The proposed diff
• Test results
• Remaining risks
• Screenshots or video for visible changes
• A pull request

Review the results at scheduled intervals instead of watching continuously.

Do not optimize for maximum output yet. Learn what information you need before confidently approving an agent’s work.

Week three: automate one recurring check

Choose a task you currently perform every week.

Good candidates include:

• Testing signup
• Checking important database jobs
• Reviewing failed CI runs
• Summarizing support tickets
• Testing checkout
• Inspecting dependency updates
• Reviewing slow queries
• Checking broken links

Build this sequence:

1. Run the check.
2. Record the evidence.
3. Summarize the result.
4. Notify you in a channel you already monitor.
5. Start a triage agent when the check fails.
6. Require human approval before production changes.

The final report should take less than two minutes to understand.

Week four: add a rubric and model routing

Define how the automation judges quality.

Then divide work into:

• Cheap, repetitive tasks
• Standard implementation tasks
• Expensive, high-stakes decisions

Record the cost of each run. Track how many findings become useful fixes.

A loop that costs $15 per day and prevents a serious customer failure may be excellent. A loop that generates five meaningless pull requests per morning is an automated confetti cannon.

Where this goes next

At (41:18), Carson gives Greg the destination: a software factory where agents eventually write, review, and ship 100% of the code.

The current version still depends heavily on its human manager. Carson chooses the priorities, defines the rubrics, protects the production credentials, notices when an experience feels wrong, and decides which pull requests deserve to survive.

The trajectory is visible already.

Agents can work inside isolated computers, inspect browser interfaces, respond to schedules, wake when external events arrive, start child agents, and return with pull requests. The human role is moving upward from implementation toward architecture, prioritization, verification, security, and product judgment.

That makes Carson’s advanced guide a direct sequel to his first one.

Chapter one taught people to build with an agent.

Chapter two teaches them to build an organization of agents.

The unresolved problem is whether human understanding can scale as quickly as agent output. When one person reviews 40 pull requests a day, the factory may be producing software faster than its operator can maintain a coherent mental model of the system.

Cloud agents can create nearly unlimited parallel work.

The next engineering breakthrough may be a reliable way to decide which of that work deserves to exist.

We just loved this episode, guys. Thanks for creating this, Greg; we learned so much!