Git Worktrees for Parallel AI Coding Agents: Setup, Failure Modes, and When to Automate

If you want two AI coding agents working on the same repository at the same time, the first thing you have to solve isn't prompting — it's that they'd both be editing the same working directory. git worktree is the cleanest answer, and it's built into git. This post shows the manual setup end to end, then the failure modes we hit running it under real load, because most worktree tutorials stop right before the interesting part.

Why worktrees and not the alternatives

Three options, and the trade-off is disk versus isolation:

• Branches alone. One working directory, one checkout. Two agents share files; the second one's edits land on top of the first one's half-finished work. This doesn't isolate anything.
• Full clones. Real isolation, but you pay for a complete copy of history per agent, and pushing work between clones is awkward.
• Worktrees. Multiple working directories backed by one .git object store. Each has its own checked-out branch, its own files, its own dirty state. Cheap, and branches are immediately visible to each other because there's only one repository.

Worktrees are the right primitive. That's the easy part.

The manual setup

Start from a clean repo and give each agent its own directory and branch:

# One worktree per agent, each on a fresh branch
git worktree add -b agent/auth    ../wt-auth    main
git worktree add -b agent/billing ../wt-billing main
git worktree add -b agent/tests   ../wt-tests   main

git worktree list
# /repo          a1b2c3d [main]
# /wt-auth       a1b2c3d [agent/auth]
# /wt-billing    a1b2c3d [agent/billing]
# /wt-tests      a1b2c3d [agent/tests]

If you keep worktrees inside the repo (convenient — one directory to clean up), exclude them or every git status in your main tree will be a wall of untracked files:

mkdir -p .worktrees
echo '.worktrees/' >> .gitignore
git worktree add -b agent/auth .worktrees/auth main

Port and database isolation

Filesystem isolation is not runtime isolation. Three agents that each run npm run dev will fight over port 3000, and three test suites pointed at the same database will destroy each other's fixtures. Give every worktree its own environment:

# .worktrees/auth/.env.local
PORT=3001
DATABASE_URL=postgres://localhost:5432/app_auth
REDIS_URL=redis://localhost:6379/1

# Create a scratch database per worktree
for wt in auth billing tests; do
  createdb "app_$wt" -T app_template
done

If you use Docker Compose, the project name is what keeps container names, networks and volumes from colliding:

cd .worktrees/auth
COMPOSE_PROJECT_NAME=auth docker compose up -d
# containers become auth-db-1, auth-redis-1, … instead of app-db-1

Merging back and cleaning up

# Integrate one agent's branch
git -C /repo merge --no-ff agent/auth -m "Merge agent/auth"

# Remove the worktree and its branch together
git worktree remove .worktrees/auth
git branch -d agent/auth

# After a crash left directories behind
git worktree prune

That's the whole happy path, and for two or three agents on a laptop it genuinely works. Now the part the tutorials skip.

Where worktrees fall apart

Every one of these bit us in production. They're ordered roughly by how long each took to diagnose.

1. .git is a file, and it holds an absolute path

Inside a worktree, .git isn't a directory — it's a one-line file:

cat .worktrees/auth/.git
# gitdir: /repo/.git/worktrees/auth

That path is absolute. Move the repo, mount the worktree into a container at a different path, or run the agent in a sandbox that doesn't have /repo — and every git command inside the worktree fails with not a git repository. Agents handle this badly: they don't report a broken environment, they just keep trying and burn their turns.

The fix is to make the path true wherever the agent runs. We mount the main repo's .git directory into the container at the identical host path, so the pointer resolves:

docker run \
  -v /repo/.worktrees/auth:/workspace \
  -v /repo/.git:/repo/.git \
  agent-image

2. The index lock is a global, and concurrency finds it

Worktrees isolate working directories, not the object store. Concurrent git writes against the same repository race on .git/index.lock and on ref updates, and you get intermittent, maddening Unable to create '.git/index.lock': File exists. Two agents finishing within the same second is enough.

We serialise every repository-level git operation behind a single mutex — worktree creation, merges, branch deletion, pruning. It costs a few milliseconds and removes an entire class of flake.

3. Removing a worktree deletes the branch with it

This is the one that actually loses work:

git worktree remove .worktrees/auth --force
git branch -D agent/auth          # ← the work is now unreachable

Run that as routine cleanup and any commit that hadn't been merged yet is gone. Worse, git's error message when you later try to merge the vanished branch is not something we can merge — which tells you nothing about what happened.

We now check the branch actually resolves before merging, and report a specific, actionable failure instead of git's version:

git rev-parse --verify --quiet "agent/auth^{commit}" || echo "branch is gone"

4. Dubious ownership when the runner isn't the file owner

If the process running git isn't the user that owns the files — a service account, a container UID, anything under systemd — git refuses to operate:

fatal: detected dubious ownership in repository at '/repo'

Every git invocation in our server passes -c safe.directory=* because the server manages every path in the workspace. Do this deliberately and scope it; it's a real check, not noise.

5. Disk grows faster than you think

A worktree shares history but checks out a full copy of the tree. Five agents on a repo with a 400MB working tree is 2GB of checkouts, plus five sets of node_modules if your agents install dependencies. On a build machine that fills a disk quietly, and the first symptom is an unrelated job failing.

6. Worktrees are local to one machine

The moment you have more than one runner, worktrees stop being a coordination mechanism. A worktree created on node A does not exist on node B — its directory isn't there and neither is its branch until someone pushes. Any scheduler that can place two tasks for the same repo on different machines has to route them to the same node or push branches to a shared remote. We route by project.

When to automate

Manual worktrees are the right call when you're driving the agents yourself: two or three of them, one repo, one machine, you watching. The setup above is maybe twenty minutes of work and you'll understand your own pipeline completely.

Automate when you cross one of these lines:

• More than about three concurrent agents, where lock contention and cleanup stop being occasional.
• Unattended runs, where nobody is watching to notice a broken .git pointer or a filled disk.
• Agents deciding their own file scope, which needs ownership arbitration that git has no concept of — git stops textual conflicts, not two agents independently rewriting the same handler.
• More than one machine, where local worktrees stop meaning anything.

Factory Nexus does exactly what's described above, hardened: a worktree per task at .factory/worktrees/task-<id> on a swarm/<id> branch, forked from an integration branch rather than main, one mutex around repository-level git, a pre-merge branch-existence check, .gitignore maintained automatically, and safe.directory set on every invocation. On top of that it adds the parts git can't provide — file-lane ownership per task, AI code review on every branch before it merges, and an integration pass over the combined result.

If you'd rather own the plumbing, own it — the commands above are the whole mechanism, and there's nothing proprietary in them.

Related: what agent swarm coding actually is, and how the swarm looks in the UI once the worktrees are someone else's problem.