Omnimancer: The Open-Source Multi-Model Coding Agent Behind Factory Nexus Swarms

Every swarm task in Factory Nexus runs on one of two coding agents. The first is Claude Code. The second is Omnimancer, an open-source, MIT-licensed coding agent we built so a swarm could run on models that Claude Code can't reach: OpenRouter's catalog, DigitalOcean's serverless inference, a vLLM box in a closet. This post is about what Omnimancer is, the contract it exposes to an orchestrator, how Factory drives it, and the five production failures that shaped its headless mode more than any design doc did.

What Omnimancer is

Omnimancer is a terminal coding agent that works like claude -p but isn't tied to one provider. Point it at Claude, OpenAI, Gemini, Bedrock, Ollama, OpenRouter, DigitalOcean inference, or any OpenAI-compatible endpoint, and you get an agent that reads files, writes code, runs commands, and iterates until the task is done, with streaming output, token and cost tracking, and structured JSON for pipelines.

pip install omnimancer-cli

# Single prompt, like claude -p
omn -p "refactor auth.py to use dependency injection"

# Pick the backend per run; flags go after the prompt
omn -p "write tests for src/api/routes.py" --provider openrouter --model z-ai/glm-5.2
omn -p "explain this repo" --provider digitalocean --model alibaba-qwen3-32b

# Pipe context in, get JSON out
cat error.log | omn -p "diagnose this crash" --output-format json

Beyond the loop itself, it has the pieces you'd expect from a serious agent runtime: MCP support over stdio, SSE, and streamable HTTP; lifecycle hooks that can veto a tool call; declarative allow/deny/ask permission rules with regex matchers; scoped subagents with their own prompt, tool whitelist, and model; and named endpoint aliases so several self-hosted OpenAI-compatible servers can sit side by side as first-class providers. Providers with native tool calling use structured function calls. Providers without it fall back to operation markers parsed out of the response text, so a model with no tool API can still edit files.

Interactive mode is a full REPL with /switch between providers mid-conversation, an approval workflow with diff previews, and /accept edits|all for Claude-Code-style auto-accept. But interactive mode isn't why Factory uses it. The headless contract is.

The headless contract

An orchestrator doesn't want a chat. It wants a process it can launch, watch, and reason about when it dies. Omnimancer's -p mode with --output-format stream-json gives it exactly that: one JSON object per line, in order, for the whole run.

{"type":"system","subtype":"init","model":"glm-5.2","session_id":"…"}
{"type":"assistant","message":{"content":"Reading the handler first.",…}}
{"type":"tool_use","tool":{"name":"file_read","arguments":{"path":"internal/api/export.go"}}}
{"type":"tool_result","tool":{"name":"file_read","content":"…","error":null}}
…
{"type":"result","subtype":"success","is_error":false,"stop_cause":"done",
 "num_turns":41,"usage":{"input_tokens":…,"cache_read_input_tokens":…},
 "total_cost_usd":0.31,"session_id":"…"}

The important field is stop_cause. A run that exits 0 and a run that did what you asked are not the same thing, and the first version of Omnimancer conflated them (more on that below). Today the result says why the loop ended:

The exit code carries the same information for anything that only reads exit codes:

Exit 3 and exit 4 are the ones an orchestrator should care about. Both mean "there is a checkpoint on disk and a --resume <session_id> picks up where this left off." Neither means "start over."

How Factory drives it

When you pick Omnimancer in the Agent Panel (or an admin has made it the default coding agent), each swarm task spawns a container from our factory-agent-omnimancer image: a python:3.11-slim base with omnimancer-cli pinned at a specific version, plus the same Node 20, Go, Rust, JDK, and Python toolchain the Claude Code image carries. The worktree is mounted at /workspace, the container runs with all capabilities dropped and no new privileges, and the command is:

omn -p "<role prompt>\n\n<task prompt>" \
    --provider digitalocean --model alibaba-qwen3-32b \
    --output-format stream-json --dangerously-skip-permissions

Three things about that command are worth knowing.

The provider comes from the model catalog, not the user. Factory's admin model catalog stores each model as {id, provider}. When a task names a model, the swarm coordinator resolves the provider from that binding and maps it onto Omnimancer's provider name (anthropic becomes claude, do_inference becomes digitalocean). The same catalog drives the gateway routing for Claude Code, so one table answers "which backend serves this model" for both runtimes. If the provider's key isn't configured, task creation fails with a 400 before any container starts, instead of spawning a task that dies on its first request.

The API key never appears on the command line. It's written to a 0600 env file, passed with --env-file, and deleted seconds after the container starts. Environment overrides are one of Omnimancer's design rules: env vars beat the saved config, apply at runtime only, and are never written back to disk.

Factory sizes the run through env vars. The coordinator injects a handful of OMNIMANCER_* variables that exist because of the incidents in the next section:

Every stream-json line goes through the same reader that handles Claude Code, with a per-runtime parser picked once when the agent starts. Text becomes the live activity feed on the task card. tool_use and tool_result become the tool timeline. The final result is persisted on the task: exit code, tokens in and out, cache reads, turns, and dollar cost. Those numbers roll up into the Runtime Parity table on the Agents page, which is how we answer "does this model actually finish work?" with counts rather than anecdotes.

Five things production taught us

Omnimancer had a working headless mode before Factory ever ran it. Almost none of that mode survived contact with a swarm. Each of these was found by watching a real orchestration fail, tracing container logs, and finding that the agent had done something defensible in a terminal and disastrous in a pipeline.

1. "I'll look into that" counted as done

The original loop ended the run on the first response with no tool calls. That's fine when the model answers a question. It's a disaster when the model narrates its plan ("Let me look into the export handler first...") and stops. GLM-5.2 narrates constantly. Those runs exited 0 with a clean result and zero file changes. Factory saw an empty diff, failed the task with no review, auto-retried into the identical dead end, and burned the retry budget with nothing visible in the UI to explain why.

The fix is a nudge. A tool-less turn now gets a follow-up: "If the task is fully complete, reply with exactly DONE. Otherwise continue working now by calling tools." The run ends only on an explicit DONE or after two consecutive tool-less turns, and stop_cause records which. The same change made the iteration cap configurable, because the default of 25 was ending genuine work mid-task, also with exit 0.

2. --dangerously-skip-permissions didn't actually skip permissions

The flag auto-approved the interactive approval prompt. Four other layers sat underneath it and kept denying: the command argument sanitizer rejected pipes, &&, redirects, and $(); the forbidden-command list blocked rm; the sensitive-filename patterns refused even to read a file named key_manager.go because it matched *key*; and a 30-second default command timeout killed any real test suite. In prod this looked like agents being permission-denied on some tasks, reporting success on others with no changes, and leaving stray swarm/* branches behind.

Headless runs with that flag now enable a full-trust mode that lifts those four layers and raises the command timeout to 600 seconds. The caller is the security boundary, which in Factory's case is a locked-down container with a single mounted worktree. Hard-restricted system paths (~/.ssh, /etc, ~/.aws/credentials) stay blocked even in full trust. The general lesson: an agent's safety layers should compose into one switch, or an orchestrator will find every layer the switch forgot.

3. Hardcoded timeouts versus 397-billion-parameter models

Omnimancer's chat calls had 30- and 60-second timeouts baked in. DigitalOcean's qwen3.5-397b routinely takes longer than 60 seconds to return its first byte on a non-streaming request. Every call timed out, the provider raised "Request to OpenAI API timed out", the process exited 1 with zero output, and Factory's retries hit the same wall three times per task.

Now the timeout is a per-provider setting with an OMNIMANCER_REQUEST_TIMEOUT override, defaulting to 120 seconds with one retry on timeout. Factory sets 300. The remaining timeouts we see today are on Factory's own review pool, not the coding agent, which is a different post.

4. The whole conversation, every iteration

An agent loop retransmits its full history on every provider call. Omnimancer inlined the project's CLAUDE.md into the system prompt at a 100 KB cap. Factory's own CLAUDE.md is 143 KB, about 25,000 tokens, sent on every one of up to 150 iterations. On top of that, every 16 KB tool result stayed in the history forever, so cost grew quadratically with the number of turns. We found this when a swarm blew through DigitalOcean's 800,000 tokens-per-minute limit.

Three changes. The tools-mode system prompt was cut to about a third of its size by dropping the sections that described the marker-based flow, which native tool calling never uses. Instruction-file inlining got the OMNIMANCER_INSTRUCTION_BYTES cap, which Factory sets to 16 KB because the task prompt already carries the context that matters. And the headless loop now elides stale tool results: it walks history newest-first, keeps results verbatim until the budget is spent, and replaces everything older with a stub telling the model to re-run the tool if it needs the output again. The newest batch is never elided. Prompt caching was wired up across providers in the same release, with cache_read_input_tokens reported back in usage so Factory can see what it's paying for.

5. A 429 threw away everything you'd paid for

A rate limit used to be terminal: the provider raised, the process died, and every token already spent on that session was gone. Factory caught the error, backed off, and respawned the agent from the original prompt in the same worktree. That works, but it re-buys the entire run-up, and during sustained contention on a popular model it exhausts five retries without finishing.

This is where the checkpoint in the diagram comes from. Omnimancer now retries 429s in-process with exponential backoff (five attempts, honoring Retry-After when the provider sends it). If that still fails, it saves a lossless checkpoint (messages, tool calls, tool results, usage, iteration index), emits an error event with stop_cause: rate_limited and a resume_session_id, and exits 4. Factory's retry path reads that exit code, waits a short backoff (Omnimancer already did the long one), and respawns with omn --resume <id> and no prompt at all. The conversation continues from the pending message. Exit 3 gets the same treatment with a doubled iteration cap. One deliberate exception: retries triggered by a review rejection never resume, because the reviewer's feedback has to reach the model as a fresh prompt.

That card shows the last piece: runtime failover. If an admin has configured a fallback chain (say, Omnimancer on DigitalOcean first, Claude Code second), infrastructure failures like rate limits, spawn errors, and agent deaths fail over automatically after two attempts on a rung, with the next runtime picking up the same worktree under a handoff prompt. Review rejections never trigger failover; those are the model's problem to fix, not the infrastructure's.

What Omnimancer is not

It is not a better coder than Claude Code. On the same task with the same model, we'd expect Claude Code to win, and the Runtime Parity table exists so that claim gets checked against completion counts rather than assumed. Omnimancer is the way to run a swarm on models Claude Code can't use: cheaper ones, open-weight ones, ones behind an endpoint you control. A 79-turn, 22-file-write task on GLM-5.2 merging cleanly through the three-reviewer gate was the proof we needed that the second runtime is real. On that same orchestration roughly 40% of first attempts still ended with no changes, and every one of them merged on retry. That gap between "the process exited" and "the work is done" is the reason stop_cause exists.

It also isn't done. The rate-limit fallback between providers inside a single run is opt-in and off by default in headless mode, because switching models mid-task silently is exactly the kind of thing an orchestrator wants to decide, not the agent. And Omnimancer has no --system-prompt flag, so Factory prepends the role prompt to -p; that's a wart we'll fix on the Omnimancer side.

Use it

Omnimancer is on PyPI as omnimancer-cli and on GitHub under MIT. It runs on its own with any of the providers above, and the headless contract described here is stable: stream-json events, stop_cause, the four exit codes, and --resume. If you're building your own orchestrator, that's the surface to build against. If you'd rather not, Factory Nexus runs it for you.