How real systems make many autonomous coding agents — from DIFFERENT model vendors — collaborate without conflicting

Version: 1.0.0 · Date: 2026-09-13 · Time: 04:55 EEST · Model: claude-opus-5 · Effort: high · Origin: DKserver
agent: claude_code (tier sub) · node: task:research.multi-vendor-agent-coordination-worldwide · landed: f988ad9b636e on origin/master

Nobody coordinates concurrent writers to one shared tree. Everybody isolates them and fights at recombination instead. MPRO already runs the winning pattern — a worktree lane per agent plus a broker — and the research says our topology is immune by construction to the worst anomaly class. We are ahead on the git seam and behind on exactly two cheap things. Both are now tracked tasks, not open questions.

What is left to do

Nothing technical. All six engineering findings are DECIDED or DELEGATED to a tracked task. No row waits on the CEO.

Money rows: none. Every recommendation uses tooling MPRO already pays for — git plumbing, taskman, the existing code graph. No new vendor, no new subscription, no spend. Treasury note: cost to act = agent time only (no euro); upside = stops paying super-linear coordination cost for sub-linear output; harm if wrong = one ignorable advisory or a few extra HTTP 409s; reversibility = every item ships advisory-first behind an existing warn valve; NET positive.

#Finding, already delegatedTracked nodeOwner · model
1Cap concurrent mutating lanes at the measured coordination kneetask:harness.cap-concurrent-mutating-lanes-at-measured-kneeclaude_code · sonnet
2Taskman status CAS covers the request window, not the generation windowproblem:taskman.status-cas-covers-the-request-window-not-the-generation-windowclaude_code · opus
3Partition work by dependency cohesion, not by filetask:harness.cohesion-aware-lane-advisory-from-the-code-graphclaude_code · opus then sonnet
4Measure and record per-vendor concurrency safetytask:harness.measure-and-record-per-vendor-concurrency-safetyclaude_code · sonnet
5Correct the two stale A2A agent cardstask:harness.correct-the-two-stale-a2a-agent-cardsclaude_code · haiku
6A curl-to-file wrote a silently truncated artifact (found while doing this research)problem:harness.curl-to-file-wrote-a-truncated-artifact-with-a-preview-footerclaude_code

The answer in five lines

  1. Isolate, then merge. The universal 2026 answer is worktree-per-agent plus gated integration — not a smarter lock. documented
  2. Topology decides, not the primitive. Of 16 canonical multi-agent topologies, 6 are structurally susceptible to the lost-update class (each shares a mutable channel inside one step) and 10 are immune by construction (disjoint channels, additive reducers, sequential staging). No primitive rescues a susceptible topology; picking an immune one is free. documented
  3. Coordination is cheap, and everyone assumes it is not. Measured: serializable snapshot isolation ≈ 8% extra tokens; pessimistic locking 1.6–2.3×; wall-clock overhead of both sits at the inference-noise floor. documented
  4. More vendors is not more throughput past a measured knee. Quality peaks at N=2 agents, tests-passing peaks at N=3, message volume goes super-linear past N=3. MPRO runs 8 vendor CLIs. documented
  5. MPRO is ahead on the git seam and behind on two cheap things only: dependency-cohesion task partitioning, and a concurrency guard over the long generation window. measured here

The measured coordination knee — and where MPRO sits on it

N=1
solo
abandonment 13.7x

N=2
QUALITY PEAKS
0.669

N=3
TESTS PEAK ~57
quality regresses

N=4
messages super-linear
further decline

MPRO = 8 vendor CLIs
past the measured knee

Diagram 1. Agent-count scaling, redrawn from AgentRoom Fig. 5 (arXiv:2608.23740, 2026-08-24). Solo agents abandon tasks at 13.7× the odds of coordinated ones (95% CI [3.9, 48], p<10-5, pooled over 12 model×task strata) — so one agent is the wrong answer too. The sweet spot is narrow.
 N=1 solo        N=2                N=3                N=4            MPRO today
 +-----------+   +--------------+   +--------------+   +-----------+  +--------------+
 | abandons  |-->| QUALITY PEAK |-->| TESTS PEAK   |-->| messages  |->| 8 vendor CLIs|
 | 13.7x odds|   | 0.669        |   | ~57 passing  |   | SUPER-    |  | past the     |
 |           |   |              |   | quality dips |   | LINEAR    |  | measured knee|
 +-----------+   +--------------+   +--------------+   +-----------+  +--------------+

The primitive-to-conflict-class map

The core deliverable. A lease is not a merge strategy; a queue is not a lock. Read each row as: this failure is fixed by this primitive, and the primitive charges this.

#Conflict classPrimitive that actually fixes itThe cost it imposesSource
C1Lost update / stale-generation — read at t0, generate for minutes, commit over a peer's newer writePessimistic lock held across the whole operation, or serializable snapshot isolation validating the read set at commitlocking: 1.6–2.3× tokens, 40.7% of operations dropped. SSI: ≈8% tokens, 20.7% aborts, plus a residual 3% leak on one workloadKhan 2026-06-15
C2Write-write on one fileAdvisory file claim, Chubby-style: claim(path) atomically assigns ownership and rejects a second holdera claim round-trip plus “revise and pick a different file”. Enforcement sits at the prompt layer, so an agent that skips the claim still writesAgentRoom 2026-08-24; Chubby 2006
C3Phantom-tool — the planned tool vanishes from the registry mid-generationTool registry snapshot pinned for the operation's lifetimeone snapshot per operation. SSI does NOT fix this — its scope is value memory, not the tool registry: measured 21.1% unchanged under SSIKhan 2026-06-15
C4Causal-cascade — a surviving commit keeps an aborted operation in its causal closureCausal-closure tracking with compensation (an abort propagates to dependents)a predecessor set per operation; retraction cascades. Prevented in all 120 retracted sessions across 3 model familiesKhan 2026-06-15
C5Tool-effect reordering — side effects externalised in an order the agent did not intendCommit-order sequencer serialising a turn's side effectsserialises what was parallel. Real defect: LangGraph ToolNode dispatches a turn's calls via asyncio.gather, which orders results but not effectsKhan 2026-06-15
C6Index / HEAD lock race — .git/index.lock: File exists, cannot lock ref HEADPer-worktree index (worktree-per-agent). Not a better global lock — a global HEAD lock cannot be tuned into being instantdisk per worktree (one report: 9.82 GB from a 20-minute session on a ~2 GB repo); environment setup per treeZylos 2026-02-22
C7Semantic conflict / merge skew — merges textually clean, breaks at build or test timeMerge queue that tests the COMBINED result: either require the branch to already contain the base, or speculatively merge the queueup-to-dateness forces a re-sync race when the base moves; speculative batching costs CI. CRDTs do not fix this — byte convergence is not intent convergenceModern Git Academy; Mergify
C8Peer-blast — one agent's bad file blocks every other agent's commitCell / bulkhead isolation: scope every gate to the calling agent's own file set; share nothing between cells except the routera per-agent gate run instead of one shared run; cross-cell work needs deliberate external coordinationAWS REL10-BP04
C9Duplicated workContract-net: announce → bid → award, one awardee per task; plus duplicate-by-meaning refusal at registrationmessage volume grows with node count — contract-net's known weakness — and it does not solve discovery of contractors at allFIPA SC00029; Smith 1980
C10Livelock / thundering herd on a shared lockLease with expiry and backoff, deliberately coarse-grained (minutes to hours), plus a lock-delay before re-acquisitionthroughput ceiling = the lock. Coarse grain is deliberate so clients are less exposed to the lock service's own crashesChubby 2006
C11Split-brain / stale lease holder — lease expired, new holder took over, the old holder (alive, just slow) still writesFencing token — Chubby's sequencer: an opaque string carrying name, mode and a lock generation number, presented on every write and validated by the resourceevery write path must carry and check the token. A lock without a fencing token does not prevent this.Chubby 2006; Hazelcast
C12Optimistic-concurrency conflict — HTTP 409, version mismatchCompare-and-set on a version plus a re-read-and-retry loop (level-triggered reconciliation): a failed write just means reconcile again from fresh statewasted work proportional to the conflict rate. Omega named the metric: conflict fraction = average conflicts per successful transaction (3 ⇒ four attempts)Kubernetes; Omega 2013
C13Optimism collapse under coupled (gang) workIncremental placement instead of all-or-nothing gang scheduling — or pessimistic partitioning for the coupled subset onlyincremental acquisition deadlocks without a backoff; hoarding wastes what it holds. Mesos' pessimistic partitioning deadlocks on gang-scheduled MPI jobs via resource hoardingOmega 2013
C14Vendor-fragile CLI under concurrency — the vendor's own CLI crashes when driven concurrentlyPer-vendor concurrency capability gate with an explicit exclusion bucketyou lose that vendor for concurrent work. Measured: Codex-mini and GPT-5.4-mini crash under concurrent MCP and were excluded from every cell; Gemini CLI 0.40+ rejects LSP-style Content-Length framingAgentRoom 2026-08-24
C15Context divergence — agents hold different interpretations of the same targetShared append-only broadcast log plus a read-before-write protocol (a blackboard with a control layer)Blackboard is one of the 6 structurally susceptible topologies — it buys shared understanding and adds C1 exposureKhan 2026-06-15; MAST
C16Coordination overhead — adding agents makes it slower, not fasterCap N, and partition by dependency cohesion rather than by fileknee: quality peaks N=2, tests N=3, messages super-linear past N=3. File-based splitting cost +44% to +60% for a 3.2% quality gainAgentRoom; Co-Coder
C17Agent abandonment — an agent silently gives up mid-taskPeer presence: a room carrying each agent's status plus a broadcast channelmeasured: solo abandonment odds 13.7× higher than coordinated (95% CI [3.9, 48], p<10-5)AgentRoom 2026-08-24
C18Crash of a worker mid-taskSupervision tree: restart per strategy (one_for_one / one_for_all / rest_for_one) with restart intensity as a circuit breaker — exceed it and the supervisor itself dies, escalating upwarda bad intensity (e.g. 10 per 1 s) lets a child restart forever and fills logs until a human intervenesErlang/OTP

The most important row is not a primitive at all

Topology beats primitive. A formally verified stale-generation detector was applied to 16 canonical multi-agent topologies. 6 are susceptible — hierarchical teams, collaboration, swarm, blackboard, competing-agents consensus, reducer-less map-reduce — each because it shares a mutable channel within one step. 10 are immune by construction — disjoint channels, additive reducers, or sequential staging. The dynamic check: every susceptible instantiation fired 20/20, every immune one 0/20, identically across three model families. (Khan, arXiv:2606.17182, 2026-06-15)

Corollary, and it is the cheapest finding here: MPRO's worktree-lane-per-agent is a disjoint-channel topology — immune by construction. Do not trade it for a shared workspace to gain “collaboration”.

L0 = { }
excludes nothing
vanilla: A1 20.7%, A2 21.1%

L1 = {A1}
read-set validation
pessimistic lock OR SSI

L2 = {A1,A3}
+ causal-closure tracking
abort propagates

L3 = {A1,A3,A6}
+ commit-order sequencer
serialise side effects

L4 = {A1,A2,A3,A6}
+ pinned tool registry
MAXIMAL

Diagram 2. The verified consistency hierarchy, redrawn from Khan Fig. 1 and Table 2. Each level excludes strictly more. A1 stale-generation, A2 phantom-tool, A3 causal-cascade, A6 tool-effect reordering. The full lattice has 16 points and 32 cover edges; this chain is 1 of 24 maximal chains. The practical message: you buy safety one level at a time, and you can stop where the cost stops being worth it.
  L4  {A1,A2,A3,A6}  + pinned tool registry            MAXIMAL
   ^
  L3  {A1,A3,A6}     + commit-order sequencer          serialise side effects
   ^
  L2  {A1,A3}        + causal-closure tracking          abort propagates
   ^
  L1  {A1}           read-set validation                pessimistic lock OR SSI
   ^
  L0  { }            excludes nothing                   vanilla: A1 20.7%, A2 21.1%

The three ways to schedule concurrent writers — settled in 2013

3. SHARED-STATE (Omega) - optimistic

conflict -> redo ONE claim

scheduler A

FULL cluster state
Paxos txn store

scheduler B

scheduler C

2. TWO-LEVEL (Mesos) - pessimistic

offer

offer

resource manager
OFFERS resources

framework A

framework B

cluster state

1. MONOLITHIC (one scheduler)

all jobs

single scheduler

cluster state

Diagram 3. Redrawn from Omega Fig. 1 (EuroSys 2013). Google rewrote Borg into the shared-state, optimistic form after experimenting with Omega. Their measured caution: the conflict fraction rises with scheduler count but stays compensated by better per-scheduler utilisation up to 32 schedulers — and optimism hurts most on coupled all-or-nothing work.
 1. MONOLITHIC          2. TWO-LEVEL (Mesos)        3. SHARED-STATE (Omega)
    all jobs               resource mgr OFFERS          sched A --.
       |                     |        |                 sched B --+--> FULL state
    one scheduler          fwk A    fwk B               sched C --'   (Paxos txn)
       |                     |        |                        conflict -> redo
    cluster state          cluster state                        ONE claim
  does not scale         pessimistic; can DEADLOCK      parallel; wasted work
                         on gang-scheduled jobs         instead of blocking

Where MPRO sits today

The two gaps

GAP 1: tasks partitioned by node,
not by dependency cohesion

GAP 2: CAS covers the REQUEST window,
not the GENERATION window

What MPRO runs today - verified 2026-09-13

base MUST be ancestor of branch
else current-integration-not-covered

peer_owned_branch_refusal
-> land / cherry-land

lane A worktree
own index

agent-git broker
fleet push-lock

lane B worktree
own index

lane C worktree
own index

merge-queue.js
merge-tree + commit-tree
NEVER a checkout

master

Diagram 4. MPRO's live path, read from source on 2026-09-13. Every box was verified by reading the file, not from memory. The two gaps are the only items in the delegated list that change architecture.
 lane A (own index) --.
 lane B (own index) --+--> agent-git broker --> merge-queue.js --> master
 lane C (own index) --'    (fleet push-lock)    merge-tree +
                                                commit-tree,
       peer_owned_branch_refusal                NEVER a checkout
             -> land / cherry-land              base MUST be ancestor
                                                of the branch, else
                                                "current-integration-
                                                 not-covered"
 GAP 1: tasks partitioned by node, not by dependency cohesion
 GAP 2: CAS covers the REQUEST window, not the GENERATION window

Already covered by MPRO — do not re-buy

Each row names our own file or service, verified by reading it this session. This section exists so no future agent spends money re-inventing what we own.

Conflict classMPRO already hasWhere
C6 index/HEAD racePer-agent lane worktrees via agent-git lane --ensure (idempotent, add-only); a raw git worktree add is refused downstream as not_ownedtools/scripts/agent-git.js, tools/scripts/agent-git.sh
C2 write-writepeer_owned_branch_refusal — refuses to write onto a branch a peer owns and redirects to land / cherry-landtools/scripts/lib/agent-git-land.js
C7 merge skewThe merge queue refuses a branch that does not already contain the base: merge-base --is-ancestor, else unproven('current-integration-not-covered'). Evidence binds to the branch tip, so an up-to-date branch's evidence is about the combined contenttools/git/merge-queue.js:739-747
C7 pre-checkTextual conflict prediction with no checkout, via git merge-tree --write-tree plumbingtools/git/fleet-integrate-sweep.js:583-616,1051
C10 lock contentionFleet push lock with wait/stat accounting (LOCK_WAIT, LOCK_STATS_FILE, push-lock); pushes route through the broker so the lock is genuinely heldtools/scripts/agent-git.js
C1 partial / C12CAS on every taskman status flip, defaulting to version so a concurrent claim (which bumps version but not status) is caught → 0 rows → 409tools/mpro-taskman/lib/store.js:1358-1404
C1 partialFresh-notes ack: when a holder flips their own task done, changelog rows appended after assigned_at are fetched and gatedtools/mpro-taskman/lib/store.js:1367-1375
C10/C11 partial24 h atomic claim lease; 409 on a conflicting live lease; 400 claim_not_resumable without session/worktree/branch; stale-claim sweep back to open; claim_active live-lease flag; per-agent claim cap; takeover after the idle windowtools/mpro-taskman/, contract at GET :3018/llms.txt
C9 duplicated workDuplicate-by-meaning refusal (search before POST; same-content re-mint → 422) plus write-time binding enforcement that refuses an unbindable acceptance contract at write with the exact repairsame — measured: my own first POST this session was refused 422
C4 partialCompletion-impact regression guard: a done flip returns the related nodes to re-verify, so a completion that broke a dependent is caughtGET /api/node/<id>/impact
C8 peer-blastC79 peer-protection clause (supreme layer) plus the proven private-index procedure, so hooks and the secret scanner run against only the calling agent's file set.claude/rules/values-bhagavad-gita.md; problem:harness.multi-agent-git-collisions
C15/C9 messagingPostbox queue plus 6 running vendor bridges, amsg, and per-vendor TO_<AGENT>.md filespostbox/, mpro-*bridge.service
C16 topologyWorktree-lane-per-agent = a disjoint-channel topology, which the structural audit classes as immune by constructionas above
Prior decision“Isolate first, then merge” is already MPRO's stated position, and isolated-worktree-by-default is already Fix-first #1 with an ownerdocs/commit-speed-and-git-gaps/index.html v1.1.0, 2026-08-27
Prior adoption passAn external orchestration-discipline document was already assessed for adopt / already-had / rejecteddocs/agent-orchestration-adoption/index.html v1.1.0, 2026-08-18

Two corrections to the brief's own list of MPRO primitives

1. tools/eidos/session-scope-fence.js is RETIRED in production. tools/eidos/session-scope-retirement.js sets SESSION_SCOPE_FENCE_RETIRED = true as an explicit compile-time production decision, commented “No ambient environment variable can resurrect the retired production path.” The 7,679-line fence file still exists and still serves forensic/replay tests. Do not plan around it as a live guard. measured here

2. The A2A agent cards are A2A-shaped but not A2A-conformant, and not discoverable. .grok/agent-card.a2a.json declares a2a_version: "1.0" yet exposes endpoints: {amsg, file, postbox} — MPRO-internal transports — with no HTTP URL, no JSON-RPC endpoint, no skills, no securitySchemes. I probed four well-known paths on ports 3000, 3003 and 3018: all 12 returned 404. It also carries stale metadata (model: grok-4 against a policy pinning grok-4.6) and contradicts itself (hierarchy_level: L2 vs a description saying L3). measured here

One correction to the fleet figure

The brief cites “8 models across 7 vendors”. The authoritative sources say: 8 vendor CLIs (VENDOR_SESSION_MARKER_TRUTH — claude, codex, kimi, grok, opencode, qwen, antigravity, ollama), of which 6 are gate-hooked (vendor-gate-manifest.json), and the dispatch policy pins 5 distinct strong models (claude-opus-5, gpt-5.6-sol, grok-4.6, kimi-code/k3, zai-coding-plan/glm-5.3) with qwen, ollama and antigravity carrying no pinned model. The order of magnitude was right; the shape is 8 / 6 / 5. This matters because the measured coordination knee is at N=2–3. measured here

What to adopt, ranked, with the honest net

A1 — Cap concurrent mutating agents at the measured knee NET: strongly positive

The knee is published and independently corroborated (practitioners report a 3–5 ceiling). MPRO has 8 vendor CLIs and no written cap on concurrent mutating lanes. Value if right: stops paying super-linear coordination cost for sub-linear output — the largest measured waste in this document. Cost if wrong: we under-parallelise and some work queues — bounded, visible, and reversible by changing one number. First step: record a cap for mutating lanes only (read-only and research agents are not capped) in the canonical C87 body, and have the agent-git lane advisory name it, reading the live count from git worktree list via tools/git/lane-roots.js. → task:harness.cap-concurrent-mutating-lanes-at-measured-knee

A2 — Close the generation window on taskman status flips NET: positive, cheap

store.js:1400-1404 compares against the version read inside the same request. That closes a millisecond race. It does not close the read→generate→write window: an agent that read a node 40 minutes ago and posts done without expected_version compares against the version as it is now and succeeds. That is precisely the formal definition of stale-generation. The contract itself calls expected_version “Optional”. Cost if wrong: more 409s, which the contract already tells every caller to handle by re-read-and-retry. Risk, named: a fail-closed flip on this seam would block every vendor's completion path — so it ships advisory-first behind a warn valve, exactly as two existing taskman valves already do. First step: surface the version the claim already mints, then warn (never refuse) on a done flip with no expected_version from a holder whose claim predates a later version bump. → problem:taskman.status-cas-covers-the-request-window-not-the-generation-window

A3 — Partition work by dependency cohesion, not by file NET: positive, input already exists

Measured against a sequential baseline: cohesion-aware partitioning gave 68.1% pass, −45% latency, −28% cost on one benchmark and 34.1% pass, −52% latency, −35% cost on a dependency-dense one, while naive file-based splitting cost +44% and +60% for a 3.2% quality gain. Benefit scales with dependency edge density (Pearson r = 0.65). MPRO already maintains the input: the code index, find_importers, get_dependency_graph, get_blast_radius. Cost if wrong: one ignorable suggestion — it ships advisory-only, never a dispatch refusal. → task:harness.cohesion-aware-lane-advisory-from-the-code-graph

A4 — Gate vendors on concurrency fragility NET: positive

Two OpenAI CLIs crash under concurrent MCP and were excluded from every published experiment cell; one Google CLI rejected a framing style until fixed. MPRO drives 8 vendor CLIs concurrently against shared MCP servers — and the MCP spec does not define isolation boundaries between servers in a multi-server client, with per-session state isolation still an open clarification request. Today a vendor that dies under concurrency looks like a task failure and gets re-dispatched, burning tokens on a structural incapacity. First step: add a measured concurrencySafe field per vendor to the dispatch capability map, starting with the three vendors that carry no pinned model. → task:harness.measure-and-record-per-vendor-concurrency-safety

A5 — Fencing token on the claim lease: DECIDED NOT TO BUILD NOW

MPRO's claim lease has no fencing token (grepped, no hits). The classic lesson says a lease without one cannot stop a stale holder writing. But the exposure is largely already closed by the version-CAS: a takeover bumps the version, so a resurrected holder's flip hits a zero-row compare-and-set and gets a 409. The residual is narrow, while the cost of acting is not — every write path would have to carry and validate a token, the kind of cross-cutting change that wedges a P0 service. Decision (mine as CTO-delegate, not a CEO question): do not build it now. Ship A2 first, then re-measure. If A2's warn counter ever records a real stale-holder flip, that measurement is the honest trigger to revisit. Recorded as the follow-up condition on the A2 node — no separate node, because a node for work we decided against is backlog dirt.

Explicitly NOT recommended — and this is the valuable half

RejectedWhy not, with the evidence
CRDT shared workspace for codeIt converges bytes, not intent — its own authors: collisions merged at byte level “typically break compilation or tests”. The 0%-semantic-conflict headline comes from its file-claim layer, not the CRDT. MPRO already has a stronger claim layer. Adopting it would add a merge engine that cannot catch what we actually fear.
Blackboard / shared mutable workspaceOne of the 6 structurally susceptible topologies. MPRO's disjoint-channel lanes are immune by construction. Trading immunity for “collaboration” is a strict downgrade.
A better global HEAD lockAlready rejected by MPRO on 2026-08-27, and independently by Omega: pessimistic two-level locking can deadlock on gang-scheduled work. A global lock cannot be tuned into being instant.
Full A2A conformance nowThe card is not served and nothing external calls MPRO. A2A's value is cross-organisation interop; our 8 vendors already share a filesystem, a board and a push broker — strictly more coupling than A2A provides. The spec is also “mostly informally specified”, its state machine described in prose. But do fix the cards' stale metadata — delegated above.
Contract-net bidding for task allocationMPRO's taskman claim is already an award-without-auction, and contract-net's known weakness is message volume growing with node count — the exact cost A1 says we already over-pay. It also does not solve contractor discovery.
Speculative merge batchingOur merge queue already refuses a branch that does not contain the base, which closes merge-skew for the landing branch. Batching buys CI throughput we do not need at this scale and costs a bisect-on-failure mechanism we would have to build. Revisit only if landing throughput becomes the bottleneck.

What actually goes wrong — the failure literature, mechanism-named

FailureThe real mechanism, and the evidenceKind
Agents proceed on corrupted dataAbsent failure signal — “agents tend to proceed silently on corrupted data rather than surfacing exceptions”reported
Neither agent knows the other existsNo mutual-awareness channel. Where two human engineers have a social mechanism to resolve a conflict deliberately, agent conflict resolution “optimizes for syntax, not intended product behavior”reported
Bugs surface lateBehavioural, not syntactic — they do not surface until a customer, QA run or demo reveals something missingreported
Hotspot files collideShared-mutable-hotspot contention — routing tables, configs, component registriesreported
Production multi-agent failure rate41%–87%, the majority attributable to coordination defects rather than base-model capabilitydocumented
Silent lost update in a shipped productReproduced live in ByteDance's deer-flow from the project's own regression test and an open issue; the fix formalised as a verified level upgradedocumented
Effect reordering in a shipped frameworkLangGraph ToolNode — asyncio.gather orders results but not side effectsdocumented
Concurrent git metadata corruptionRunning concurrent git commands across worktrees can corrupt shared metadata, so git operations should be serialised — which is exactly what MPRO's broker and push-lock doreported
The popular taxonomy does not cover thisMAST's 14 failure modes and the concurrency anomalies do not overlap. The verified detector fired 0 of 600 mapped MAST traces (95% CI [0, 0.61%]) — because that corpus mostly does not share mutable state concurrently. Reading MAST and concluding “concurrency is not our problem” is a population error. MPRO shares one repo, one board, one machine.documented

Where nobody has solved it — honest answers

  1. Semantic conflict between two independently-correct agent changes is unsolved. Every mechanism found is a detector (run the tests on the combined tree), never a preventer. No system reasons about intent.
  2. Worktree isolation solves filesystem isolation and nothing else. Ports, databases, .env, Docker daemon and caches stay shared — “no tool yet connects worktree code isolation with full environment isolation.” MPRO shares one machine and one service fleet, so this gap is ours directly.
  3. Cross-vendor heterogeneity has no established benefit, and the evidence is thin in both directions. One measured composite puts a Sonnet+Codex pair (0.721, n=3) above homogeneous Sonnet×2 (0.669, n=14), and Sonnet+Gemini (0.542, n=3) below it. At n=3 per heterogeneous cell, neither direction is established; the authors' own conclusion is that heterogeneity is “not a free lunch”. So: justify MPRO's multi-vendor fleet on cost (two cheap agents beat one expensive one at matched cost), on rate-limit resilience, and on genuine capability differences — never on a quality claim nobody has demonstrated.
  4. MCP does not define isolation between servers in a multi-server client, and per-session state isolation inside a server is left to implementers. Every multi-vendor harness sharing MCP servers, MPRO included, carries an undefined boundary.
  5. Nobody publishes a multi-vendor concurrency-safety matrix. The only measured instance found exists as a footnote excluding two models from an experiment. MPRO would be building A4 from scratch because nobody has built it.

Anomaly found while doing this research

Not part of the brief; investigated and recorded because a tool did not answer what it should.

A curl … > file in this session produced a silently TRUNCATED file with a fabricated footer. Fetching the taskman contract wrote 2,959 bytes / 32 lines, whose last line was the literal text ... (138 more lines, 16759 bytes total) — a read-preview footer written into the artifact on disk. The same URL fetched via a Node HTTP client gave 16,756 bytes / 169 lines. Five A/B replays of the identical command shape to fresh filenames all wrote the full file, so the trigger is not the command shape. Honest status: one-shot, real, unreproduced.

Why it matters beyond this session: an agent that downloads a spec, dataset or contract this way can get a corrupted artifact with no error and no non-zero exit — and the fabricated footer even looks like a legitimate truncation notice, so a careful reader concludes “that is all there is”. It is the same class as every failure in the section above: a silent-corruption path with no failure signal. Remedy delegated as a detector (validate byte count against the response length, refuse a preview-footer-shaped tail), since the trigger is not reproducible. → problem:harness.curl-to-file-wrote-a-truncated-artifact-with-a-preview-footer

Sources

Every claim above carries a source. Primary research first, then primary vendor documentation, then practitioner reports (labelled as such throughout).

SourceLinkDate
Khan — Verified Detection and Prevention of Concurrency Anomalies in Multi-Agent LLM Systemsarxiv.org/abs/2606.171822026-06-15
Cho & Lee — AgentRoom: Concurrent Multi-Agent Codingarxiv.org/html/2608.23740v12026-08-24
Yang et al. — Co-Coder: Cohesion-Aware Task Partitioningarxiv.org/html/2606.00953v12026-05-31
Nechepurenko & Shuvalov — Coordination as an Architectural Layerarxiv.org/pdf/2605.033102026-05-06
Cemri et al. — Why Do Multi-Agent LLM Systems Fail? (MAST)arxiv.org/abs/2503.13657NeurIPS 2025
Schwarzkopf et al. — Omega: flexible, scalable schedulersresearch.google.com/pubs/archive/41684.pdfEuroSys 2013
Burrows — The Chubby lock service (sequencers = fencing tokens)research.google.com/archive/chubby-osdi06.pdfOSDI 2006
Hazelcast — Distributed Locks are Dead; Long Live Distributed Lockshazelcast.comread 2026-09-13
AWS Builders' Library — Workload isolation using shuffle-shardingaws.amazon.comread 2026-09-13
AWS Well-Architected REL10-BP04 — bulkhead architecturesdocs.aws.amazon.com2024-06-27
AWS — cell-based architecture FAQ (cells vs shuffle sharding)docs.aws.amazon.comread 2026-09-13
A2A Protocol specification · Life of a Task · reference implementationspec · lifecycle · coderead 2026-09-13
Survey of agent interoperability protocols (MCP / ACP / A2A / ANP)arxiv.org/pdf/2505.022792025
MCP architecture spec · session-isolation issue #1087 · SEP 2575 stateless MCPspec · issue · SEP2025 – 2026
Claude Code — parallel sessions with worktrees · agents in parallelworktrees · agentsread 2026-09-13
Erlang/OTP — Supervisor Behaviour (restart strategies, restart intensity)erlang.orgread 2026-09-13
Kubernetes — optimistic concurrency with resourceVersion · retry on conflictOCC · retry2026-02-09
Modern Git Academy — merge queues and semantic conflicts · Mergify — what is a merge queueMGA · Mergifyread 2026-09-13
Zylos Research — git worktree isolation patterns for parallel AI agentszylos.ai2026-02-22
MindStudio — git worktrees for AI coding · parallel agentic development playbookworktrees · playbookread 2026-09-13
Augment Code — multi-agent coding workspace · worktrees for parallel agentsworkspace · worktreesread 2026-09-13
OpenHands platform · OpenHands Agent SDK · SWE-agent · Inside the Scaffold taxonomy2407.16741 · 2511.03690 · 2405.15793 · 2604.035152024 – 2026
Contract Net Protocol (Smith 1980; FIPA SC00029)overviewread 2026-09-13
Blackboard lineage — “The Blackboard Is Back” · Fowler, An Accidental Blackboard · LLM blackboard systemTianPan · Fowler · 2510.012852025 – 2026
CRDTs go brrr (Diamond Types benchmarks) · Approaches to CRDTsjosephg.com · 2310.182202023 – 2026
MAST taxonomy + dataset · GitKraken — questions to ask about agent orchestrationMAST · GitKraken2025 – 2026

MPRO internal sources read this session

tools/scripts/agent-git.sh (live --help) · tools/scripts/agent-git.js · tools/scripts/lib/agent-git-land.js · tools/scripts/lib/lane-ownership.js · tools/git/merge-queue.js:739-747,1145 · tools/git/fleet-integrate-sweep.js:583-616,1051 · tools/git/lane-roots.js · tools/mpro-taskman/lib/store.js:1358-1404 · tools/mpro-taskman/lib/core.js · GET :3018/llms.txt (169 lines) · tools/eidos/vendor-session-markers.js · tools/eidos/vendor-gate-manifest.json · tools/eidos/model-effort-policy.js · tools/eidos/session-scope-retirement.js · core/ports.lock · .grok/agent-card.a2a.json · problem:harness.multi-agent-git-collisions · docs/commit-speed-and-git-gaps/index.html · docs/agent-orchestration-adoption/index.html

Full markdown edition, with the complete 18-entry diagram inventory and the mermaid sources: docs/research/multi-vendor-agent-coordination-worldwide.md (landed f988ad9b636e).

Six-dimension task check (C107)

DimensionAnswer
SecurityTouched. The taskman credential was derived in-process and written to the session scratchpad at mode 0600, never echoed into output. No permission, identity or allowlist changed. Web content was treated as untrusted data; no fetched instruction was executed.
ObservabilityThis report and the six taskman nodes are the observability artifact. The anomaly section records a silent-corruption path that had no failure signal — which is the observability defect this pass found.
ConcurrencyTouched, and it decided an action. A lane I was handed held 13 staged deletions of a peer's only-copy files (present on that lane's HEAD, absent from master). I did not commit from that index, removed my own copy from their lane, left their index byte-identical (13 entries before and after), and landed via the path-scoped land route instead.
Data lossNothing unrecoverable. Two new files, no deletions, no history rewrite; reversible by deleting them. A peer's only-copy work was actively protected rather than swept.
VulnerabilitiesUntrusted input reaching this work = web page text, consumed as prose. No dependency added. Two arXiv PDFs parsed locally with the already-installed pdftotext; diagrams rendered by the already-installed mermaid-cli against a cached headless binary.
Harm and resource wasteThe whole point of the “already covered” section: 15 conflict classes are named as owned so nobody re-buys them, and 6 candidate adoptions are rejected with evidence. Refactor-debt shapes declared: (a) this report duplicates no predicate or list from code — it cites file:line rather than copying logic, so it cannot become a second source of truth; (b) the one enumeration that grows is the vendor list, so the authoritative source is named rather than trusted from the table; (c) A2, A3 and A4 are all specified advisory-first behind existing valves, chosen precisely so a wrong call is reversed by a flag rather than by a refactor.

Changelog

VersionDateChange
1.0.02026-09-13First edition. 18-row primitive-to-conflict-class map; 4 rendered SVG diagrams with ASCII companions; 4 ranked adoptions plus 1 reasoned deferral and 6 reasoned rejections, each delegated to a tracked node; 15-row already-covered-by-MPRO section; 2 corrections to the brief's premise (session scope fence retired; A2A cards non-conformant and unserved, 12/12 probes 404); 1 correction to the fleet figure (8 vendor CLIs / 6 gate-hooked / 5 pinned models); 5 honest “nobody solved this” findings; 1 measured anomaly.

Created by claude_code (claude-opus-5, effort:high, tier sub) for the MPRO CTO · 2026-09-13 04:55 EEST · Origin: DKserver · node task:research.multi-vendor-agent-coordination-worldwide · landed f988ad9b636e