Results
Every sync, moved or stayed
Graft moves a copy only when every change lands and your tests pass. That means many syncs stay, and today most syncs with a conflict stay: an agent that isn't sure, a question for you, or tests that can't vouch for a fix all stop the move. That's by design. A wrong move is worse than a stay, because a stay changes nothing and tells you why.
Hosted syncs since launch (live)
Every sync Graft has run for users since 2026-10-28, counted together: no repository names and nothing about anyone. Our own smoke tests are left out. Updated every 5 minutes.
Loading...
The benchmark: 24 syncs on 12 real forks
Each fork synced from its base onto upstream's next release and its newest, with one Claude Code agent and the project's own tests, on 2026-10-28. Every moved result was read by hand against the fork's diff and upstream's history. 10 moved, 14 stayed, 0 moved wrong.
What stopped the 14 that stayed
- 5 the agent wasn't sure enough of its fix
- 4 the fork's own changes already fail upstream's tests
- 3 the agent asked the person for a decision
- 2 a partial drop: part of a change was already upstream
| Fork | From | Onto | Outcome | Why, and the hand check |
|---|---|---|---|---|
| guyernest/arrow | 1.2.0 | 1.2.1 next | stayed | A partial drop: part of a change was already upstream. agents dropped arrow/locales.py, tests/test_locales.py as covered by the new release, but the rest of the change would still land, leaving half a change: arrow/locales.py: The new release's HebrewLocale already has the same 2/ten/higher timeframes for hours, days, weeks, months and years, and the same `_format_timeframe` logic (including the delta == 0 case), so our change is fully covered upstream.; tests/test_locales.py: The new release already has our Hebrew test assertions for 80 days ("80 יום") and 15 years ("15 שנה"), so THEIRS covers our change; the only thing left from OURS is a... |
| guyernest/arrow | 1.2.0 | 1.4.0 newest | stayed | A partial drop: part of a change was already upstream. agents dropped arrow/locales.py, tests/test_locales.py as covered by the new release, but the rest of the change would still land, leaving half a change: arrow/locales.py: The new release already has our Hebrew change: the 'ten'/'higher' plural forms for hours, days, weeks, months and years, and the `_format_timeframe` logic that uses 'ten' for 0 and 3 to 10 and 'higher' above 10.; tests/test_locales.py: The new release's TestHebrewLocale already contains both assertions our change added (`days 80 == "80 יום"` and `years 15 == "15 שנה"`), so upstream covers the more accurate plural for counts... |
| pattonw/tqdm | v4.66.1 | v4.66.2 next | moved | correct: The carried diff is the fork's own diff line for line; upstream didn't touch those lines. No later history on the fork. |
| pattonw/tqdm | v4.66.1 | v4.70.1 newest | moved | correct: As for v4.66.2: identical content, tests pass. |
| inspectorG4dget/tqdm | commit ebda712, between releases | v4.32.0 next | stayed | The fork's own changes already fail upstream's tests. the carried copy fails a check the new release passes |
| inspectorG4dget/tqdm | commit ebda712, between releases | v4.70.1 newest | stayed | The agent asked the person for a decision. agents could not settle it: tqdm/_tqdm.py: it needs your decision: Our right-aligned n_fmt change targets code that now lives in tqdm/std.py, which is not part of this change. Should it be dropped, or ported into std.py as a separate follow-up? |
| emmanuel-ferdman/marshmallow | commit ba8b512, between releases | 4.2.2 next | moved | correct: Both changes dropped: upstream merged this fix as #2894 and released the same changelog entry under 4.2.2. |
| emmanuel-ferdman/marshmallow | commit ba8b512, between releases | 4.3.1 newest | moved | correct: As for 4.2.2. |
| danja100/python-tabulate | commit abf2ff5, between releases | v0.10.0 newest | stayed | The agent wasn't sure enough of its fix. agents could not settle it: tabulate.py: the agent was not sure (0.6) |
| danja100/python-tabulate | commit abf2ff5, between releases | v0.8.7 next | moved | correct: The fork's two commits carried unchanged (identical content); upstream added two unrelated commits. |
| lcvisser/jsonschema | commit 61382d7, between releases | v4.0.0 next | moved | correct: Rewritten: v4.0.0 renamed the registry meta_schemas to _META_SCHEMAS, and the agent's set_validator_for writes to _META_SCHEMAS, the dict validator_for reads; it also left out the fork's stray 'from re import A', so the second commit (which removed it) came out empty and was dropped. Keeping the old name would raise NameError when called, and no test calls set_validator_for, so the tests could not have caught a wrong rewrite here. |
| lcvisser/jsonschema | commit 61382d7, between releases | v4.26.0 newest | stayed | The agent wasn't sure enough of its fix. agents could not settle it: jsonschema/validators.py: the agent was not sure (0.75) |
| cmckain/rich | commit 594c06e, between releases | v13.0.0 next | stayed | The fork's own changes already fail upstream's tests. the carried copy fails a check the new release passes |
| cmckain/rich | commit 594c06e, between releases | v15.0.0 newest | stayed | The agent asked the person for a decision. agents could not settle it: pyproject.toml: it needs your decision: Should the fork still mark its version as a beta (now "15.0.0b"), or was the "13.0.0b" marker only for that release, so the new release's "15.0.0" should stand? |
| TheTechromancer/typer | commit 24f6ba1, between releases | 0.15.2 next | moved | correct: Both changes dropped: upstream merged this fork's PR as #1103 in 0.15.2 (STYLE_COMMANDS_TABLE_FIRST_COLUMN). |
| TheTechromancer/typer | commit 24f6ba1, between releases | 0.27.3 newest | moved | correct: As for 0.15.2. |
| Bluware-Inc/typer | commit a1520dc, between releases | 0.27.3 newest | stayed | The agent asked the person for a decision. agents could not settle it: pyproject.toml: it needs your decision: Which pydantic version should the fork's pydantic support depend on now that the new release targets Python 3.10+ and uses pydantic v2 elsewhere? |
| Bluware-Inc/typer | commit a1520dc, between releases | 0.4.1 next | moved | correct: Eight commits carried unchanged (identical content); the fork's own pydantic tests pass on 0.4.1. |
| borissmidt/typer | commit f83c66d, between releases | 0.27.3 newest | stayed | The agent wasn't sure enough of its fix. agents could not settle it: tests/test_annotated.py: the agent was not sure (0.72) |
| borissmidt/typer | commit f83c66d, between releases | 0.9.1 next | moved | correct: Five commits (asyncio support) carried unchanged over 46 upstream commits; identical content, and the fork's async tests pass with upstream's 881. |
| ExodusForks/commander.js | v9.5.0 | v10.0.0 next | stayed | The fork's own changes already fail upstream's tests. the carried copy fails a check the new release passes |
| ExodusForks/commander.js | v9.5.0 | v15.0.0 newest | stayed | The fork's own changes already fail upstream's tests. the carried copy fails a check the new release passes |
| expo/commander.js | commit ebc8b41, between releases | v15.0.0 newest | stayed | The agent wasn't sure enough of its fix. agents could not settle it: index.js: the agent was not sure (0.72) |
| expo/commander.js | commit ebc8b41, between releases | v5.1.0 next | stayed | The agent wasn't sure enough of its fix. agents could not settle it: CHANGELOG.md: Upstream 5.1.0 only shows the first alias in help, and our fork still adds chalk output, variadic `.alias()` and all-alias help, so the fork's changelog section is placed above the new 5.1.0 entry.; index.js: the agent was not sure (0.7) |
Runs not counted
| Fork | From | Onto | Outcome | Why, and the hand check |
|---|---|---|---|---|
| ExodusForks/commander.js | v9.5.0 | v12.1.0 extra | stayed | The fork's own changes already fail upstream's tests. the carried copy fails a check the new release passes |
| ExodusForks/commander.js | v9.5.0 | v15.0.0 superseded | moved | unchecked move: Moved with three agent rewrites while no test ran: v15 switched to node --test and ESM, Jest 29 couldn't load any suite, both runs named the same 104 files, and the old rule counted that as 'already failing, adds none'. Read by hand, the content matches the owner's own hardening on 12.1.0 (same throws, imports removed, @exodus name), with no dangling references, so it isn't a wrong move, but nothing checked it. The rule is now fail-closed and this run is superseded by commander-exodus@v15.0.0. |
| ali-security/node-semver-162 | v7.5.3 | v7.5.4 next | moved | plausible, untestable: Trivial fork (CI workflows only). The rewrite pins runners and removes schedules as the fork did; tests can't see workflow files. |
| ali-security/node-semver-162 | v7.5.3 | v7.8.5 newest | moved | plausible, untestable: As for v7.5.4. |
Left out of the counts: ExodusForks/commander.js onto v12.1.0 (an extra release, to compare with the fork owner's own move); ExodusForks/commander.js onto v15.0.0 (a move under the old rule that no test had checked, kept in the data and not counted); ali-security/node-semver-162 onto v7.5.4 (a trivial change set); ali-security/node-semver-162 onto v7.8.5 (a trivial change set).
Data: data/bench/sync-real.json; method and history in the evidence repository. Readable write-ups of five of them are on the examples page.