Skip to content
g! grite refs/grite/wal grite init

← Back to field notes

Migration: 3,000 GitHub issues to grite in one afternoon

Neul Labs · ·
migrationguide

We get asked the same migration question often enough that it deserves a proper write-up: what does it actually look like to move a real backlog from GitHub Issues into grite. The honest answer is that it took us one afternoon for a three-thousand-issue project, and the parts that surprised us were not the parts we had been worrying about.

This is a literal walkthrough of that migration. Numbers, decisions, and the cleanup steps. Nothing has been smoothed over.

The starting state

The repository in question was a five-year-old internal tool with 3,127 issues. About 2,400 were closed; the remaining 700 were the live queue. Labels were used heavily — fourteen of them carrying real semantic weight, another twenty-something used twice and then forgotten. Milestones had been used inconsistently. There were three project boards, two of which were stale.

The team’s reasons for migrating were the ones we expected: an internal queue did not belong on a hosted surface, agent-driven work was making the rate-limited API painful, and the existing offline-first stack for code did not include the tracker.

The reasons we did not expect: the engineers wanted the audit trail. Several edits and closures over the years had been impossible to attribute, because GitHub does not surface the actor of every state change. The team had quietly developed a workaround of including the actor in commit messages. Moving to a signed WAL was a small but real upgrade.

The plan in three steps

The migration plan ended up being three steps, in this order:

  1. Export the GitHub state via the REST API and shape it into the grite event log format
  2. Replay the events into a fresh refs/grite/wal on a migration branch
  3. Reconcile the live state — labels, milestones, assignees — and merge to main

Each step had a couple of gotchas, and the total elapsed time was about four and a half hours including a long lunch.

Step one: export

The GitHub API gives you everything you need, but the shape is annoying. Issues come back paginated, comments are a separate endpoint, and events (label add/remove, state change, assignment) are on a third endpoint. To replay accurately you need all three.

We wrote a small script that walked the three endpoints and produced a single JSON stream of normalized events:

{"type": "issue_create", "ts": "...", "actor": "github:alice", "id": "gh-1234", "title": "...", "body": "..."}
{"type": "issue_label_add", "ts": "...", "actor": "github:bob", "id": "gh-1234", "label": "bug"}
{"type": "issue_comment", "ts": "...", "actor": "github:alice", "id": "gh-1234", "body": "..."}
{"type": "issue_close", "ts": "...", "actor": "github:alice", "id": "gh-1234"}

A few decisions worth flagging:

We mapped GitHub usernames to grite actor IDs through a small lookup file. Each GitHub login became a synthetic actor with a label like github:alice so the historical attribution survived the move. New activity goes through the engineer’s real grite actor identity.

We kept the original GitHub issue numbers as a field in the body, prefixed with [migrated from #1234]. Grite IDs are 128-bit randomly generated, not sequential, so the old numbers do not survive in the IDs themselves. Preserving them as searchable text was enough.

Comments became their own event type. Grite’s comment model maps cleanly to GitHub’s, so there was no shape conflict.

Reactions, mentions, and code references we dropped on the floor for v1. The team decided they were not load-bearing for the queue itself.

Step two: replay

The grite CLI has a hidden-but-documented --from-export flag on grite issue create that accepts the export shape. The replay loop is literally:

cat exported-events.jsonl | while IFS= read -r event; do
  echo "$event" | grite event apply --from-export
done

We ran the replay against a fresh sled database on a migration branch. The whole 3,127-issue replay took about forty-five seconds locally, which is consistent with the published WAL throughput of more than a thousand events per second. The materialized view rebuilt in under a second.

The one thing we tripped on: the original GitHub timestamps had to be respected for the WAL to project the right end state. Grite uses logical clocks per actor for CRDT ordering, but the historical events came with wall-clock timestamps. We threaded those through as a --at flag on each event so the projection matched the GitHub end state.

We then ran grite doctor against the migration branch. It surfaced two integrity issues — both turned out to be GitHub events without a clear actor (older webhook deliveries with the integration user) — and we patched those with a github:system actor.

Step three: reconcile

This was the slowest part by clock time, and the most valuable.

We sat down with the engineering lead, ran grite issue list --open, and walked the seven hundred open issues. Three things came out of that pass:

About eighty issues were closed during the migration. The team had been ignoring them for years; the move was an excuse to ack that.

About forty got merged via grite issue link related_to because they were duplicates we had been sitting on.

The fourteen “real” labels got formalised. The twenty-something garbage labels got removed. Grite’s label model is a commutative set, so removing a label from every issue that carries it is a one-line script over grite issue list --json | jq | xargs.

We also added a dependency graph for the next quarter’s work — about thirty issues with blocks and depends_on edges. That graph would have been technically possible on GitHub through milestones and projects, but in practice it was easier in grite because the CLI made the edits one-shot. The new graph immediately surfaced two cycles, which grite doctor --fix flagged and we resolved by hand.

What hurt

Three things were harder than expected.

Reaction-driven prioritisation. The team had been using GitHub reactions as a vote-counting mechanism. We dropped reactions in the export, and people missed them. Our compromise was to add a votes label with a count in the body, which is ugly but works while we figure out whether to ship a richer reactions primitive.

Cross-repo references. GitHub auto-links references like org/other-repo#1234. Grite does not, because grite is repo-local. The references survived as text but stopped being live links. The fix was a sed pass that rewrote them into URLs pointing at the GitHub history, which is fine because those references were already to closed historical issues.

Notification habits. Engineers were used to GitHub emailing them when an issue was updated. Grite does not have a notification surface, because it does not have a server. The team set up a daily grite sync cron and a shell function that diffs the WAL since the last run, which produces a morning standup digest. That habit took about a week to settle in.

What was easier than expected

Search. Grite’s grite issue list --search is fast because the materialized view is on disk. We expected we would miss GitHub’s full-text search and we did not — partly because the local search is faster, partly because grep over the export covers the cases the built-in does not.

History inspection. “When did this issue change state” is one command in grite. In GitHub it had been a Network tab in DevTools and the events API.

Agent integration. The team’s coding agents started using the new queue immediately, with zero changes to the harness. AGENTS.md showed up after grite init, the agents discovered it, and the next time we looked the queue had a handful of issues with agent: labels and reasonable bodies. That migration step was free.

The afternoon, in time

For posterity:

  • 13:00 — Started writing the export script
  • 13:45 — First successful export pass; 3,127 issues, 11,400 events
  • 14:10 — Replay completed; doctor surfaced two integrity warnings, resolved
  • 14:30 — Lunch
  • 15:30 — Reconciliation pass with engineering lead
  • 17:00 — Merged migration branch; old GitHub repo set to read-only inbound for external reports
  • 17:30 — Coffee

Four and a half hours, end to end, for a five-year backlog. The dominant cost was the reconciliation pass, which was work the team had needed to do for two years anyway. The export and replay were a rounding error.

If you are eyeing a migration, the right shape is: shape the events, replay them, sit down with the lead, and walk the queue. The tooling will not be the bottleneck. The conversation will be, and it is a productive one to have.