AI Agent News/stories/github-details-new-git-storage-architecture-built-for-agent-scale-writes.md

Infrastructure

GitHub details new Git storage architecture built for agent-scale writes

GitHub has described a new Git storage architecture that it says can handle thousands of writes per second to a single repository. The design is in limited private preview, with a broader rollout planned for early next year.

Source: github.blog/engineering/architecture-optimization/inside-githubs-new-git-storage-architecture

In internal testing, GitHub says the architecture supported 5,000 concurrent agents working on one repository and 7,200 pushes per second to that repo, while read latency stayed consistently low. Git objects and packs are stored in Azure Blob Storage, while references and the pack manifest live in Ref-CAS, a replicated log service that stores only pointers to object data.

Reads are served by virtual replicas, which are disposable compute nodes whose local disks act as caches. A replica first checks a version number with Ref-CAS, so an unchanged repository needs only one comparison. Writes upload their packs to Blob Storage before a reference transaction is committed, so references never point at data that is not yet durable.

Compaction and garbage collection run in the background on separate nodes that work directly against storage, with leases keeping one node per shard at a time. GitHub says these changes keep the pack count roughly flat as pushes grow. Git commands and existing protections stay the same for developers.

Why it matters if you build agents

Agent fleets working on a single repository can expect much higher push and read capacity than today's bursts, although the new storage is only in limited private preview. Existing Git workflows should not need changes. Teams should not plan around the listed follow-up work, such as partial replicas and an atomic commit API, until GitHub ships it.

Get stories like this every weekday

One short email with the day's agent tooling news.

Subscribe