Overview of Scaling Slack and GitHub with Maude Lemaire
In this episode, Gergely Orosz talks with Maude Lemaire—principal engineer and author of Refactoring at Scale—about what it takes to keep massively scaled products working as user demand grows. Drawing from her years at Slack, work on database scaling at GitHub, and recent performance work at Cursor, she explains the real-world engineering tradeoffs behind reliability, sharding, load testing, and AI-assisted development. The conversation is especially useful for anyone interested in how high-scale systems are actually maintained inside products millions of people use every day.
Slack: Real-Time Scaling Lessons
Early Slack architecture
- Slack initially scaled by sharding data by workspace/team.
- Large customers could often fit on a single larger database shard for a long time.
- WebSocket connections were also tied to the customer/workspace model, which worked until enterprise use cases made the assumptions break down.
Enterprise and performance pressure
- Enterprise customers like IBM exposed major scaling issues:
- very large channels,
- many concurrent users,
- cross-team messaging needs,
- heavy load on shared infrastructure.
- Slack’s early product shortcuts were fine at startup scale but became expensive technical debt as usage grew.
A migration story
- Maude describes a major refactor where Slack’s channel membership data was spread across separate tables for:
- public channels,
- private channels,
- DMs,
- group DMs.
- To reduce complexity and query cost, she:
- created a unified abstraction layer,
- used double writes,
- ran backfills,
- shadow-read old vs. new data,
- then carefully cut traffic over.
- She also shares a memorable mistake: accidentally flipping the new-read flag on for Slack internally and temporarily wiping sidebar data until the old path was restored.
Load Testing at Slack
Why traditional load tests weren’t enough
- Slack had load tests that covered API/database throughput, but they did not properly simulate active WebSocket clients.
- That meant they were missing the parts of the system most likely to fail under real usage.
High-fidelity vs. practical simulation
- A first attempt used headless Chrome instances to mimic real Slack clients.
- That approach was too expensive and too resource-heavy:
- tens of thousands of Chrome instances,
- huge CPU/memory costs,
- very brittle infrastructure.
- Slack eventually switched to a lighter approach:
- small Go processes,
- each maintaining a WebSocket connection,
- replaying API and event flows that matched real client behavior.
Koi Pond and recording real user flows
- Maude built tooling that could record a real Slack workflow and turn it into a replayable test script.
- This let teams test specific high-risk paths rather than generic traffic.
- Key lesson: load testing should model the product’s real failure modes, not just raw request volume.
GitHub: Monoliths, Shards, and Scatter Queries
The state of GitHub’s infrastructure
- GitHub was still heavily based on a large Ruby monolith with ActiveRecord.
- It was in the middle of infrastructure evolution, including a move toward Azure and more dynamic sharding.
What she found
- GitHub’s sharding setup was much tighter than what she had seen at Slack:
- only 16 shards for all pull requests and issues.
- That was partly because of scatter queries:
- queries that do not include the primary key,
- forcing the system to search across many shards.
Why this was hard to fix
- ActiveRecord made many queries easy to write but hard to reason about.
- Many expensive queries were hidden behind small-looking function calls.
- Some pages on GitHub required 100+ MySQL queries across multiple clusters just to render.
- A major takeaway was that abstractions can hide the true cost of a query, making performance problems much harder to see and fix.
Cursor and AI-Assisted Engineering
Where AI helps—and where it doesn’t
- Maude is very AI-using, but she says LLMs are not especially good at performance investigation.
- They can help with:
- implementation once the path is known,
- basic instrumentation,
- generating initial ideas or diagrams.
- They are weaker at:
- understanding asynchronous systems,
- choosing the right place to sample traces,
- reasoning about edge cases,
- understanding product tradeoffs.
A practical example: time to first token
- At Cursor, she worked on improving time to first visible token.
- This involved unpacking a complicated pipeline:
- prompt submission,
- context assembly,
- caching,
- inference proxy,
- backend streaming,
- and other asynchronous pieces.
- She used a large sequence diagram, printed on paper, and manually annotated it to verify what mattered and what didn’t.
Mutation testing and trace discipline
- She found that LLM-generated tests often only validate what was just written.
- To compensate, she leans on mutation testing to force edge cases and reveal weak tests.
- She also warns about trace overuse:
- too many spans become useless,
- sampling must be done consistently and in one place.
Platform Engineering Career Advice
Why product and platform should rotate
- Maude strongly advocates for rotation between product and platform teams.
- Product engineers often notice repeated pain points and eventually want to solve them once, at the platform level.
- But staying too long on platform can disconnect engineers from real product development.
What good platform engineers need
- Deep understanding of how product teams actually ship.
- A willingness to build guardrails rather than assume users will behave ideally.
- Constant cross-pollination so platform teams don’t become detached from reality.
Her core advice
- If you’re on a product team:
- care about reliability and performance early,
- help platform teams understand your workflows,
- contribute to building reusable solutions.
- If you’re on a platform team:
- stay close to product teams,
- understand how they move fast,
- design systems that tolerate that pace.
Key Takeaways
- Scaling problems are usually product-shaped, not just infrastructure-shaped.
- The real bottlenecks are often hidden behind convenient abstractions.
- Load testing should target the actual failure modes of your system, not just generic throughput.
- A stable platform requires both engineering rigor and constant feedback from product teams.
- AI can speed up implementation, but humans still need to validate architecture, correctness, and tradeoffs.
Notable Insights
- “It’s not because you’re good at something that you like doing it.”
- “You have to put your product hat on when making performance decisions.”
- “The heaviest users of the product often suffer the worst reliability issues.”
- “The spaghetti now wants off the plate.”
Bottom Line
This episode is a practical tour through the hidden complexity of scaling modern software systems. Through Slack, GitHub, and Cursor, Maude Lemaire shows that performance work is less about clever tricks and more about understanding real usage, designing the right abstractions, and continuously tightening the feedback loop between product and platform.