Creating checkpoints by gaslighting a Postgres database

Summary of Creating checkpoints by gaslighting a Postgres database

by The Stack Overflow Podcast

35m•June 9, 2026
Listen to Episode

Overview of Creating checkpoints by gaslighting a Postgres database

In this Stack Overflow Podcast episode, host Ryan Donovan talks with Ryan Clark, Director of Product for Lakebase at Databricks, about how AI agents are changing the way databases are created, used, and managed. The conversation centers on Postgres, branching databases, and how Lakebase uses a custom storage/file-system layer to make database copies fast, cheap, and ephemeral enough for AI-driven workflows, preview environments, and CI/CD.

Main themes

AI agents are creating database sprawl

  • Clark says the biggest shift is that AI agents now create a huge share of databases, often in fan-out workflows where multiple agents spin up isolated environments to solve one task.
  • In this model, many databases are temporary: only one agent’s result “wins,” while the rest are discarded.
  • This makes cleanup, startup speed, and cost control more important than ever.

Databases need to behave more like Git

  • A core design goal is making databases branchable like code:
    • instant copies
    • isolated changes
    • easy cleanup
    • cheap, fast resets
  • Clark argues that traditional database workflows are too slow compared with modern development tools.
  • He compares the shift to the leap from old VCS tools like SVN/CVS to Git: the mental model changed, but the real breakthrough was speed.

CI/CD and preview environments are a major use case

  • One of the original motivations for this architecture was speeding up integration tests and preview environments.
  • Traditional workflows often require:
    • spinning up a new Postgres instance
    • waiting for startup
    • seeding schema and data
    • keeping environment scripts in sync with production
  • Lakebase aims to eliminate that delay by making branching nearly instant.

How branching works

Copy-on-write instead of full duplication

  • Lakebase’s branching uses a file-system trick called copy-on-write.
  • A branch initially points back to the original data instead of physically copying everything.
  • Only when the branch writes to a file/page does it create a unique copy of that piece.
  • This makes branching effectively O(1) at creation time rather than scaling with database size.

Postgres is being “gaslit”

  • The episode’s memorable phrasing: Postgres is “gaslit” into thinking it is writing to a normal file system.
  • Under the hood:
    • Postgres writes WAL-like changes
    • a separate service captures those changes
    • page servers reconstruct the file system view Postgres expects
  • Postgres itself is not modified; the platform wraps around it with custom infrastructure.

Branches are checkpoints

  • A branch is treated as a checkpoint: a snapshot of the primary at a moment in time.
  • This avoids inconsistencies where a branch would otherwise see files changing underneath it.
  • The same approach can support read replicas and other ephemeral copies.

Why this matters for AI and infrastructure

Speed matters more than cleanup

  • With agent workflows, saving time is often more valuable than saving a little money.
  • Agents can’t be assumed to clean up branches or databases manually.
  • The platform therefore needs:
    • scale-to-zero behavior
    • TTLs for branches
    • fast spin-up/spin-down
    • low-friction isolation

Security and identity become harder

  • AI-generated databases can inherit dangerous defaults if branching is done naively.
  • Clark highlights that copying databases can also copy credentials, which is a security risk.
  • Lakebase/Neon’s approach emphasizes:
    • separate passwords for branches
    • centralized permission management
    • integration with Databricks’ Unity Catalog
  • This is especially relevant for sensitive workloads like HR data and other regulated datasets.

Database sprawl is an enterprise problem

  • Clark frames Lakebase as part of a broader “lakebase” category:
    • Postgres is not the source of truth
    • the lakehouse is
    • data is synced into open formats like Iceberg or Delta
  • This helps organizations manage thousands of databases and reduce the chaos of scattered credentials, permissions, and isolated data copies.

Key technical takeaways

Why not just use a key-value store?

  • Clark says relational databases are still useful because they provide:
    • typed storage
    • querying
    • indexes
    • relational understanding of data
  • Key-value systems are simpler and can be faster for some operations, but they become awkward for richer queries and analytics.
  • The likely future, in his view, includes more auto-indexing and more hidden database management behind the scenes.

Postgres remains the engine

  • Lakebase is not a forked Postgres replacement.
  • It keeps Postgres intact but changes the storage and delivery architecture around it.
  • The result is:
    • extremely fast compute startup
    • ephemeral databases
    • isolated branches
    • better fit for AI and modern dev workflows

Notable insights

  • “Agents are like teenagers” because they create work and then don’t clean up unless forced to.
  • Branches should be cheap enough that developers and agents can create them freely.
  • Traditional database infrastructure has not evolved at the same pace as application development tools.
  • Security, identity, and data lineage are becoming as important as raw database performance.

Closing takeaway

The episode argues that AI is turning databases from long-lived, manually managed assets into disposable, branchable infrastructure. Lakebase’s approach is to make Postgres behave more like Git: fast to branch, cheap to copy, isolated by default, and easier to manage across sprawling fleets of AI-created environments.