Why this collection exists
The methodology described in Evolutionary Database Design and operationalized in Refactoring Databases: Evolutionary Database Design has been clear for twenty years. The seven practices, the catalog of 70+ named refactorings, the transition mechanics – all of it documented, peer-reviewed, taught.
That methodology reached CI/CD in 2010 with Steady Supply (Chapter 12: Managing Knowledge). Migrations turned first-class artifacts within the deployment pipeline. The self-discipline of database-changes-as-code reached the broader CI/CD motion. What CD did not resolve was per-pipeline isolation: pipelines might run migrations, however they nonetheless wanted a goal database, and that concentrate on was shared. Follow #4 – Everyone will get their very own database occasion – has stayed aspirational on most groups as a result of true per-developer production-shaped databases value time, cash, and DBA cycles. The compensating layer that emerged to work across the hole (mock objects, shared staging environments, in-memory database substitutes, DBA ticket queues) turned foundational methodology by default, not by design.
In 2026, copy-on-write database branching arrives in Databricks Lakebase. A one-second, zero-storage-at-creation department of a terabyte-scale manufacturing database is now an O(1) operation. The constraint that stored Follow #4 aspirational has lifted.
This collection describes what adjustments when the constraint lifts: not the methodology – that holds – however the practices that emerge for the primary time, the team-scale governance that turns into automated, the position evolution for the DBA, and the brand new substrate that brokers share with their human counterparts.
Meet Jen
Jen is the developer character from Evolutionary Database Design. In that essay she carried out a database refactoring – splitting an inventory_code area into location_code, batch_number, and serial_number – as a routine person story, illustrating that DBAs and builders can collaborate, schemas can evolve in small increments, and migrations carry the change ahead safely.
The collection picks up with Jen twenty years later. The methodology she follows is similar one she adopted in 2003. What’s new is the substrate beneath her workflow: copy-on-write database branching, which makes the practices she has been studying about operationally actual at manufacturing scale. Throughout the three components of this collection she is similar Jen at three scopes – her day (Half 1), her new playbook (Half 2), and her staff (Half 3).
Half 1: Jen’s story: one characteristic, one database change
To know how this works, let’s stroll by means of the journey of how a developer named Jen implements a job that states that the person ought to be capable to see, search and replace the situation, batch and serial variety of a manufacturing in stock.
The next describes the assorted steps Jen has to take to perform this job, whereas describing the steps we’ll attempt to examine how Jen’s workflow adjustments when working with conventional databases and utilizing Lakebase that enables database branching at minimal value.
Jen begins engaged on her characteristic job
Jen picks up what seems like an easy characteristic. The product staff needs to permit customers to seize location, batch and serial variety of an merchandise throughout stock addition and use it later within the software move. From the skin, the change feels small: add a area to the display, save the worth, present it within the Stock display for an merchandise, and possibly use it in a downstream determination later.
For Jen, the applying change is simple to image. She is aware of the place the shape lives. She is aware of which service handles the request. She will be able to see the mannequin object that wants extra attributes. However the second she traces the change right through, she sees the true dependency, the database has to alter too.
Some new columns are wanted, present knowledge within the manufacturing setting must be preserved and must be semantically right. The appliance should deal with outdated and new knowledge safely and he or she wants so as to add exams to show that the brand new fields are saved, learn, and displayed appropriately. What seemed like a easy characteristic is now a coordinated software and database change, with the added duty of making certain present manufacturing schema and knowledge is migrated to the brand new schema.
Shared database
Jen creates a code department for the work she about to embark on, and since they’re utilizing a shared database and the remainder of the staff is utilizing the identical database for improvement, she instantly begins desirous about all of the adjustments she goes to introduce within the database layer that would have an effect on different customers of the shared database and begins planning on how she will make it protected for others, might she run make the applying change domestically and be capable to run her unit and integration exams? Every possibility has prices. She will be able to wait. She will be able to ask the staff to coordinate. She will be able to rise up her personal native Postgres in Docker, seed it with a stale pg_dump from every week in the past, and hope the variations do not matter. She will be able to fall again operating a neighborhood database in a container or to an in-memory database H2 or SQLite that runs quick however makes use of the fallacious dialect, so her exams cross domestically and floor unknown failures on actual Postgres. Can she even check her schema and knowledge migration scripts? This worry of breaking others slows her down and on the identical time doesn’t enable her to experiment with a number of choices of constructing the characteristic.
Since in a shared database, one developer could also be testing a enterprise logic change, one other is debugging a knowledge migration, another person created check knowledge that Jen doesn’t perceive. If Jen applies her schema change to the shared database, she could break another person’s work. If another person adjustments the schema whereas she is testing, her outcomes could now not be dependable. If she provides check knowledge, it might intrude with one other developer’s assumptions.
Jen can wait till the shared database is free, which protects the staff from collisions, nevertheless it turns a small characteristic right into a scheduling drawback and productiveness loss. She will be able to coordinate manually with the opposite builders: “Are you utilizing dev proper now?” “Can I run a migration?” “Please don’t reset the info for the subsequent hour.” one thing like a baton in a relay race, That works for some time, nevertheless it doesn’t scale, particularly with a distant or multi timezone staff.
Jen thinks of another choice, utilizing a neighborhood in-memory database, she is aware of that this setup doesn’t match the state of the database utilized by the remainder of the staff, which suggests she is not going to have the boldness in her resolution because the change may match domestically and nonetheless fail later when its meets the true knowledge and schema in greater environments like staging and manufacturing.
The true drawback Jen is encountering is of slower suggestions she will make the change, however discovering out if the change works, however quick and reasonable suggestions and with out this suggestions the database change turns into one thing the staff treats rigorously and finally ends up selecting the primary resolution that works and by no means experiments or tries a number of options, thus resulting in suboptimal options, lowered productiveness and dissatisfied builders.
Particular person database branches
Utilizing Lakebase, Jen has the power to department a database for her particular person use and this functionality fully adjustments the best way she works.
As an alternative of ready for the shared improvement database to grow to be accessible, Jen creates a database department databricks postgres create-branch for her characteristic or utilizing a VS Code / Cursor Extension. This adjustments the form of the work instantly. She is now not asking the staff for a quiet window. She is now not negotiating with different builders about who can run which migration and when. She is now not making an attempt to guard her half-finished change from everybody else’s half-finished adjustments. She has her personal remoted database house, created from the identical form of database setting the applying will ultimately use in manufacturing.
The department provides Jen a quick copy of the database state she must work towards. She now has the identical Postgres engine, the identical schema, the identical governance insurance policies, and the identical production-shaped knowledge she’d see if she queried manufacturing instantly. The one distinction: this department will be modified, discarded, or recreated with out affecting some other workload. She isn’t testing towards a simplified native database that behaves in another way from manufacturing. She is working with the identical database kind the staff makes use of in manufacturing, with the identical sorts of schema guidelines, constraints, indexes, reference knowledge, and migration historical past that make database adjustments succeed or fail in the true world. That realism issues as a result of many database issues don’t seem in remoted unit exams. They seem when a brand new migration meets present construction, present knowledge, present assumptions, and present software habits.
Now Jen can deal with the database change as a part of design, not simply as a deployment step. She will be able to attempt the plain model first: add the brand new columns, set a default logic to separate the present column, create a database migration script, replace the applying, and run the exams. Then she will ask higher questions. Ought to this migration script work for manufacturing knowledge volumes, is the info high quality in manufacturing the best way her script expects them to be? Is a knowledge migration script hiding lacking enterprise data? Ought to the desire be modeled as easy columns, a lookup desk, or a separate item_information desk as a result of extra data is more likely to come later? Will the question sample want an index? Will this design make downstream reporting simpler or tougher? Within the outdated workflow, these questions usually get compressed as a result of altering the database is pricey.
Within the branched workflow, Jen can discover them whereas the characteristic continues to be being formed. The DBA can pair along with her to information her on manufacturing nuances and knowledge volumes, thus offering invaluable enter within the design of the answer as a substitute of being an after the actual fact reviewer.
Making the applying and database change collectively
Jen writes the migration script. No matter her staff makes use of – Flyway, Liquibase, Alembic, Knex, Prisma – the script lives within the code repo, alongside the applying adjustments. Schema and knowledge migration travels with code.
(That is the Break up Column refactoring – considered one of ~70 patterns catalogued in Refactoring Databases, the ebook that operationalized the seven practices.)
She applies the migration to her department utilizing flyway migrate. The device runs in underneath a second towards real-shaped knowledge. She updates her repository code to learn and write the three new columns. She runs her check suite. Assessments cross towards actual Postgres no mocks, no in-memory substitutes.
If she needs a clear slate to attempt a unique strategy, she discards the department and creates a recent one off manufacturing. One other second. No cleanup tickets. No DBA concerned.
Identical Jen. Identical refactoring. What modified is the aptitude.
Area to fail quicker
The power to experiment is necessary. Evolutionary design and improvement is not only about transferring rapidly by means of a predefined guidelines. It is usually about studying because the work turns into extra concrete. Jen could uncover that the primary schema design works however creates awkward software logic. She could uncover that the second design is cleaner however makes migration of present information extra difficult. She could uncover {that a} small normalization determination now would make future adjustments simpler. The primary migration script she wrote the SUBSTRING indexes are off by one. The harmful DROP COLUMN ran earlier than she might confirm the brand new columns have been populated appropriately. As a result of she has her personal department, these discoveries are cheap. She will be able to apply a migration, run the applying, examine the info, roll ahead with one other migration, or reset and check out a unique path.
The department additionally adjustments the emotional posture of the work. Jen doesn’t need to be overly cautious as a result of another person is likely to be relying on the shared improvement database. She doesn’t need to announce each experiment to the staff. She doesn’t have to wash up check knowledge instantly as a result of one other developer may journey over it. Her department is a protected place for unfinished pondering. It may possibly include momentary tables, failed migration makes an attempt, awkward check knowledge, and half-formed designs with out creating noise for anybody else.
On the identical time, isolation doesn’t imply detachment from the staff’s requirements. Jen nonetheless writes migration scripts. She nonetheless retains the applying code and database change collectively. She nonetheless runs exams. She nonetheless expects the ultimate design to be reviewed. The distinction is that she will do the messy a part of the work privately and rapidly earlier than asking the staff to purpose concerning the polished model. By the point she opens a pull request, the dialog can deal with whether or not the design is correct, not whether or not she had a protected place to check it.
That is the important thing shift: the database department provides Jen quick, reasonable, remoted suggestions that she will additionally get reviewed from her tech leads or DBAs, by exhibiting her database department. Quick means she will create the setting when she wants it, not when somebody provisions it for her. Sensible means she is testing towards the identical form of database habits that issues in manufacturing. Remoted means her experiments don’t interrupt anybody else. Collectively, these three properties flip database change from a bottleneck into a traditional a part of characteristic improvement.
Jen can now transfer the applying and database ahead collectively. Her code department and her database department grow to be two sides of the identical job. One holds the applying adjustments. The opposite provides these adjustments an actual database to dwell towards. As an alternative of ready, coordinating, or pretending with a simplified setup, Jen can design, check, revise, and be taught. The characteristic continues to be small, however now the database is now not what makes it sluggish.
Opening the pull request
Jen commits each the applying code and the migration script. She opens a PR.
CI does what Jen simply did, however for the staff: it creates its personal momentary Lakebase department, applies the migration, runs the applying check suite, runs database exams towards the migrated schema, validates the migration itself (applies cleanly, idempotent, reversible), and posts a schema-diff touch upon the PR exhibiting precisely which database objects modified.
The reviewer can now see what the schema change does inline with the code that makes use of it, altering their contextual understanding from summary to concrete.
Screenshot of the Department Diff Abstract view from the Lakebase SCM Extension
Reviewing the change
Within the outdated workflow, the database overview query was “will this break the database?” – gated by a DBA who had to take a look at each change in isolation as a result of each change had production-scale penalties if it bought free. Critiques had been synchronous. Schedules collided. The DBA’s calendar turned a queue and generally the DBA would get skipped for “Time to Market” causes.
Within the new workflow, the query is “is that this the fitting design?” The DBA has already seen the schema diff posted by CI. They’ve already seen the migration run efficiently towards a real-data department. Jen may also pull within the DBA for a dialogue, to point out what she is pondering of and all the opposite choices she has tried. The DBA can overview on their schedule, not Jen’s. They will present overview a lot earlier within the resolution improvement cycle and enhance the answer round knowledge integrity, indexing technique, future extensibility or long-term maintainability, not on the protecting gatekeeping that used to take all their time.
The staff opinions code and database collectively. One PR. One dialog. Identical window.
Merging with confidence
The migration has already been examined towards an actual knowledge department. The appliance has already run towards the modified schema. The schema migration has been reviewed. The CI construct has run the identical actual steps and has been inexperienced for an hour.
When Jen merges, the migration applies to the subsequent setting, the branches for database and code for CI setting and Jen are cleaned up. Thus making certain that the database change is now not a release-night shock.
What Jen simply did is the fifth observe from the 2003 essay: steady integration of database adjustments.
What Jen’s journey reveals
Database change turns into a part of regular improvement. Branching reduces ready, threat, and coordination overhead. Jen’s each day loop now provides her quick, remoted suggestions on the database layer.
In Half 2 – Jen’s New Playbook, we clarify what lifted and why the compensating layer Jen labored round her entire profession can come out: copy-on-write branching, the structure that makes it work, and the methodology optimizations that observe.
In Half 3 – Jen’s Workforce at Scale, we take a look at what Jen’s story seems like when she’s considered one of fifty builders, or possibly she is engaged on a white labeled product, or she is engaged on a modular monolith with a lot of domains inside it – governance at department creation, the DBA reframe, the agent-in-the-loop, and the platform-design work that opens up when the DBA’s calendar is not a ticket queue.
For readers who need the tour of the IDE tooling Jen used on this publish, there’s the Companion: Plugin Walkthrough – the Lakebase SCM Extension for VS Code / Cursor, finish to finish.
Lastly, a Lakebase App Dev Equipment for brokers to make use of accompanied by an book for people to observe will probably be launched shortly.
