vault

Shared Database Anti-Pattern

36 min read

Shared Database Anti-Pattern

A Shared Database in a microservices architecture is the situation where two or more services have direct access — not through APIs but through actual database connections — to the same database tables, with both read and write permissions. The pattern is named in microservices literature as one of the most reliable signs that an architecture has failed to deliver on its promises. Sam Newman’s Building Microservices (2nd ed., 2021, chapter 4) and Chris Richardson’s Microservices Patterns (2018, chapter 2 and §5.1) both identify shared databases as the single most damaging anti-pattern in microservices, with Richardson’s microservices.io site dedicating a full pattern entry to the negative pattern (https://microservices.io/patterns/data/shared-database.html) alongside its positive counterpart, Database per Service (https://microservices.io/patterns/data/database-per-service.html). Mathias Verraes’s 2014 essay Distributed Big Balls of Mud connects the pattern to the broader Big Ball of Mud Anti-Pattern dynamics; the shared database is essentially a Big Ball of Mud Anti-Pattern at the data layer, manifesting through the schema rather than through code.

The problem is structural: when services share a database, the database schema becomes the implicit, undocumented contract between every service that touches it. Any column rename, type change, or constraint addition affects every service that reads or writes that column. Schema migrations require coordinating with every service team. The boundary between services — supposedly the architectural mechanism that allows independent deployment, scaling, and evolution — is effectively erased by the shared schema. The services are physically separate processes but logically one system whose components happen to communicate through database state rather than network calls. The result is what Newman calls “the worst of both worlds”: the operational cost of distributed systems (network calls, partial failure, deployment automation) plus the deployment coupling of a monolith (because schema changes force lockstep coordination). This is why the shared database is the single most common cause of Distributed Monolith Anti-Pattern — they are partner pathologies, and remediating the distributed monolith almost always requires breaking the shared database first.

This note treats the shared-database anti-pattern in depth: the precise definition (and the subtle distinctions between “shared instance,” “shared schema,” “shared tables,” “read-only access,” and “via stored procedures”); why it’s an anti-pattern even when “performance” arguments are made for it; how it happens (almost always: incomplete migration from monolith); the canonical recognition signs; the cure (database-per-service, with Event-Driven Architecture for cross-service data needs); the explicit relationship to the Distributed Monolith Anti-Pattern; real-world fixes (Shopify’s modular monolith approach, Netflix’s per-service Cassandra, Stripe’s per-service PostgreSQL); a worked example showing what happens during a flash sale when order-service and inventory-service share the inventory table; and the pitfalls — including the seductive “read-only is fine” myth and the “we have triggers” coordination layer that makes things worse.

0.1 Visualizing the Pattern

flowchart TB
    subgraph "Database per Service (healthy)"
        SvcA1[Service A] --> DbA[(DB A)]
        SvcB1[Service B] --> DbB[(DB B)]
        SvcC1[Service C] --> DbC[(DB C)]
        SvcA1 -.->|"API"| SvcB1
        SvcB1 -.->|"API"| SvcC1
    end
    subgraph "Shared Database (anti-pattern)"
        SvcA2[Service A] --> SharedDb[(Shared DB)]
        SvcB2[Service B] --> SharedDb
        SvcC2[Service C] --> SharedDb
        SvcA2 -.->|"sometimes API"| SvcB2
        SvcB2 -.->|"sometimes API"| SvcC2
    end

What this diagram shows. Top: each service has its own database; cross-service data access goes through APIs. The schema of any one database is private to its owning service; changes in one don’t affect others. Bottom: all three services connect directly to a single shared database; the schema is implicitly shared across them. Even though the services have APIs between them, the direct database access creates a parallel, undocumented contract — every service depends on the schema, and schema changes affect all of them simultaneously. The arrows make the difference visible: in the healthy topology, the database-touching arrows are short and isolated to each service; in the anti-pattern, the database-touching arrows converge on a single shared resource.

1. Definition — What Counts as Shared

The term “shared database” hides a spectrum of arrangements. The architectural impact depends on which is in play.

Spectrum of “sharing”:

  1. Shared physical instance, separate logical databases (separate schemas). Service A and B both run on the same Postgres instance but each owns its own logical database (service_a_db and service_b_db). They cannot see each other’s tables. This is not the shared-database anti-pattern; it is just shared infrastructure. It might have operational tradeoffs (a single instance failure affects both services) but does not create coupling.

  2. Shared logical database, separate schemas (Postgres-style schemas, MySQL-style databases). Service A and B share the database but each owns a separate schema namespace within it. They could in principle see each other’s tables, but the convention is they don’t. Same pattern as #1 but slightly closer; the convention is what protects them. Still not severely problematic if discipline holds.

  3. Shared schema, separate tables. Service A and B work in the same schema, but each owns specific tables. A reads/writes only its tables; B reads/writes only its tables. There is one source of truth per data point. This is the gray zone; it works if the team-discipline is rigorous about ownership. In practice, it tends to drift toward #4.

  4. Shared tables, separate access patterns. Service A and B both access the same tables. Maybe A reads, B writes; maybe A and B both write to different columns; maybe they both read everything. This is the textbook shared-database anti-pattern. Schema changes affect both services.

  5. Shared tables with cross-service joins. Beyond #4: service A and B not only access the same tables, but join across them — a query in service A’s code joins table X (owned conceptually by A) with table Y (owned conceptually by B). This is the worst form: the queries themselves embed assumptions about both services’ schemas, so even a refactoring within one service’s tables can break queries in another service.

The transitions between these levels are gradual, and a system can drift from #1 to #5 over years if no one is enforcing the boundary. The anti-pattern strictly is #4 and #5; #3 is a slippery slope; #1 and #2 are fine.

The defining property: the schema becomes a coupling surface. Any change to a table’s structure affects every service that touches the table. The services have not been decoupled from each other; they have been decoupled in their code but coupled in their data. Code-level decoupling without data-level decoupling is mostly cosmetic.

2. Why It’s an Anti-Pattern — The Coupling Through Schema

The argument against shared databases is not aesthetic; it is structural. The shared schema is a dependency, and a particularly damaging one because it is implicit.

Argument 1: schema changes require multi-service coordination. A column rename in a shared table requires every service that uses the column to be updated simultaneously. The migration becomes a cross-team coordination event: “We’re renaming users.email_addr to users.email. Every service must update their queries on the same day.” The team velocity of every service is bottlenecked on the slowest team. In a single-team monolith, the rename is one pull request; in a shared-database microservices estate, it’s a cross-team project. The supposed benefit of microservices — team autonomy — is negated by the schema coupling.

Argument 2: implicit contracts are unversioned and undocumented. A REST API has an explicit contract (the OpenAPI spec, the Swagger documentation, the published schema). Consumers know what they depend on. When the API changes, the change is visible. A database schema is also a contract, but an implicit one — there’s no published version, no change log, no consumer registry. Service B reading from users.email_addr is not visible to service A unless A’s team is told. Schema changes routinely break consumers nobody knew existed.

Argument 3: data-ownership becomes ambiguous. When two services write to the same table, who owns the data? Whose business rules apply? When the data is invalid, which service is responsible? The shared schema dissolves the bounded contexts that microservices are supposed to establish. The architectural promise of clear ownership is lost.

Argument 4: cross-service joins create runtime coupling. A query in service A that joins tables owned by both A and B has a runtime dependency on B’s schema. A schema change in B breaks A’s query. This is the worst case; the coupling is in the query strings themselves and is hardest to detect.

Argument 5: deployment coupling. Schema migrations must be coordinated. The deploy of one service may require a schema state that another service hasn’t yet been updated to handle. Either the migration is done in lockstep (deploy coupling) or in expand-contract phases that themselves require cross-team coordination.

Argument 6: scaling is undermined. One reason microservices are valuable is that each service can be scaled independently. With a shared database, scaling decisions are made at the database level — adding read replicas, sharding the database, choosing the database type — and these decisions affect every service that touches the database. A service that needs document-store semantics is stuck with a relational database (or vice versa) because the rest of the services are using it.

Argument 7: technology choice is undermined. Each service should be free to choose the data storage that fits its needs (relational, document, graph, time series, key-value, search index). A shared database forces all services to use the same technology, regardless of fit.

Argument 8: failure-domain confusion. When the shared database is degraded, every service that depends on it is degraded. There is no failure isolation. An issue in service A’s query patterns (a runaway query, a lock contention) affects service B’s performance. The blast radius of any database issue is the entire estate.

Argument 9: testing is harder. Service A’s tests can’t realistically exclude service B’s data (it’s all in the same database). Test isolation requires careful test-data setup; tests are slower because they fight for shared resources.

Argument 10: it is a Big Ball of Mud Anti-Pattern at the data layer. All the dynamics of BBoM — promiscuous information sharing, no clear boundaries, change-everywhere effects — manifest at the database level. Verraes’s “Distributed Big Balls of Mud” essay (2014) names this connection explicitly.

The arguments compound. Together, they say: a shared database makes services not actually services. It collapses the architectural decoupling that justifies the operational cost of distribution.

3. Why It Happens — Almost Always: Incomplete Migration

Shared databases rarely arise from greenfield design. Almost every shared database is the residue of a monolith-to-microservices migration that didn’t complete the data-decoupling step.

The migration story. A team decides to migrate from a monolith to microservices. The monolith has one big database with hundreds of tables. The team starts by:

  1. Splitting the application code into multiple services.
  2. Each service runs as a separate process with its own deployment.
  3. The database remains shared because:
    • The migration would take too long to also migrate the database.
    • Performance reasons (cited but rarely justified).
    • “We’ll fix the database later.”
    • Lack of clear ownership decisions.

The result is what the team calls “microservices” but which is structurally still a monolith. The migration has separated the deployment but not the data. Every subsequent decision is made under the constraint of the shared database, and over time the constraint calcifies.

The “performance” justification. Frequently, teams justify the shared database with performance arguments. The arguments are usually wrong:

  • “API calls are too slow.” Yes, an API call is slower than a local database join (~5-20ms vs ~1-5ms for an in-DB join). For most read patterns, the difference is negligible relative to the overall request latency. For genuinely hot paths, alternatives exist (caching, read replicas, materialized views, denormalization within a service’s own database).
  • “Cross-service joins are needed for complex queries.” Often true, but the right answer is to denormalize — service A maintains its own copy of the relevant data from service B, kept current via events. The cross-service join becomes a local join. This adds eventual-consistency considerations but is the textbook microservices answer.
  • “We have transactional requirements across services.” This is the stronger argument, but the answer is not “shared database”; the answer is “Saga Pattern for cross-service workflows” or “redesign the boundary to keep the transactional needs within one service.”

Lack of bounded-context thinking. The deeper cause is that the migration didn’t begin with bounded-context analysis. The team didn’t ask “what are the natural domain boundaries?” — they asked “how do we split the code?” Splitting code without splitting data leaves the services with code-level boundaries and data-level coupling. The fix requires going back to bounded-context modeling.

Organizational dynamics. Sometimes the shared database persists because no team owns it. The original DBA team has been disbanded; the database is “everyone’s responsibility,” which means no one’s. Schema migrations happen ad hoc; ownership is contested. The path of least resistance is to leave the shared database alone and focus on code refactoring.

4. Recognition Signs — Diagnosing a Shared Database

Concrete diagnostics that confirm the pattern:

The same table is accessed by multiple services. The most direct signal. Run SHOW PROCESSLIST (or the equivalent for the database engine) during business hours; identify the database connections. If connections from multiple service hostnames are issuing queries against the same table, you have a shared database.

Service code contains queries against tables it doesn’t conceptually own. Grep across service codebases for SQL queries (or ORM mappings). Identify which tables each service queries. If service A’s code references tables that service B “owns” (or vice versa), you have a shared database.

Cross-service joins in code. SQL queries in service A’s code that JOIN across two tables, where the second table is conceptually owned by service B. These are the worst case because the query string itself embeds assumptions about both services’ schemas.

Schema migrations are cross-team coordination events. Migrations require Slack channels with multiple teams represented; deployment windows are scheduled to minimize cross-team disruption; rollback playbooks are extensive. In a healthy database-per-service estate, migrations are owned by single teams and shipped on their own cadence.

The DBA team gets paged for issues that affect specific services. If “the inventory service is slow” is investigated by the DBA team rather than the inventory team, the database is the boundary, not the service.

“Schema drift” between staging and production. Multiple services migrating the schema independently produces inconsistencies; staging and production drift apart. This is a sign that nobody owns the schema.

Service tests touch tables outside their domain. Service A’s integration tests insert into table B, or query table B, because they need data from “the other service” without going through “the other service’s API.” The test code reveals the shared-database dependency.

Backups are at the database level, not the service level. Backup and restore operations span all services because the database is one logical unit. Per-service backup/restore is impossible.

Deployment coordination Google Docs. Cross-team coordination for schema changes is documented in shared docs; the existence of these docs is itself diagnostic.

When 3+ of these signs are present, the shared-database anti-pattern is confirmed. The next question is severity: how much sharing, across how many tables, between how many services?

5. The Cure — Database per Service, with Event-Driven Architecture for Cross-Service Data

The architectural prescription is unambiguous: each service owns its data exclusively. Other services access that data only through the owning service’s API. Cross-service data needs are met via events that propagate updates to consumers, who maintain their own local copies.

5.1 Database per Service

Richardson’s microservices.io describes the Database per Service pattern (https://microservices.io/patterns/data/database-per-service.html) as the antidote. The rules:

  1. Each service has its own database. Physically separate (different instances or different logical databases).
  2. Only the owning service connects to its database. Other services do not have credentials for it.
  3. Cross-service data needs are met via the owning service’s API. Service B needs data from service A’s database? Service B calls service A’s API.
  4. Services are free to choose their database technology. Service A can use PostgreSQL while service B uses Cassandra and service C uses Elasticsearch. The choice is per-service.

The pattern decouples services at the data layer. Schema changes are local to one service; cross-service contracts are explicit (the API). Independent deployment, scaling, and evolution become structurally possible.

5.2 Event-Driven Data Propagation

For scenarios where service B needs frequent read access to service A’s data (where API calls would be too slow or too numerous), the pattern is event-driven data propagation:

  1. Service A emits events on data changes: UserCreated, UserUpdated, UserDeleted.
  2. Service B subscribes to these events.
  3. Service B maintains its own local copy of the relevant subset of A’s data, kept current via the events.
  4. Service B reads from its own local copy at request time — no API call needed.

This is the Event-Carried State Transfer pattern (cf. Event Notification vs Event-Carried State Transfer in this batch). The cost: eventual consistency (B’s view of A’s data may be slightly stale). The benefit: B doesn’t need a synchronous call to A, doesn’t need to share A’s database, and can scale independently of A.

The event transport is typically Distributed Log System Design (Kafka) or Publish Subscribe System Design (Pub/Sub, SNS+SQS). The choice depends on durability and replay needs.

5.3 Saga Pattern for Cross-Service Transactions

For scenarios where multiple services must coordinate to maintain consistency (the “transaction across services” need), the Saga Pattern replaces the shared-database transaction. The saga is a sequence of local transactions, each in its own service’s database, with compensating actions on failure. The saga preserves business consistency without sharing databases.

5.4 Outbox Pattern for Reliable Event Publication

A subtle but critical pattern: how does a service reliably emit an event after committing a database transaction? The naive approach (commit DB, then publish event) has a window where the DB commit succeeds but the event publish fails (or vice versa). The Outbox pattern (https://microservices.io/patterns/data/transactional-outbox.html):

  1. The service writes the event to an outbox table in its own database, in the same transaction as the data change.
  2. A separate process (or Change Data Capture tooling like Debezium) reads the outbox and publishes events to the message bus.
  3. The outbox guarantees that if the data change committed, the event will eventually be published.

The Outbox pattern is the reliability primitive for the event-driven cross-service-data approach. Without it, the consistency guarantees are weaker.

5.5 Change Data Capture for Migration

For migrations from shared database to database-per-service, Change Data Capture (CDC) tools (Debezium, AWS DMS, etc.) read the database’s transaction log and produce a stream of change events. During migration, CDC can be used to:

  1. Replicate data from the shared database to per-service databases.
  2. Keep them in sync during the transition.
  3. Eventually cut over services to read from their own database.

CDC is the transitional infrastructure for the database split.

5.6 Worked Migration — Detailed View of One Table

A worked example showing the migration of one shared table in detail.

The setting. An e-commerce system with a shared inventory table. Currently accessed by:

  • OrderService: reads quantity, decrements on order placement.
  • InventoryService: reads/writes quantity, manages restocking.
  • WarehouseService: reads quantity for fulfillment.
  • AnalyticsService: reads quantity periodically for reporting.

The migration’s goal: InventoryService becomes the sole owner of inventory data; all other services access via API or events.

Step 1: API design. The InventoryService team designs APIs that other services can use:

  • GET /inventory/{product_id} — current quantity.
  • POST /inventory/{product_id}/reserve — atomically reserve quantity.
  • POST /inventory/{product_id}/release — release reservation.
  • POST /inventory/{product_id}/restock — add quantity.

Step 2: API implementation. InventoryService builds these endpoints, including the concurrency logic (how to atomically reserve under load — typically using row-level locking or optimistic versioning). The endpoints are tested and deployed.

Step 3: Migrate OrderService. OrderService changes its inventory access from direct database queries to InventoryService API calls. The change is tested in staging; integration tests verify the order-placement flow still works. The change is rolled out gradually with feature flags.

Step 4: Migrate WarehouseService. Similar to OrderService. WarehouseService now calls GET /inventory/{product_id} instead of querying the table.

Step 5: Migrate AnalyticsService. AnalyticsService is special because it needs bulk reads. The InventoryService either adds a bulk-read API or — better for high-volume reads — an event stream that AnalyticsService can subscribe to. AnalyticsService builds its own local cache of inventory data, kept current via events.

Step 6: Verify no direct access. Database connection logs are reviewed; only InventoryService should be connecting to the inventory table. Other connections are investigated and migrated.

Step 7: Physical separation. The inventory table is moved to a dedicated database instance owned by InventoryService. CDC tools replicate during the cutover. After cutover, the old shared database loses the inventory table.

Outcome. The inventory table is now owned by InventoryService. Schema changes are made by the InventoryService team without coordination. Other services are unaffected by inventory schema changes (their API/event contracts are stable). The migration took 4 months end-to-end.

This pattern repeats for every shared table. The total estate-wide migration is the cumulative effort.

6. The Explicit Relationship to Distributed Monolith Anti-Pattern

The shared database is the single most common cause of distributed monoliths. The relationship is causal:

  • A shared database creates schema-level coupling.
  • Schema-level coupling forces deployment coupling (schema migrations require lockstep updates).
  • Deployment coupling means services cannot deploy independently.
  • The inability to deploy independently is the defining property of a distributed monolith.

The implication for remediation: if you have a distributed monolith, eliminating the shared database is the highest-leverage remediation step. Other steps (asynchronous messaging, contract testing, feature flags) help but are downstream of the data ownership. Without database ownership, the distributed monolith persists.

Conversely, if you have proper database-per-service but tight synchronous coupling between services, you might still have distributed-monolith pathologies (failure cascades, deploy coordination for behavioral coupling) — but the schema-level coupling is gone, and the remaining issues are more tractable.

The two anti-patterns are not identical, but the shared database is so frequently present in distributed monoliths that they often appear together. The remediation playbook for one usually includes substantial work on the other.

7. Real-World Fixes — Industry Examples

7.1 Shopify’s Modular Monolith — Avoiding the Pitfall

Shopify’s 2019 essay Deconstructing the Monolith (https://shopify.engineering/deconstructing-monolith-designing-software-maximizes-developer-productivity) describes their explicit decision to not migrate to microservices, and instead build a modular monolith. The reasoning: many of the benefits attributed to microservices (clear boundaries, team autonomy, independent evolution) come from modularization, not from physical distribution. Shopify chose to keep one Rails application but enforce strict module boundaries within the codebase, with explicit contracts between modules and tooling that prevented cross-module dependencies that violated the boundaries.

A key architectural decision: each module has its own domain models and the modules communicate through defined APIs (Ruby method calls within the process, but with contracts). The data lives in one database (shared instance) but each module owns specific tables and other modules access them only through the owning module’s API — even though the API is in-process.

The result: Shopify gets the benefits of clear data ownership and modular evolution without the operational cost of a database-per-service estate, and explicitly without the shared-database anti-pattern at the cross-module level (each module owns its tables, even within the shared instance). The modular monolith is a counter-example to the assumption that microservices are the only path to clean data ownership.

7.2 Netflix’s Cassandra-per-Service

Netflix’s microservices architecture went the opposite direction: each service has its own Cassandra deployment. Netflix Tech Blog has multiple posts describing the per-service Cassandra pattern. The benefits: each service can tune its Cassandra cluster to its own workload (different replication factors, different consistency levels, different cluster sizes); failure isolation between services; technology choice within the broader Cassandra family.

The cost: operational complexity of running many Cassandra clusters. Netflix invested heavily in tooling (the Priam project for Cassandra automation; later, internal evolution) to make this manageable.

The Netflix approach demonstrates: database-per-service is operationally feasible at very large scale, but it requires investment in operational tooling. Smaller organizations cannot duplicate the Netflix model directly because they don’t have the operational scale to justify the tooling investment.

7.3 Stripe’s PostgreSQL-per-Service

Stripe’s engineering posts describe a per-service PostgreSQL pattern: each service has its own database (sometimes its own physical instance, sometimes its own logical database within a multi-tenant cluster). Schema changes are owned by service teams; cross-service data is propagated via events. The result is a microservices estate that lives up to its promises — each service can deploy independently, scale independently, and evolve independently.

Stripe’s pattern is more replicable for medium-sized organizations than Netflix’s because PostgreSQL is operationally simpler than Cassandra. Many organizations adopting “database per service” use PostgreSQL or MySQL with multi-tenancy (logical database per service within shared instances) to control infrastructure cost.

7.4 Other Examples

  • Uber: evolved from a shared-database monolith to per-service databases over many years, with extensive use of Cadence (now Temporal) for cross-service coordination.
  • Airbnb: maintains a service-oriented architecture with strong data-ownership discipline; cross-service data via events.
  • GitHub: has historically been more monolithic than many of its peers (one Rails app, one big database for many years), with gradual extraction of services and per-service databases over time.
  • Many smaller companies: went through painful migrations after recognizing their distributed monolith / shared-database pathology, typically taking 18-36 months and significant engineering investment.

8. Worked Example — The Flash Sale Problem

A concrete scenario showing what happens with a shared database under load:

The setup. An e-commerce company has two services:

  • OrderService: handles order placement. Reads inventory.quantity, writes orders.
  • InventoryService: handles inventory management. Reads and writes inventory.quantity.

Both services connect directly to the same Postgres instance. The inventory.quantity column is read and written by both services.

The flash sale. Black Friday at 9 AM. A popular product goes on sale. 10,000 customers attempt to purchase simultaneously.

What happens with the shared database:

  1. OrderService receives 10,000 concurrent purchase requests.
  2. For each request, OrderService:
    • Reads inventory.quantity for the product.
    • If quantity > 0, decrements it and creates an order.
  3. Without proper locking, multiple OrderService instances simultaneously read the same quantity value, both see it as positive, and both create orders. The quantity goes negative.
  4. With pessimistic locking (SELECT FOR UPDATE), OrderService instances serialize on the row. Throughput collapses to ~100 orders/second instead of the desired 10,000.
  5. Meanwhile, InventoryService is also writing to the same column for restocking events. The locks contend across services.
  6. The database becomes a hot-spot bottleneck. Queries time out. Customers see errors. The flash sale is a disaster.
  7. Postmortem: the inventory.quantity column was a coupling point between OrderService and InventoryService. Both were reading and writing it. Coordination via row locks didn’t scale.

The fix with database-per-service:

  1. InventoryService owns inventory.quantity exclusively. OrderService can no longer access it directly.
  2. OrderService calls InventoryService’s API. InventoryService.reserve(product_id, quantity) returns success/failure.
  3. InventoryService internally manages concurrency. It can use optimistic concurrency, distributed locks, or sharded reservation logic — all internal to the service, optimized for the workload.
  4. The API call is the boundary. OrderService doesn’t know how InventoryService manages reservations; it just gets a yes/no response.
  5. Performance optimizations are local. InventoryService can shard inventory by product, scale specific shards independently, use specialized data structures. OrderService benefits from these without being aware of them.
  6. The flash sale works: InventoryService’s optimized reservation logic handles 10,000 requests; OrderService receives reservations and creates orders. The database hot spot is owned by one service; that service handles it.

The eventual-consistency variant. For even higher throughput, InventoryService can use the Event-Carried State Transfer pattern: emit InventoryUpdated events; OrderService maintains a local cache of inventory; reservation is via events rather than synchronous API. This adds complexity (compensations on overselling) but scales much further.

The lesson: the shared database makes the flash sale a coordination disaster. Database-per-service, with the owning service handling concurrency internally, makes it tractable. The architectural pattern matches the operational need.

9. Migration Path — From Shared Database to Per-Service

A typical migration plan:

Phase 1: Audit (1-2 months).

  • Map every table to its conceptual owner.
  • For each table, list every service that reads and writes it.
  • Identify cross-service joins in code.
  • Prioritize: which tables are most-shared and most-changed? (These are the highest-risk and highest-value migration targets.)

Phase 2: Establish ownership (1-2 months).

  • Assign each table to a single service owner. Document the ownership decisions in Architecture Decision Records.
  • Establish a “no new direct access” policy: new code goes through APIs, not direct database access.
  • Add lint rules to enforce the policy in code review.

Phase 3: Migrate non-owners off direct access (3-12 months, longest phase).

  • For each non-owning service that accesses a table, migrate it to use the owning service’s API.
  • This often requires building or extending APIs on the owning service.
  • Migration is per-service, in priority order.

Phase 4: Physical separation (3-6 months).

  • Once logical ownership is established and direct access is eliminated, move tables to separate physical databases.
  • Use Change Data Capture (Debezium) to replicate during the transition.
  • Cut over services to read from their own database.
  • Physical separation is the visible milestone but is mostly mechanical once the logical work is done.

Phase 5: Event-driven cross-service data (ongoing).

  • For scenarios where API calls are too frequent or too slow, introduce event-driven data propagation.
  • Service A emits events on data changes; service B maintains a local copy.
  • Outbox pattern for reliable event publication.

Total timeline: 12-24 months for a moderately-complex estate. The longest phase is migrating direct database access, because every cross-service usage must be addressed individually.

The migration is parallelizable across services and tables, which helps. It requires sustained engineering investment and leadership commitment to avoid being deprioritized.

9.1 Migration Sequence Diagram

sequenceDiagram
    participant Team
    participant SharedDb as Shared DB
    participant SvcA as Service A
    participant SvcB as Service B
    participant Kafka as Event Bus
    participant DbA as DB A (new)
    participant DbB as DB B (new)

    Team->>SharedDb: Audit table ownership
    Team->>SvcA: Establish: Service A owns Table X
    Team->>SvcB: Service B must access Table X via API
    SvcB->>SvcA: Migrate to API calls (no direct DB access)
    Team->>Kafka: Stand up event bus
    SvcA->>Kafka: Publish events on Table X changes (Outbox pattern)
    Kafka->>SvcB: Service B can subscribe instead of querying
    Team->>DbA: Move Table X to dedicated DB
    SvcA->>DbA: Service A connects to DB A
    SharedDb-->>DbA: Migrate data via CDC
    Team->>SvcB: Verify Service B has no direct access to old shared DB
    Team->>DbB: Move Service B's tables similarly

What this diagram shows. The migration sequence. The first step is auditing — mapping current table ownership. Then ownership is established (Service A owns Table X). Service B is migrated off direct database access to API calls. An event bus is added; Service A publishes events on its data changes; Service B can subscribe instead of repeatedly polling Service A’s API. Once direct access is eliminated, the table can be physically moved to a dedicated database for Service A. CDC tools replicate data during the transition. The same process repeats for Service B’s data. The migration is sequential (audit → migrate access → physical separation) but per-table; multiple tables can be in different phases simultaneously.

9.2 Per-Service Database Operational Considerations

Adopting database-per-service has operational implications. The team that succeeds with the pattern invests in:

Per-service schema management. Each service owns its schema migrations. The migration tooling (Liquibase, Flyway, Alembic, etc.) is configured per service. Migrations are part of the service’s deployment.

Per-service backup and restore. Each database has its own backup schedule. Restore tests are per-service. Cross-service restore is not coherent (data may be inconsistent across services); restoration is per-service or system-wide rebuild from events.

Per-service monitoring. Each database has its own monitoring dashboards. Database metrics are tagged with the service name; alerts route to the service’s on-call.

Per-service capacity planning. Each service’s database is sized for that service’s workload. Some services have small databases; others large; the operational cost is per-service.

Cross-service consistency tooling. Sagas, outbox patterns, event-driven projections — the tooling that compensates for the lack of cross-service transactions. This is its own engineering investment.

Database technology choice. Each service can use the database that fits its needs. Operational complexity grows if many different databases are deployed (more expertise required, more tooling).

The operational cost is real but bounded. Teams that have made the investment (Netflix’s per-service Cassandra, Stripe’s per-service PostgreSQL, Uber’s per-service mix) report it pays off in development velocity and resilience. Teams that haven’t made the investment often regret the shared-database shortcut.

10. Common Interview Discussion Points

  • “What is the shared database anti-pattern?” Multiple services accessing the same database tables directly (not through APIs). Creates implicit schema-level coupling; schema changes require multi-service coordination; it’s the most common cause of distributed monoliths.
  • “Why is it bad?” Schema becomes implicit unversioned contract; cross-team coordination for migrations; ambiguous data ownership; cross-service joins create runtime coupling; deployment coupling; scaling and technology choice undermined; failure-domain confusion.
  • “What’s the alternative?” Database per Service: each service owns its data; cross-service data via APIs or events; Saga Pattern for cross-service workflows; Outbox pattern for reliable event emission.
  • “Why does it happen?” Almost always: incomplete migration from monolith. Code split into services but database left shared. Performance arguments are usually wrong. Bounded-context analysis was skipped.
  • “How do you migrate away from it?” Audit, establish ownership, migrate non-owners off direct access (longest phase), physical separation, event-driven cross-service data. 12-24 months for moderate estates.
  • “What about read-only access? Is that fine?” No (see Pitfall 1). Read-only access still creates schema coupling; consumers depend on the schema, schema changes break them.
  • “What if cross-service joins are needed for performance?” Denormalize within services; use event-driven data propagation; service A maintains a local copy of relevant data from service B kept current via events. The cross-service join becomes a local join.
  • “What’s the relationship to distributed monolith?” The shared database is the most common cause. Schema coupling forces deploy coupling, which is the defining property of distributed monolith. Eliminating shared database is usually step 1 of distributed-monolith remediation.
  • “How does Shopify avoid it?” Modular monolith: one Rails application, one database, but strict module boundaries with each module owning specific tables and accessed only through APIs. The data ownership is enforced at the module level, even within shared infrastructure.
  • “What tooling helps?” Change Data Capture (Debezium, AWS DMS) for migration; Outbox pattern for reliable event emission; lint rules for cross-service database access; service registry for tracking ownership.

11. Pitfalls

Pitfall 1: the “read-only is fine” myth. Some teams allow service B to have read-only access to service A’s tables, reasoning that without writes there’s no coupling. This is wrong. Reads still create schema coupling: service B’s queries depend on column names, types, and constraints. A column rename in A breaks B’s read query. The schema is still an implicit unversioned contract. Read-only access is a slightly less bad shared database, not a non-shared database.

Pitfall 2: “we have triggers” coordination layer. Some teams attempt to manage cross-service data via database triggers — service A writes to table X, a trigger updates table Y owned by service B, service B reads from table Y. This makes things worse, not better. Triggers are: invisible (they’re schema-level magic that doesn’t appear in service code), hard to test, hard to debug, hard to evolve. They embed inter-service logic in the database, where neither service team owns it. Triggers as cross-service coordination are an anti-pattern within the anti-pattern.

Pitfall 3: partial extraction with one service migrating away while another still has direct access. Service A is migrated to use service B’s API; service C is left behind, still accessing service B’s tables directly. The schema coupling is reduced but not eliminated. C will fail when B does its next schema migration. The migration must be comprehensive — every consumer migrated — before the schema can be evolved freely.

Pitfall 4: the “stored procedures are the API” argument. Some teams claim that stored procedures defined on the shared database constitute an API and therefore the shared database is fine. This is partially true (stored procedures are an interface that hides schema details from callers) but operationally weak: stored procedures are not versioned, are slow to evolve, are testing-hostile, and lock the team into a specific database vendor. The “stored procedure API” is a poor substitute for a real service API.

Pitfall 5: cross-service replication via dual writes. Service A wants to keep service B’s data updated, so A writes to its own database and to B’s database in the same transaction (or in two transactions). This is even worse than read-only sharing because it creates active coupling: A’s write logic depends on B’s schema. The right pattern is events: A writes to its own database and emits an event; B subscribes and updates its own database.

Pitfall 6: under-budgeting the migration. Database split migrations are routinely under-budgeted. Teams estimate “3 months” and actually need 18 months. The reasons: every cross-service database usage must be migrated; APIs must be built or extended; data must be replicated during transition; tests must be updated; on-call rotations must be re-organized. Budget realistically.

Pitfall 7: ignoring the operational dimension. Database-per-service multiplies operational complexity. More database instances means more backups, more monitoring, more upgrade cycles, more security patching. Without investment in database operational tooling, the per-service database estate becomes its own pain point. Plan for the operational scale-up.

Pitfall 8: not addressing the “shared cache” mini-version. Even after splitting the database, teams sometimes share a Redis cache, an Elasticsearch index, or a S3 bucket. The same anti-pattern dynamics apply at smaller scale. Each piece of state needs a single owner; cross-service access goes through the owner’s API.

Pitfall 9: assuming the migration is one-time. New code can re-create the anti-pattern. Without ongoing discipline (lint rules, code review, ADRs), new shared-database scenarios accumulate. Make the discipline ongoing, not one-time.

Pitfall 10: forgetting the read-replica trap. Teams sometimes give “read-only” services access to a read replica of the shared database, reasoning that read replicas don’t impact production performance. The schema coupling is unchanged; the read-replica access is just as problematic as direct read access on the primary. Read replicas are an availability/scalability optimization, not a decoupling mechanism.

12. Open Questions

  • When is shared infrastructure (one Postgres instance, multiple logical databases) operationally cheaper than per-service instances, and when is the savings worth giving up some isolation? The tradeoff is workload-specific.
  • How does multi-tenant SaaS interact with database-per-service? Per-service-per-tenant explodes the database count; the answer often involves logical-database-per-service patterns within shared instances.
  • What is the right level of “shared infrastructure, separate logical databases” to allow without falling into the anti-pattern? Some teams strict, others permit shared instances; the decision depends on operational maturity.
  • How does the rise of “database-as-a-service” (managed Postgres, managed MySQL) change the per-service-database economics? Lower operational cost may make the discipline easier to maintain.
  • What is the role of distributed SQL databases (CockroachDB, YugabyteDB, Spanner) in this discussion? They offer cross-shard transactions but encourage “logical separation within one database,” which can drift toward sharing.
  • How do data lakes / lakehouses fit into the architecture? Often, “the lake” is shared across services for analytics, but the operational data is per-service. The boundary between operational and analytical data is its own architectural question.

12.11 The Lessons from Industry Failures

Several public post-mortems and engineering blog posts have documented teams discovering the shared-database anti-pattern in their own systems and the cost of remediation. The recurring lessons:

Lesson 1: the migration is always longer than estimated. Teams initial estimate “6 months”; the actual timeline is 18-24 months. The reasons: every cross-service usage must be migrated; APIs must be built; tests must be updated; ownership must be established; data must be replicated. Each step is non-trivial.

Lesson 2: leadership commitment is essential. Without sustained leadership commitment, the migration is deprioritized for feature work. The migration must have explicit funding and protected engineering time. Teams that try to do “20% of engineering capacity” cleanups typically fail.

Lesson 3: tooling investment pays off. CDC tools (Debezium, AWS DMS), schema-management tools, lint rules to enforce service-data ownership — all pay back during the migration. Teams that try the migration without these tools spend more engineering time and hit more issues.

Lesson 4: ownership decisions are political. “Who owns this table?” is sometimes contested. The decision affects which team has on-call responsibility, which team is the contact for schema changes, which team is empowered to evolve the data. Resolving these decisions requires leadership intervention; technical teams alone cannot decide.

Lesson 5: events solve more problems than they create. Once event-driven cross-service data propagation is set up, many of the problems that motivated the shared database (cross-service joins for performance) become non-issues. The local data copies (kept current via events) are faster than database joins anyway.

Lesson 6: re-accumulation is constant. New code can create new shared-database scenarios. After the cleanup, the team must maintain ongoing discipline (lint rules, code review, ADRs) or new mud will accumulate.

Lesson 7: incomplete migrations are often worse than no migration. Half-migrated estates have inconsistencies and unclear ownership. The cleanup must be carried through to completion or it’s a partial mess.

These lessons recur in industry post-mortems with consistent shape. They are predictable; they are also routinely under-anticipated by teams beginning the migration.

13. See Also

Graph View

Backlinks