Adding Alembic to a ShipKit Project: The Migration Layer It Doesn't Ship

Last week's deploy guide ended with a deliberate refusal: ShipKit ships no migration tooling — no Alembic, no migrations directory, no upgrade command — and evolving your schema once real data exists is your problem, which that guide declined to solve. We re-checked before writing this one, because this article's entire premise depends on it: still true, in both tiers, today. So this is the follow-through. It is also written in an unusual order, because the tutorials already on the internet all start from alembic init on an empty database, and that is not where you are. You have a ShipKit app that has been running for weeks or months. Base.metadata.create_all built your tables, users are in them, and the dangerous part of adding Alembic is not installing it — it's the first ten minutes after, where one wrong command tries to re-create tables that exist, and the first careless autogenerate offers to drop things you love.

So the retrofit comes first, the greenfield case falls out of it for free, and the MySQL section is not optional reading — ShipKit is MySQL, and a guide written from Postgres habits produces migrations that fail in ways Postgres never taught you.

The Shape of the Problem

Alembic's model of the world is a chain of revisions plus a one-row table (alembic_version) in your database recording which revision that database is at. Every environment's schema is then “its position in the chain.” Your problem is that your database has a schema but no position — it was built by create_all, outside the chain. The retrofit is the act of giving your existing schema a position without re-running its creation. Get that one idea straight and every command below is obvious; skip it and the commands look interchangeable when they are not.

Step 1: Install and Wire It to ShipKit

In your project venv:

pip install alembic
alembic init alembic

That creates alembic.ini and an alembic/ directory. Two edits wire it to ShipKit. First, in alembic.ini, leave sqlalchemy.url alone — we will inject the real URL from ShipKit's own settings so there is exactly one place your database credentials live. Second, edit alembic/env.py: near the top, import ShipKit's metadata and config, and point Alembic at both. ShipKit's layout puts the SQLAlchemy Base in app/db.py and settings in app/config.py, so:

# alembic/env.py — additions near the top
import sys
from pathlib import Path

# Make the project importable when running `alembic` from the repo root
sys.path.insert(0, str(Path(__file__).resolve().parents[1]))

from app.config import get_settings
from app.db import Base
import app.models  # noqa: F401 — imported so every model registers on Base.metadata

config = context.config
config.set_main_option("sqlalchemy.url", get_settings().database_url)

target_metadata = Base.metadata

Two lines there do quiet, load-bearing work. The import app.models looks unused and is not: models register themselves on Base.metadata at import time, and if nothing imports them, autogenerate compares your database against an empty metadata and proposes dropping every table you own. And set_main_option reads the URL from ShipKit's get_settings() — the same mysql+pymysql:// URL your app uses — so Alembic and the application can never disagree about which database they mean.

Step 2: The Baseline — and the Trap

Now give the chain a first link that represents the schema you already have. The clean way is to autogenerate it against an empty scratch database, not your real one: create a throwaway schema locally, point DATABASE_URL at it for one command, and run:

alembic revision --autogenerate -m "baseline: full ShipKit schema"

Against an empty database, the diff between “nothing” and your models is your entire schema, so the generated file is a complete, honest record of every table — users, password reset tokens, all of it. Review it, keep it, and from now on a brand-new environment is built with alembic upgrade head instead of create_all.

Step 3: Distrust Your First Real Autogenerate

Here is where retrofits actually get hurt. Weeks later you add a column to a model and run alembic revision --autogenerate against the real database, expecting a one-line diff — and the file that comes out proposes your new column plus a handful of changes you never made. Autogenerate is not wrong; it is comparing honestly. It's your history that lies: any hand-run ALTER TABLE, any index added in a console at 2am, any column type MySQL normalized silently — all of it surfaces now, as a difference between models and database, rendered as operations. And the operations that reconcile “database has something the models don't” are drops.

So the standing rule, forever: read every autogenerated migration before applying it, and treat any drop_column, drop_table, or drop_index you didn't explicitly intend as an investigation, not an instruction. Usually the fix is to update the models to match reality (that index was a good idea — write it into the model) and regenerate, rather than letting the migration “correct” the database back to a fiction. Autogenerate is a diff tool wearing a code generator's clothes; the judgment stays with you.

The MySQL Section (Do Not Skip This One)

ShipKit is MySQL — the models are written against it and the default URL is mysql+pymysql:// — and Alembic on MySQL differs from the Postgres experience most tutorials assume in three ways that matter.

1. DDL does not roll back. Ever. On Postgres, a migration that fails halfway rolls back as a unit and you fix it and rerun. MySQL cannot wrap DDL in a transaction: a migration that fails on operation three of five leaves operations one and two applied, with the version table saying you're still on the old revision. There is no automatic way back — you repair by hand. The discipline that follows is not optional style: keep each migration to one logical change. Five small migrations that each either fully apply or trivially fail beat one heroic file every time, and this is the single biggest habit Postgres refugees have to unlearn.

2. Type reflection produces noise, and Booleans are the classic case. ShipKit's models use Boolean columns like is_active and is_admin; MySQL stores them as TINYINT(1), and depending on your Alembic version's type-comparison settings, autogenerate may “notice” that difference and propose pointless type changes. The same goes for a few other reflected types. Apply the same skepticism as with drops: a type-change op you didn't cause is noise to be deleted from the migration until proven otherwise, not a change to apply to production.

3. New NOT NULL columns need a server default at migration time. ShipKit's models set defaults in Python (default=True, default=datetime.utcnow) — the application supplies the value, the database schema carries no default. That's fine day to day, but it means adding a non-nullable column to a table with existing rows will fail or misbehave, because MySQL has millions of rows and no value to put in them. The pattern that works is to hand the migration a temporary server_default, and drop it afterward if you want the schema to stay Python-default-only:

def upgrade() -> None:
    op.add_column(
        "users",
        sa.Column(
            "marketing_opt_in",
            sa.Boolean(),
            nullable=False,
            server_default=sa.text("0"),  # backfills existing rows at ALTER time
        ),
    )
    # optional: keep DB defaults out of the schema, ShipKit-style
    op.alter_column("users", "marketing_opt_in", server_default=None)


def downgrade() -> None:
    op.drop_column("users", "marketing_opt_in")

One more MySQL reality, stated briefly because we've covered it in depth before: ALTER TABLE on a large table can lock it while it rebuilds, and “the migration worked in staging” says nothing about a table a hundred times bigger in production. When tables get big enough for that to matter, the techniques in our zero-downtime migrations guide — expand-and-contract above all — are the next skill up, and everything here is compatible with them.

Where This Meets Your Deploy Pipeline

Last week's guide made an argument we are not going to un-make this week: migrations and code deploys are two steps, migration first, because they have different failure modes and different rollback stories — code rolls back by redeploying the old build; a MySQL migration, as covered above, sometimes doesn't roll back at all. So resist the tidy instinct to add alembic upgrade head as a line in the CI job that guide built. The pipeline stays a code pipeline. Migrations run as their own deliberate act — run the migration, verify the schema, then ship the code that needs it — and the app you deploy in between must tolerate the new schema before the new code lands, which the expand-and-contract pattern in the Alembic guide exists to guarantee.

The Bottom Line

ShipKit not shipping a migration layer is a real limitation and we have never pretended otherwise — it's in the fit guide in bold. But the layer is genuinely yours to add in an afternoon: one install, one wired env.py, one baseline, one stamp. The afternoon's only real dangers are the two commands that look alike — stamp declares, upgrade executes — and the first autogenerate's confident offer to delete your drift. Read the diff, keep migrations small, run them before the code ships, and the schema stops being the scary part of the project. If you're earlier in the journey than this — no server yet, no data yet — start with the getting-started guide and come back when there's something worth migrating.