Maaz Zindani

Loading the index…

↑↓ move↵ openesc close4072 pages

Help for owners

My site goes down every time I release an update

Short answer

Releases cause downtime when the old version is stopped before the new one is ready. The fix is to start the new version first, check it works, then switch traffic across. Do not release at peak hours while this is unsolved, and make sure a way back exists.

  • Written for business owners
  • Updated
  • By Maaz Zindani, Senior Software Engineer

Does this sound like your situation?

  • Customers see errors for a few minutes after every release.
  • I only dare to release at night.
  • Users get logged out or lose a basket when an update goes out.
  • The last update broke the site and rolling back took hours.
  • A release means stopping the app and copying files by hand.

Why does this happen?

  1. The old version is stopped before the new one is ready

    A simple release stops the app, replaces files and starts it again. Everyone who visits in between sees an error.

  2. Servers can be told to hand over gradually

    nginx documents that on a configuration reload the master process checks the new configuration first, and if it fails it continues with the old one. Old worker processes finish serving existing clients before they shut down. That same idea is used for application releases.

  3. Database changes lock the app

    A change to a large table that the old and new versions both use must be done in steps, not in one jump.

  4. Releases are manual

    A person copying files at night makes mistakes. A scripted release is the same every time.

What can I do about it today?

  1. Release outside your busiest hoursLook at when customers are least active, and keep releases there.
  2. Tell customers about planned downtimeA short notice on the site is better than an unexplained error.
  3. Keep the previous versionBefore every release, make sure the old files and a database backup exist, so you can go back.
  4. Release small changes more oftenSmall releases are easier to undo than a big one every few months.
  5. Write down the steps a release takes todayEven rough notes help plan an automatic version.

When should I bring someone in?

  • Downtime during a release costs you orders.
  • Only one person knows how to release.
  • Database changes are involved.

How do I handle it?

  1. Map the release as it isI follow what happens at release time and where the break comes from.
  2. Start the new version beside the old oneThe new version runs, and is tested, before it takes any traffic.
  3. Switch traffic when the check passesA health check decides when the switch is made, and the old version keeps serving until then.
  4. Change the database in two stepsChanges are made compatible with both versions first, and cleaned up later.
  5. Prove the rollbackI test going back to the previous version so it is not a guess on release day.

The price depends on what I find, so I quote in writing after a short discovery call. Tell me what is happening and I reply within 24 hours.

What do owners ask about this?

Is zero downtime always possible?

Not for every app. Some changes need a short break, and I tell you which before it happens.

Does this need a bigger server?

Often two copies of the app must run briefly, so it may need a little more room. I check it first.

Do I need Kubernetes?

No. A small app can switch versions with simpler tools.

Sources

How was this page made?

I drafted this page with AI assistance from the official documents listed under Sources. Found a mistake? Tell me.

For your developer: the technical fixes

Next step

Want this off your desk?

Send me a few lines on where things stand. I reply within 24 hours.