Maaz Zindani

Loading the index…

↑↓ move↵ openesc close4072 pages

Help for owners

When something breaks in my live app there is no safe way to fix it fast

Short answer

An emergency fix process means agreeing in advance who decides something is urgent, who makes the smallest possible change, how it is checked, how it reaches the live app and how it is undone if it makes things worse. Having that written down turns a panic into a routine.

  • Written for business owners
  • Updated
  • By Maaz Zindani, Senior Software Engineer

Does this sound like your situation?

  • When something breaks everyone panics and nobody knows who does what.
  • A quick fix last time created a second problem.
  • Fixes are made directly on the live server.
  • The fix works for a day and then disappears at the next release.
  • Nobody tells customers or staff what is going on.

Why does this happen?

  1. Nobody decided what urgent means

    Without a shared definition, every issue is urgent or none is. Real emergencies get mixed up with ordinary requests.

  2. The fastest route skips the checks

    Editing the live server by hand is quick, but it skips review and testing, and the change may not exist in the code afterwards.

  3. The fix is bigger than the bug

    Under pressure developers often fix several things at once. The more changed, the more can go wrong.

  4. No one owns the decision to release

    If nobody has authority to say go, the fix waits for approval or is released without it.

  5. There is no way back

    If the fix makes things worse and the previous version cannot be restored quickly, a small problem becomes a long one.

What can I do about it today?

  1. Write down who to callA short sheet with the name and phone number of the person who decides, the developer who fixes and the person who tells customers.
  2. Define an emergencyAgree a few examples: customers cannot pay, data is lost, private data is exposed, the whole app is down. Everything else goes through the normal list.
  3. Ask for the smallest possible fixDuring an incident, ask the developer to change as little as possible, and do the tidy-up later.
  4. Ask that fixes go through the codeEven urgent changes should be saved in the repository, so the next release does not undo them.
  5. Prepare one message for customersHave a short, honest notice ready that says something is wrong and being worked on, without technical detail.

When should I bring someone in?

  • The app is down or customers cannot pay right now and your developer is unreachable.
  • Fixes keep disappearing or making things worse.
  • You have no developer who can be reached out of normal hours.

How do I handle it?

  1. Triage firstI confirm what is affected, who is affected and whether it is a true emergency, before changing anything.
  2. Make the smallest safe changeI change only what the problem needs, on a separate branch, and keep it out of the rest of the planned work.
  3. Check, release, watchThe fix is tested on the exact case that failed, released by the normal route, and watched after release.
  4. Keep a way backA backup is taken before the release and the previous version is ready to put back.
  5. Write it up afterwardsA short plain note records what broke, why, and what stops it happening again.

The price depends on what I find, so I quote in writing after a short discovery call. Tell me what is happening and I reply within 24 hours.

What do owners ask about this?

Should I fix urgent bugs on the live server?

It is faster but risky. Changes made there can vanish at the next release and are not reviewed. The change should end up in the code.

Who should decide when something is an emergency?

One named person on the business side, with agreed examples, so developers do not have to guess.

Can an emergency process stop outages?

It does not prevent every outage. It makes the response calmer, shorter and safer.

How was this page made?

I drafted this page with AI assistance. It is general guidance for business owners, not advice for your own system. Found a mistake? Tell me.

Next step

Want this off your desk?

Send me a few lines on where things stand. I reply within 24 hours.