Incident response: the first five minutes after the alert

A monitor gives you minutes. What you do with them was decided earlier, or it was not decided at all. Across the incidents in this series the time lost was rarely between the chain and the alert. It was between the alert and someone with the right key doing the right thing.

Some timings, all from public post-mortems and our own on-chain checks¹:

• IoTeX ioTube, February 2026. The first sign was an OwnershipTransferred at 01:07:35. The team found out at 08:01, almost seven hours later. Nearly all the damage happened in that window.
• Orbit Bridge, December 2023. The vault was drained in five transfers over 17 minutes. The bridge was deactivated about an hour after the first transfer.
• Ronin Bridge, August 2024. The team had a pause switch. It was used 38 minutes after the first withdrawal, 86 minutes after the upgrade that caused it.
• WazirX, July 2024. A failed call to the Safe came 3 to 4.5 minutes before the Safe was upgraded to the attacker's code. After that, the owners had no control left.
• PlayDapp, February 2024. The team had 66 hours between two waves and could not stop the second: the token's roles could not be revoked.

Four of the five are about people and design, not about detection. This part is about those four.

What the first five minutes look like

Take an alert that says "ownership of the vault changed" at 02:14. The five minutes should go roughly like this.

The failure points are not mysterious. Nobody sees the message. Two people see it and each assumes the other has it. The person who sees it cannot do anything. The person who can, is asleep without notifications. Everything below deals with one of these.

1. Know who holds the pause key

Write one line per contract: what stops it, and who can do that right now.

Fill the blanks honestly. Typical answers are uncomfortable:

• The pause key is in the same multisig as upgrades, and needs five signers, and four of them are asleep. Many teams split it: a narrow pause role held by a small group or a single hot key, and the broad upgrade authority behind a slow multisig. Pause is a lever you want to be easy to pull, and it should not be able to move funds. Ronin had exactly this, a separate PauseEnforcer with one triggerPause() call.
• There is no pause. Then the lever is something else: revoke a role, remove a minter, upgrade the code back. If it is none of these, the lever is the next section.
• The lever does not exist in the code. PlayDapp's token used OpenZeppelin 2.x roles, where you cannot revoke another account's minter role. The team removed its own roles, the attacker kept theirs, and the fix was migrating the token². WazirX had no way to take the Safe back once it was upgraded³. Check this before launch. Afterwards it is a migration.

SmartAlert checks this for you: the contract page has a block "What you can do in an incident" with pause, owner (one key or a Safe N of M), who can upgrade the code, and an OpenZeppelin 2.x minter that cannot be revoked. Under each warning there is a rule to set up.

2. Route alerts to people who can act

An alert in a channel nobody reads is a log. Decide per rule where it goes.

• Critical alerts, to a channel that makes noise at night: ownership, upgrade, roles, pause, state rules. Telegram is free. Slack, Discord, webhooks, PagerDuty and email are on paid plans.
• Warning alerts, to a channel read in working hours: low gas runway, failed Safe transaction, unusual rates. These matter but do not need a phone call.
• Put the right people in it. The alert is useful only to someone who can answer "is this ours?" and someone who holds the key. If those are different people, both need the channel.
• Escalate if nobody takes it. On Pro and above, a rule can have one escalation step: if nobody acknowledges the alert ("I'm on it") within 5, 10, 15, 30 or 60 minutes, it goes to a second channel, for example PagerDuty or a different person. For a critical alert we would choose 5 or 10 minutes. Acknowledging stops the timer and records who has it.

Do not send everything to the pager. A channel that fires on every spike trains people to mute it, as in part 11. Few rules, each with a clear action, are read.

3. Freezing the money downstream

Some incidents give no lever: the key is gone, the Safe is not yours, the drain happened in one transaction. What remains is to make the stolen money hard to cash out.

• Exchanges. Stolen funds usually go to an exchange or a bridge to be converted. Exchanges can freeze a deposit, if they hear in time and trust the report. Have a security contact at the exchanges your users use, and send the attacker's addresses.
• Issuers of centralized tokens. The issuers of stablecoins like USDT and USDC can freeze an address. They act on a verified request, so you need to know how to reach them before you need to.
• Security responders. SEAL 911 helped Ronin with its response⁴. Know the channel to such a group.

At UXLINK, the owners were removed 1 to 2.5 minutes before the first withdrawal, so by the time an alert could arrive the legitimate owners were out⁵. For such cases, calls to exchanges and issuers are what the team can still do. Keep a list in a place you can open at 3 a.m.: names, a contact for each, what to send (addresses, transaction hashes, amounts).

4. A night test

Everything above is a theory until it runs once.

1. Test the channel. In SmartAlert, each channel has a test button that sends a test message. Press it for every channel and every person. It tells you the address and the token are right, and nothing more.
2. Run a drill at night, without warning. Make a harmless alert fire: for example, a rule on your own test contract with a threshold you cross on purpose. Note the time. Who saw it, how long until "I have it", how long until someone with the key confirmed they could pull the lever.
3. Fix what broke. The usual findings: phone in silent mode, the person who can pause is on a plane, nobody knows where the guardian key's device is.

SmartAlert has no built-in "drill" feature, so the test is a rule you set up for the purpose and remove afterwards. Do it once a quarter, and after anyone joins or leaves the on-call group.

What the first five minutes cannot fix

If everything happens in one transaction, there are no five minutes. See part 8: for that you need prevention, not response.

How to catch it

Alerts that start a response, with the severity we would use:

1. Admin events, Critical: ownership-transferred, role-granted, proxy-upgraded, safe-owners-changed. To a channel that is read at night, with an escalation step at 5 or 10 minutes.
2. State, Critical: contract-state-paused and a state rule on each safety parameter (part 10).
3. Outflows, Critical: large-transfer-usd and treasury-below-usd. These tell you the size of the loss, often after the first money has left.
4. Safe, Warning or Critical: a rule on failed execTransaction and "allowed senders" on ExecutionSuccess (part 9).
5. Keepers, Warning: keeper-gas-runway, low-balance-no-topup.
6. Run the night test above.

The whole series

Block 1. How a contract is built

1. What a smart contract actually is
2. Owners, roles and admin keys
3. Upgradeable proxies
4. Multisigs and Safes

Block 2. How contracts are broken

1. Admin key compromise: IoTeX, PlayDapp, Zoth
2. Why bridges are the biggest target
3. Oracle and price manipulation, flash loans
4. Single-transaction exploits and what monitoring cannot do

Block 3. How to watch

1. What to monitor after an audit
2. Events vs state
3. Fixed thresholds vs an hourly norm
4. Incident response (this part)

The short version: find the doors, watch who uses them and what changes without an event, set limits that follow the contract, and decide in advance who pulls the lever.