An automation that has run correctly for six months is not a finished thing. It is a thing that has not been disturbed yet.
This is the part of the work that gets discussed least, because it is not where the interesting engineering is and it does not demo well. It is also where the money quietly goes — not in the build, but in the six weeks after something broke silently while everybody downstream kept working from bad output.
The five things that break
Something upstream changed shape. The most common cause, and it almost never arrives as an error. A supplier adds a column to their export. A form gets a new field. A system starts sending dates as year-month-day where it used to send day-month-year. Everything continues to run. The output is subtly wrong, and it stays wrong until a human notices a number that looks off.
An API changed or a version was deprecated. This one at least tends to fail loudly, which makes it the least dangerous of the five. The variant worth watching for is the deprecation that ships as a warning for months and then becomes an error on a date nobody wrote down.
A credential expired. Tokens rotate. OAuth grants get revoked when somebody leaves and IT tidies up their account. Password changes cascade further than the person changing them expects. This is the most common cause of an automation that "just stopped," and the fix takes minutes once you know — the expensive part is the three days before anyone realised.
Volume moved. A workflow built around forty items a day meets four hundred during a busy month and hits a rate limit, a timeout, or a queue that never drains. Nothing is wrong with the logic. It was correct at the volume it was built for, and nobody restated that assumption when the volume changed.
Somebody changed the process and did not say. A team starts using a field for something new. A step that used to happen in one system now happens in another. The automation faithfully continues to implement the old process, which is now the wrong one. No technology detects this. Only a person who talks to the team does.
Only one of those five produces an error message. That is the whole problem with reading "it hasn't alerted" as "it's working."
Loud failure beats correct-looking output
The design decision that matters most is where a workflow chooses to stop.
The instinct when building is to make things resilient: catch the error, apply a sensible default, keep going. In an automation that writes to a system of record, that instinct is close to backwards. A workflow that fails cleanly and tells someone costs an hour. A workflow that carries on with a plausible default writes bad data into a system other things read from, and you find out weeks later, from the far end, usually from whoever trusted it most.
So: validate what arrives before acting on it, and treat anything unexpected as a stop rather than a shrug. A run that halts and reports is not the automation failing. It is the automation doing the most valuable thing available to it — being the first to notice.
What monitoring actually has to catch
"Did it error" is the easy half, and most setups stop there. The other half:
- Did it run at all? A scheduled job that stops firing produces no errors, because it produces nothing. Absence has to be alertable, which means something has to know what "on time" looks like.
- Did it process a plausible number of things? Zero invoices on a Tuesday is worth surfacing. So is four hundred when it is normally forty.
- How long since a human looked at the output? Not the logs — the output. For anything with a judgement step in it, this is the only check that catches quiet degradation.
None of that is sophisticated. All of it is the difference between finding a problem yourself and hearing about it from a client.
The ownership question
Every automation needs a name attached to it: someone who gets told when it stops, and who has the authority to decide what happens next. This sounds administrative. It is the strongest single predictor of whether a system is still running usefully two years on.
The failure pattern is consistent. Automation gets built for a team, works well, and the person who commissioned it moves on. It becomes infrastructure nobody owns. It keeps running. It keeps being trusted. It stops being checked. When it eventually breaks, the people relying on it do not know it well enough to know it has broken.
Documentation helps, but not the kind people usually write. What survives a handover is not how it works — it is what it is for, what it touches, and who to tell. Three sentences of that beat a diagram nobody opens.
What we do about it
Every system we build ships with error handling that stops rather than guesses, alerting that covers silence as well as failure, and a written description of what the thing is for and what it depends on. Not out of thoroughness — because we have been the ones called in to work out what an undocumented workflow was supposed to be doing, and it is a bad way to spend a week.
For systems we maintain, that becomes a standing arrangement: someone is watching, and the reporting says what ran, what did not, and what changed underneath it.
If you have automations running that nobody has looked at in a while — built by someone who has since left, or by a vendor you no longer work with — an audit is usually the cheapest useful thing you can commission. Tell us what you have and we will tell you what we would check first.