Earning trust in stages: when a routine moves from checked to spot-checked

A new routine arrives, and you check every result. Three weeks later you are still checking every result, because nobody ever said when you could stop. The checking has become a chore of its own, and you could not say what it would take for you to trust the routine.

Trust in a routine, whether a colleague, a script or an AI assistant runs it, is easier to extend when the rules for extending it are written down before the first run. A simple plan has three stages of checking and a short list of what moves a routine up or down. The examples are illustrative.

Stage 1: Check everything

For the first few runs, open every result and check it against a short list: the count, the period and three spot facts. Stay here until you have seen three clean runs in a row, and make sure at least one of them had an awkward input, such as a holiday week or an empty source. A clean run on an easy week proves very little. Keep a one-line log of what you found on each run, even when the answer is "nothing". The log is the evidence that moves the routine up.

Stage 2: Check a sample

Look at a fixed sample instead: the first item, the last item and two picked at random. Read the count and the run time on every run. The sample should be small enough that you really do it. A sample you skip is a stage you have already left. Vary the random picks from run to run, so that the same two items are not always the ones you see.

Stage 3: Check the signals

Read only the signals: the count, the run time and any exceptions the routine reports. Open the results themselves when a signal looks odd, and on a regular day you choose, such as the first Monday of the month. At this stage the routine is trusted to run, and you are watching for changes, not reading every line. Put the monthly look in your calendar, because the day you skip it is the day the signal stops being checked.

What counts as clean

Decide in advance. A clean run might mean no corrections needed, a count within the expected range and delivery on time. Say what a single correction does: some people let one small fix pass, and others send the routine back a stage. Either rule works as long as it is written down before the first miss, when mood would otherwise decide. Count a late run as unclean even when its content is perfect, because lateness is a fault too.

What sends it back

Anything that changes the ground sends a routine back a stage: a change to the source, a new person or tool running it, an edit to the brief, a mistake that mattered or a long gap since the last run. After a change, the old evidence no longer describes the new situation.

Write the stage where everyone can see it

Put the current stage at the top of the brief, or in the routine's name: "Stage 2, since 14 October". Anyone who picks up the routine then knows how much checking it has earned, without having to ask you. When the stage changes, add the date and the reason to the log.

A hypothetical routine

A weekly digest starts at Stage 1. After three clean runs, one of them over a short holiday week, it moves to Stage 2, and the sample takes two minutes. Eight weeks later the source system is renamed and the routine drops to Stage 1 for three runs, then moves up again. The plan did not prevent the rename. It made the response to it routine.

Where Octuo fits

We're the Octuo team. Octuo is a personal assistant and one starting point for work across AI, tools and specialist services. It operates external solutions on your behalf, you grant the permissions, and it reports back when a job is done. How much to check at each stage is a decision worth writing down before you hand a routine over. Octuo is available for macOS.