Human in the loop AI: your agents wait on supervisors

Human in the loop sounds like a safeguard. On a shift site it is also a capacity question. Before an AI agent goes live, count the minutes of review it will hand your shift supervisor, hour by hour, and set them against the minutes that supervisor is actually free. If the two lines don't meet, the agent's speed buys you nothing: its work sits in a queue until someone on the floor gets back to a screen.

This post gives you a one-night review count you can fill in for your own site, a worked example from a depot night shift, and three changes that shorten the queue without hiring anyone.

What the Anthropic numbers say, and what they don't

On 16 October, Anthropic published how employment is changing at Anthropic, using its own staff's data. One figure has been quoted everywhere since: in interactive sessions, its agents spent around 80% of the time idle, waiting for a person to respond. Anthropic measured this by subtracting each agent's active working time from the session's total time. The report's own conclusion is that human attention is now the bottleneck for agent output.

That is a software company, measuring people who sit at a screen all day. Nothing in it describes a depot, a production line or a field crew, and it would be wrong to carry the 80% across. What carries across is the shape of the problem, and on a shift site it is worse. The person who has to say yes to the agent's work is also the person walking the floor, covering a break, and answering the radio. They get to a screen at a few points in the shift, not whenever a request arrives.

Count the review minutes, not the requests

The usual question before go-live is "how many things will need approving?" It is the wrong unit. A cover offer takes a supervisor a minute or two to check; an agency booking takes longer, because they look at the rate, the person's induction and the start time. Count minutes.

Then count when the supervisor is actually free to review: the start of the shift, any planned quiet hour, the end. Be honest about it. A supervisor who is "always reachable" on a radio is not free to read a request properly.

A worked night: the depot that looks fine on paper

The figures below are illustrative: a distribution depot's night shift, 10pm to 6am, with one shift supervisor. An agent drafts four kinds of decision the supervisor must approve.

Across the night the supervisor has 70 free minutes: 10 at the start, 5 in each of the quiet hours, 15 in the 2am lull and 20 before handover. Sixty minutes of review, seventy minutes free. The plan says it fits.

Now spread both across the night, hour by hour.

The agent raises most of its work early, when the trailers are being unloaded and the shortfalls show up. The supervisor's free time comes later. Carry the leftover from hour to hour and the queue is never empty from 10pm until the 5am window clears it: 2 minutes of review waiting at 11pm, then 7, 10, 11 at 2am, down to 2 after the lull, then 5 and 6 before handover. Work raised around midnight is mostly not looked at until the 2am window.

Whether that matters depends on what is waiting. A carrier slot request that sits for two hours may mean the slot has gone. An overtime offer that sits until 2am is an offer made to someone who has already gone home.

Three changes that shorten the queue

Work through these in order. Each one is a decision you can make this week without new software.

1. Turn the repeat approvals into a written rule. Look for the approval the supervisor gives the same answer to nearly every time. In the example it is the damaged-stock write-off: eight a night, two minutes each, almost always yes. Write the rule down with a limit, such as "write-offs below the agreed value approve themselves, are logged, and the supervisor reads the log at handover". In the chart, the middle bars show the night with that rule in place: 16 review minutes gone, and the queue peaks at 4 minutes instead of 11.

Not every approval should go. Keep a person on anything that changes someone's pay, hours or safety, and anything where the answer genuinely varies.

2. Batch what can wait, and say so. Some approvals cost nothing if they wait until the next window. Mark those as "next window" and let them queue on purpose. The cost of a queue only matters for the items that lose value while they wait.

3. Give the urgent approvals a second pair of hands. For the items that go stale (in the example, carrier slots and same-night cover), decide who else may approve them when the supervisor is on the floor: a team lead with a set limit, or the on-call manager. Put that person's name on the rota for the hours it applies.

Fill in your own night

Use your worst ordinary shift, not your best. One sheet per supervisor role.

1. List every decision the agent will hand a person. Name the approval, not the agent.
2. Per night, minutes each, review minutes. Time a handful of real approvals with a watch; guesses run short.
3. When they arrive. Mark the hour each kind usually appears.
4. Supervisor free minutes by hour. Ask the supervisor, then watch one shift to check.
5. Run the hour-by-hour carry. Leftover from one hour adds to the next. Note the biggest queue and when it clears.
6. Cost of waiting, per approval type. What is lost for each hour it sits?
7. Decide per type: rule, batch, or second approver.

If the biggest queue is more than one hour's worth of free time, or a costly approval waits more than an hour, fix the sheet before the agent goes live.

Where Fivexer fits, and where it stops

Fivexer puts the work an agent hands to people in the same queue as everyone else's: agents take what they can, each person step goes to someone qualified to take it, and every choice has a recorded reason. A crew lead can pull one view of their team, with counts and parked work, so the supervisor sees what is waiting the moment they get to a screen.

It does not count review minutes or forecast when the agent's requests will arrive. The sheet above is still paper, or a spreadsheet, and it should be filled in before you decide anything about software. Fivexer also doesn't decide which approvals are safe to turn into rules. That call belongs to whoever answers for the outcome.

Count one night this week

Pick the agent you are closest to switching on, fill in the sheet for one supervisor's worst ordinary night, and run the hour-by-hour carry. When the queue clears within the hour, see how Fivexer puts agents and people in one queue.