Assigning agents

Fanning out

A job has a list of agents and at every scheduled time it runs on all of them. Each agent gets a dispatch of its own and makes a run of its own, with its own output, state and exit status; in the history you see them grouped as one occurrence, because they share the tick that made them. That is how one "renew certificates" job serves twenty machines and you still know which of them failed.

What an agent added to a job does about the occurrences that came before it is nothing: a new agent runs from the next tick. An agent removed from a job finishes the run it is holding and is given no more.

One run per job per agent

The same job never runs twice at once on one agent. When a tick arrives while the previous run on that agent is still going — pending, leased or running — the new occurrence is skipped for that agent; the job's other agents are untouched and run. The reverse does not hold: different jobs may run on one agent at the same time, up to AGENT_MAX_PARALLEL and the ceiling from the plan.

The database holds this rule, not a process's memory: a partial unique index over active runs refuses the second row. So it survives a restart of Runjet and two ticks arriving together, and running by hand does not get around it either — on such an agent it reports a run of this job is already in flight on this agent.

A skipped occurrence is not a run and is not in the history. A five-minute job whose run takes twelve minutes therefore runs every fifteen, not every five, and the only trace of the skipped ticks is the gaps in the history. When this is happening, the answer is a longer interval or a faster command, not a second agent — that one would run alongside.

A skipped agent

A tick skips an agent for three reasons, and each has its own sentence, which you will see at running by hand:

An agent that is offline — registered, enabled, but not calling in — is not skipped: the run is created as pending and waits for the agent to take it, for a bounded time. If nobody comes, it ends as skipped.

A job with no agent

An enabled job must have at least one agent; the form will not save it otherwise. A job whose only agent was deleted stays enabled with nowhere to run — the job list marks it and the dashboard shows it among the problems. Give it another agent, or switch it off.