Concourse conformance
steps makes specific, scattered claims that it "mirrors Concourse" or matches "Concourse's model" — in code comments (internal/config, internal/resource, internal/pipeline) and in docs. Until 2026-07-23, none of these claims had ever actually been checked against real Concourse behavior. One of them was wrong: internal/config/config.go's Step.Version doc said version: every "mirrors Concourse's get.version field," but internal/pipeline/pipeline.go's runGetStep aborted its entire version fan-out on the first version whose build failed — real Concourse's version-selection cursor (atc/db/versions_db.go's NextEveryVersion) advances regardless of build status. The bug was found by accident, not by any check; this doc and the tests it describes exist so the next divergence is found on purpose.
Out of scope (decided, don't re-propose)
- No vendoring/importing Concourse's Go code.
github.com/concourse/concourseis Apache-2.0 and Go, so license isn't the blocker — its version-selection/passed:logic (atc/scheduler/algorithm) is hard-coupled to Postgres viaatc/db.VersionsDB(a concrete struct, hand-written SQL, not an interface). Confirmed impractical: even Concourse's own unit tests for this logic spin up a real Postgres instance (atc/scheduler/algorithm/suite_test.go). - No full Concourse (docker-compose: ATC+worker+Postgres) in CI. No lightweight/in-memory distribution exists upstream. Disproportionate to a project that deliberately implements a subset of Concourse's model, not the whole thing.
- Instead: hand-transcribed characterization tests, each citing exactly which Concourse doc page or source location (at a stated version/ref) it was transcribed from.
Claim inventory
Living checklist — update when a claim is added, resolved, or found to diverge.
| Claim | Location | Status |
|---|---|---|
get.version: latest/every/pinned; every fans out per version, continuing past a failed one | config.go's Step.Version, resource.go's VersionMode | Conformance test: TestConformanceGetVersionEveryContinuesPastFailure (internal/pipeline/conformance_test.go) |
| A triggered job builds what history holds, not a fresh check's output | the sqlite driver's resource_versions, internal/pipeline's loadResourceHistory, resource.go's WithResolvedVersions | Tests: e2e/trigger_e2e_test.go (nine scenarios, each driving a real steps web daemon with the pipeline set into it); TestRecordVersionsKeepsDiscoveryOrder, TestRecordVersionsPrunesOldestAndCascades, TestUsingAVersionIsNotTheSameAsCheckingForIt (internal/store/storetest, run by every driver through storetest.Run). This is now Concourse's own shape rather than a workaround for lacking it: a build resolves against stored versions and nothing is re-derived. A job used to be handed the versions its poll found, carried on its queue row, because a cursor-driven check answers differently the second time — which made a dropped enqueue data loss and needed a merge, a restart fold and a hand-queued clear rule to be safe. All of that is gone. One distinction has no Concourse counterpart, because Concourse has no equivalent of a build that resolves its own versions: resource_versions.from_check separates what a CHECK reported (history a job may read instead of checking) from a row that exists only because something referenced it (a steps run recording what it took). Conflating them lets one remembered version hide every version a check would have found. |
every takes each version once, not everything check currently returns | internal/pipeline/cursor.go, the sqlite driver's job_version_cursor (a high-water mark over check_order), resource.go's WithConsumed | Conformance test: TestConformanceGetVersionEveryTakesEachVersionOnce (internal/pipeline/conformance_test.go). Concourse source: atc/db/versions_db.go's NextEveryVersion @ v8.2.4 (read from source, not a spec page). Matches NextEveryVersion's rule that a version is taken when a build is CREATED with it, with no filter on build status anywhere — so a failed version is not retried (TestConformanceGetVersionEveryContinuesPastFailure pins that too, and previously asserted the opposite, which was this repo's own interpretation rather than Concourse's). The cursor suppresses; resource_versions is what makes a version available to suppress in the first place (row above), so the two are now separate questions rather than one doing both jobs badly. It is a high-water mark, as Concourse's is — NextEveryVersion takes the next version above the highest check_order a job has built, and this now records exactly that. It was a SET of consumed versions while versions had no order to appeal to; check_order supplied one. The change is not cosmetic: a set must be capped or it grows forever, and a capped set forgets its oldest members while the versions they name are still offered, so they read as unbuilt and run again. That was live for any pipeline setting version_history: above the consumed cap. A mark has no members to forget. |
Re-running one build against the versions it was CREATED with (--resume) | internal/pipeline/cursor.go's reopen, resume.go's resumedRunInputs, the sqlite driver's run_inputs | Tests: TestResumeReachesAVersionTheCursorAlreadyTook, TestResumeDoesNotReopenAnotherRunsVersions (./e2e). Concourse source: build_resource_config_version_inputs @ v8.2.4, the table fly rerun-build re-runs a build against. This row is the other half of the row above, and it was missing. The cursor takes a version when a build is CREATED with it and never looks at status again — correct, and Concourse's rule — which leaves a build that failed halfway with its version spent and no ordinary way back to it. Concourse's answer is an explicit re-run of that ONE build against the inputs it began with; internal/pipeline/get.go had said so in a comment ("which here is --force or --resume") since the cursor landed, and only --force was true. --resume selected nothing, ran no steps, and reported SUCCESS — a failed run turned green by the act of trying to recover it. The versions a run was created with are now recorded per run, and a resume re-opens those and no others. Not --force's semantics, deliberately: forcing re-opens every version any run ever took, so resuming one failed build would rebuild the green history behind it — invisible while a plan is cacheable, and a re-pushed branch and re-opened pull request the moment it holds a put: or an agent:, which is what the second test pins. |
check returns versions oldest→newest, latest last; steps doesn't re-sort | resource.go's CheckVersions | Conformance-annotated: TestSelectVersion/"latest when unpinned" (internal/resource/resource_test.go) |
check/in/out JSON stdin/stdout contract | resource.go | Conformance-annotated (CheckVersions/RunOut doc comments); gap-filled: TestConformanceRunOutUnparsableStdoutIsNilNotError |
| A resource has a CURRENT version even when its check reports nothing | internal/trigger's checkResource, the sqlite driver's resource_checks | Conformance test: TestPassedSurvivesAQuietCursorCheck (internal/trigger). Concourse keeps this for free — its version DB always knows a resource's current version, whatever the latest check returned. steps derives it from the last recorded version instead. This was a real defect, not a theoretical one: a passed: constraint is evaluated against the current version on every poll, so treating "the check reported nothing" as "the resource has no version" dropped it out of the poll's observations and held every downstream job back forever, silently. Before the cursor a check re-reported its whole window each time and this could not arise; a cursor-driven check answers with nothing almost every poll, which made it the steady state. |
check is given the current version, not just the source | resource.go's CheckVersions, the sqlite driver's resource_checks, internal/trigger's checkResource | Conformance tests: TestConformanceCheckReceivesCurrentVersion (internal/resource), TestPollOnceSecondPollPassesLastVersionToCheck and TestConformanceRunDoesNotDependOnPollerCursor (internal/trigger). Concourse doc: concourse-ci.org/docs/resource-types/implementing/ ("check ... is given the configured source and current version on stdin"). This row previously read as covered by the row above; it wasn't — steps passed the source alone, so a type talking to a web API had to guess a window instead of asking for what it hadn't seen. Three divergences to know. (1) The cursor is delivered as a template key ({{ .version }}), not on stdin, since steps renders commands rather than piping JSON — the shape source: already takes. (2) It is stored per RESOURCE (resource_checks, which watch already kept as its dirty baseline), where Concourse scopes a check to a resource config. (3) Only steps web is given it, and a job does not re-derive its versions at all — it reads resource_versions (row 18), so the cursor never reaches a job's own check because that check never runs. This arrangement is forced: the cursor advances when a poll ENQUEUES work, so a job re-deriving against an already-advanced cursor gets nothing and goes green having processed the very versions it was triggered for (TestConformanceRunDoesNotDependOnPollerCursor), while re-deriving WITHOUT the cursor hands it the whole window. Concourse has neither problem because it resolves an input set per build against its version history — which is now steps' model too. The optional-field spelling applies here as in row 31: the first-ever check gets an empty map and missingkey=error, so a cursor field is read as {{ index .version "ts" | default "0" }}. |
get: resource: aliasing shares version history by the real resource name | config.go's Step.Resource | Conformance-annotated: TestResourcesAndAffectedJobsResolveGetAlias (internal/trigger/trigger_test.go) |
Task input_mapping/output_mapping rebind only the external plan-artifact name | config.go's InputMapping/OutputMapping | Conformance-annotated: TestRunJobIsolatedGetAliasMappingAndPutAll (workspace_integration_test.go) |
Five hook modifiers (on_success/on_failure/on_error/on_abort/ensure) with Concourse's exact firing conditions | hooks.go | Conformance-annotated: TestRunHooksRouting, TestRunHooksAbortGracePeriod (internal/pipeline/hooks_test.go); on_failure-on-timeout pinned by the doc-tested deadline fixture in attempts-timeout.md, on_error by TestEndToEndAgentInfraErrorFiresOnError (root, unreachable provider), on_abort by TestConformanceAbortFiresOnAbortHook |
privileged: and container_limits: map to docker's --privileged/--cpu-shares/--memory | internal/config/limits.go, internal/shell's dockerStartArgs | Test: TestDockerStartArgsPrivilegedAndLimits (internal/shell), which also pins that an unset limit is OMITTED rather than passed as 0. Concourse doc: concourse-ci.org/docs/steps/task/. Both require image:, like network:. |
interruptible: decides whether a shutdown waits for a running build | config.Job.Interruptible, internal/web's LocalRunner.runContext | Conformance test: TestConformanceNonInterruptibleBuildSurvivesShutdown (internal/web); TestDrainCancelsWhatDoesNotWait beside it pins what does not wait, and TestWebShutdownWaitsForANonInterruptibleBuild (./e2e) sends a real steps web SIGINT mid-build. Concourse doc: concourse-ci.org/docs/jobs/ — same field, same default (false = wait). Scoped divergence: it applies to steps web only; steps run always interrupts immediately, since that is a person at a terminal. Only a SHUTDOWN waits: destroying or renaming the pipeline cancels its build at once. The daemon ignored the field until 2026-09-10 — only internal/trigger's drainer honoured it, and that drainer's last caller, steps web --once, went with #104 — so every shutdown cancelled every build. |
serial_groups: keeps two jobs sharing a tag from running at once | config.go's SerialGroupsByJob, the sqlite driver's ClaimNextJob | Conformance test: TestConformanceSerialGroupsBlockAcrossJobs (internal/store/storetest). Concourse doc: concourse-ci.org/docs/jobs/. |
serial: and job-level max_in_flight: cap concurrent builds of one job | config.Job.EffectiveMaxInFlight, the sqlite driver's ClaimNextJob | Conformance tests: TestConformanceMaxInFlightAdmitsUpToTheLimit, TestConformanceSerialForcesOneInFlight (internal/store/storetest), TestEffectiveMaxInFlight (internal/config). Concourse doc: concourse-ci.org/docs/jobs/ — unset is unlimited, serial:/serial_groups: force 1. This closes a divergence previously recorded here: every job used to be serial unconditionally, which made serial: true a no-op. One deliberate narrowing: max_in_flight: beside serial:/serial_groups: is a load error rather than being silently forced to 1, since the number would do nothing. |
passed: requires versions to have been green in the SAME upstream build | internal/pipeline's VersionSetPassedUpstream, Versions.HasPassedVersionSet | Conformance tests: TestConformancePassedRequiresVersionsToPassTogether, TestConformancePassedRowsWithoutABuildCannotCorrelate (internal/trigger). See Known gaps for what this found and the two limits that remain. |
| A job that never got a workspace fires no hooks | pipeline.go (RunJob) | Not a Concourse claim — Concourse has no literal job-level hook construct to compare against; comment corrected to state this as steps's own design choice, not parity |
docs/infra.md's passed: note, docs/workspace.md's detect-default note | docs | Already-honest, self-documented divergences — precedent for how to handle a real one, not something to test |
get.params reach in:, as put.params reach out: | config.go's Step.Params, resource.go's RunIn | Conformance test: TestConformanceGetParamsReachIn (internal/pipeline/conformance_test.go). Concourse doc: concourse-ci.org/docs/steps/get/. Cache half pinned separately by TestGetParamsAffectHash/TestResourceCacheKeyParams (internal/merkle) — params change what lands in the artifact, so they must key both the node and the cross-build resource cache. Divergence to know: templates render with missingkey=error, so an OPTIONAL get param is spelled `{{ index .params "x" |
do: runs several steps as one, so a hook covers the group | config.go's Step.Do, internal/pipeline/do.go | Runnable fixture: the doc-tested do: example in control-flow.md, plus internal/pipeline/do_test.go. Concourse doc: concourse-ci.org/docs/steps/do/. Divergences, deliberate: a get: inside is a load error (a get fans the rest of the plan out per version, which has nowhere to go inside a block), and to:/max_visits: on a CHILD are load errors rather than silent no-ops — a child has no plan position, and accepting them is the defect already paid for once with to: on the step a try: wraps. |
try: masks failures, errors and timeouts; only an abort propagates | config.go's Step.Try, internal/pipeline's toleratedByTry | Conforms — divergence closed 2026-08-18. Concourse: "Performs the given step, ignoring any failure and masking it with success" (concourse-ci.org/docs/steps/try/); its TryStep swallows failures, errors AND timeouts, but PROPAGATES a context cancellation (read from source @ v8.2.4). steps used to tolerate only outcome.Failed — an infrastructure error or expired timeout: stopped the run — and now draws Concourse's exact line: everything but outcome.Aborted is tolerated. Pinned by TestTolerateTryFailureClassifies (internal/pipeline), TestEndToEndTryToleratesInfraError (root), and the doc-tested timeout fixture in control-flow.md. |
put: runs an implicit get afterward, with get_params:/no_get: | Removed, deliberately — a put runs out: and nothing else, and produces no artifact | Documented divergence. Concourse fetches the produced version automatically (concourse-ci.org/docs/steps/put/); steps did too until the DSL audit removed it: an artifact appearing in the build that no step declared is ambient data flow, and every flow here is opt-in. The spelling is an explicit get: after the put — but note what it fetches: a run resolves every get's version at PLAN time, before any step executes, so a get after a put fetches the version pinned when the plan was built, not the one the put just published (verified empirically; docs/resources.md states it). A put's version reaches gets only in later runs or downstream triggered jobs — see Known gaps. get_params:/no_get: went with it. A separate divergence stays: Concourse expects out: to print a version; steps allows printing none (RunOut returns nil rather than erroring, relied on by resource types that publish without versioning). |
Task step file: — loading a task config from a file | config.go's Task.RunFile/Task.File, internal/config/files.go's Files/Bundle, docs/agents.md's "External files" section | Documented divergence, and the reason for it has changed. Concourse's file: is artifact-relative and document-scoped ({platform, image_resource, inputs, outputs, run}) because a Concourse pipeline is uploaded by fly set-pipeline and has no sibling filesystem at all. steps now uploads too — steps pipeline set sends {source, includes} — but its run_file:/file: stay pipeline-relative and field-scoped: the includes are read on the SENDER's disk, where the paths mean something, and travel with the configuration as a bundle the daemon resolves against and nothing else (see config.Bundle, which is closed precisely so a crafted run_file: cannot read the daemon's own files). "The build script lives in the repo" is still spelled run: sh repo/ci/build.sh with inputs: [repo]. |
| A pipeline is UPLOADED to the server under a name, not read from a path | internal/cli/pipeline.go, internal/web/api.go, internal/cli/daemon.go; e2e: TestPipelineSetServesAndPollsWhatWasSet, TestPipelineSetAppliesAnEdit, TestPipelineSetCarriesIncludes (./e2e) | Conforms — this is fly set-pipeline's shape. Concourse's server holds pipelines and fly set-pipeline -p <name> -c <file> -v k=v uploads one; steps pipeline set is the same verb against steps web, including the diff-and-confirm and the -n a script uses. Two deliberate divergences. A new pipeline starts UNPAUSED, where fly set-pipeline creates paused: steps' cold-start rule already limits a fresh pipeline to one build of the newest version (row below), so the thing pausing protects against cannot happen here — and Concourse's own set_pipeline: step creates unpaused. Vars are substituted client-side, before the upload, so the daemon never holds them and one file under two var sets is two configurations rather than one pipeline with instance vars; -y/--yaml-var is therefore unnecessary, since a textual substitution already makes -v flag=true parse as a boolean. |
| Setting a pipeline is compare-and-set, and a rename keeps history | internal/cli/daemon.go's check, the sqlite driver's Rename/Delete; conformance: TestRenameKeepsTheHistoryUnderTheNewName, TestDeleteForgetsThePipelineAndItsHistory (internal/store/storetest); e2e: TestPipelineSetIsCompareAndSet, TestPipelineRenameKeepsHistory | Divergence, deliberate, in both directions. Concourse has no compare-and-set: two fly set-pipelines race and the last one wins silently. steps sends the sha it diffed against and refuses a set whose configuration moved in between (409), because the diff is the whole point of showing one. Concourse's old_name: renames a pipeline as part of a set; here it is its own verb (steps pipeline rename), and it keeps the history because every scoped row reaches the pipeline by ROW ID rather than by name — which is also why a rename is not the identity change it was when the name came from a filename. |
| A first-ever check builds the newest version and seeds the rest | internal/trigger's seedColdStart/coldStartMark/recordHistory; e2e: TestWatchColdStartAnswersTheNewestNotTheBacklog, TestWatchColdStartWithVersionEveryBuildsOnce, TestWatchLockstepColdStartBuildsOneSet (./e2e); docs/infra.md, docs/resources.md | Matches Concourse on the trigger; one divergence left, and it is the cursor contract, not the scheduler. Concourse fires a trigger: true job once with the version its first check reports, and its check contract makes that exactly ONE version: concourse/git-resource's README, "If no version is given, the ref for HEAD is returned" — so a Concourse first check has no backlog to consider. steps hands a check an empty cursor instead (row above), and this repo's convention is for a check to report everything it can see, so a first check commonly reports N. It therefore builds the newest and records the N-1 below it as already taken for every job that reads the resource: one build, never N, never zero. Zero was the rule until 2026-08-20, on the reasoning that a fresh (or freshly lost) state database must not mass-re-run every job — true, but it also meant a slowly-changing resource (an open PR, a release) was never built at all until something new arrived, and deleting .steps/ to "start fresh" re-armed the same silence. The backlog below the newest is still reachable only by --pin, which is an instruction rather than a discovery (row on pins). |
An attempt that expires its timeout: ends the step; remaining attempts: are skipped | internal/pipeline/attempts.go (retry.StopOnDeadline), docs/attempts-timeout.md ("An expired timeout is not retried") | Documented divergence, deliberate. Concourse treats a timeout as a failed attempt and runs the remaining attempts, each with a fresh timeout; steps stops — the same work against the same budget expires again, so a retry only doubles the wall clock and, on an agent step, the bill. A deadline from inside an attempt (an MCP or HTTP client's own) stays retryable; only the step's own timeout: ends the loop. |
Timeout expiry classifies as failed — on_failure fires, as in Concourse | internal/outcome's FailOnDeadline, applied at every seam owning a step's own deadline (retryWithTimeout, the agent cascade/CLI/fix loops); pinned by the doc-tested deadline fixture in attempts-timeout.md and TestEndToEndTimeoutFiresOnFailure (root) | Conforms — divergence closed 2026-08-18. Concourse marks a timed-out step FAILED. steps used to classify it errored ("a deadline is infrastructure"); the parity reading won: the step was given a budget and did not finish inside it, which is the step saying no. Errored stays reserved for the machinery breaking (docker, transport, workspace — pinned by TestEndToEndAgentInfraErrorFiresOnError), and only the step's OWN timeout: is reclassified — a deadline from inside an attempt (an MCP/HTTP client's own) still classifies by its nature. |
| A webhook delivery treats the version as changed even when it matches what was recorded | internal/trigger/webhook.go's checkNow (obs.dirty = true), docs/infra.md's webhook section | Documented divergence, deliberate. Concourse's webhook endpoint only triggers an out-of-band check — an unchanged version queues no builds. Here the delivery itself is evidence: the sender knows something the check output may not show yet, so downstream jobs are enqueued regardless. |
((var)) substitution is textual and happens before the parse | internal/config/vars.go's InterpolateVars, docs/templating.md | Documented divergence, deliberate. Concourse interpolates structurally, at the YAML node level: a multi-line or special-character value stays one scalar however it is spelled, and ((foo.bar)) reaches into a var's subfields. steps substitutes into the source text before parsing — a value containing YAML syntax changes the parse, and there is no subfield access. The trade is stated in InterpolateVars's own comment: a structural pass must enumerate every field a var may appear in, and that list goes stale every time a field is added. |
load_var: produces a pipeline-namespace ((name)) | config.go's Step.LoadVar/Step.VarFile, docs/templating.md | Documented divergence, deliberate. Concourse's load_var: produces a build-local var spelled ((.:name)) and offers format: (json/yaml/trim/raw/…) and reveal:; steps's lands in the same namespace --var fills (((name))), always trims, and has neither field. It also requires the file to sit inside a declared inputs: — a step's directory holds only the artifacts it declares. |
passed: naming a job that never gets the resource is a load error; entries are literal job names | internal/config/passed.go's validatePassedStep | Deliberate narrowing, two halves. Concourse validates only that the named jobs exist, and its passed: entries are glob patterns (path.Match @ v8.2.4) — a constraint on a job that never uses the resource passes validation and silently waits forever. Here that is a deadlock spelled as a typo, refused at load, and the entries are literal names, not globs. |
A branch consuming a sibling branch's output inside in_parallel: is a plan-time error | internal/workspace's validateParallelArtifactFlow | Deliberate narrowing. Concourse allows it — a branch step sees whatever artifacts exist when it happens to run, so reading a sibling's output is a race that sometimes works. Here every branch is checked against the artifacts available when the BLOCK started; what the branches produce joins the view only after the block, where a later step may legitimately consume it. |
tags: places a step on ONE worker named by the invocation, not by intersection against a pool | internal/config's validateTagRules, internal/pipeline's ValidateWorkerPlacement, docs/infra.md ("Remote workers") | Deliberate narrowing, three parts. Concourse schedules a step onto any worker advertising ALL of its tags, from a pool workers register themselves into; there is no pool here, so a tag is a name one --worker mapping resolves to one machine, and a second tag would name a second machine with no rule for choosing. An unmapped tag is a run-start error rather than a build that waits for a worker that will never register — the same reasoning as passed: naming a job that never gets the resource. And placement does not enter the cache key, because a tree that crossed the wire digests identically to one that never left; Concourse has no equivalent question, having no cross-worker cache to keep honest. |
A resource's tags: places its get and put steps too, not only its check | internal/config's inheritResourceTags, docs/infra.md ("Resources on workers") | Documented divergence, deliberate. Concourse's resource-level tags: places only the check, and its docs warn it does not apply to get/put steps — each must repeat the tag. Here a get or put with no tags: of its own inherits the resource's, and a step's own still wins: a get's version check and its in have to land on the same machine, and the per-step repetition is the papercut Concourse documents. The fetched tree is brought back to the orchestrator rather than streamed worker-to-worker, since there are no per-worker volumes here. |
attempts: on an agent step retries the failing provider REQUEST within one conversation, not the whole step | docs/attempts-timeout.md ("What attempts: costs on an agent"), internal/agent's request retry | Documented divergence, deliberate — the redefinition attempts-timeout.md records as a breaking change. On task/get/put steps attempts: re-runs the whole operation, as Concourse's does; an agent step's failing operation is one HTTP request, so a transient 500 costs one extra request rather than a re-billed conversation rebuilt from nothing. Concourse has no agent step to compare against, but the field means less than a Concourse reader expects on this one kind, so it is recorded. |
Not implemented, on purpose
Recorded rather than left silent: an absent primitive reads as an oversight until something says otherwise. This is the "we looked at it and decided no" list, distinct from Known gaps below, which is "we want it and have not done it".
| Concourse feature | why not |
|---|---|
set_pipeline: | A step that reconfigures the server from inside a build. There IS a server now — steps web holds pipelines and steps pipeline set uploads them (#104) — so this stops being impossible and becomes unbuilt: it needs a step that can reach the daemon's API, and the credential story that goes with letting a pipeline reconfigure the box it runs on. Deliberately out of #104's scope; see Known gaps. |
check_every: | Check cadence is one global --interval on steps web. Per-resource cadence is a real Concourse feature and a deliberate omission here, not an oversight: a pipeline that needs a slow poll for one expensive resource can give it a webhook instead (see docs/infra.md). Revisit if someone actually hits it. |
image_resource: | steps names an image with a plain image: string rather than through a resource. A real divergence — Concourse's form is versionable and digest-pinnable — but a deliberate simplification, not an oversight. |
public:, icon:, groups:, old_name:, build_log_retention: | Presentation and retention concerns rather than execution primitives. steps web may want groups:/icon: and a retention policy later; tracked in #68, not here. |
| File-level portability | The goal is that the MODEL conforms, not that a Concourse pipeline file runs unmodified. It does not and is not intended to: version: sits on the get rather than the resource, webhook_token_env: names a variable rather than holding a token, and the fields above do not exist. Readers assume the stronger promise unless told, so this row says it. |
How to add a conformance test
- Find the specific claim: grep this repo for
mirrors Concourse/per Concourse/Concourse's— a conformance test traces to one of those, not to a general belief about how Concourse works. - Pin a Concourse reference: a doc page (concourse-ci.org/docs/...) or, if the behavior isn't documented, a source location at a release tag (e.g.
@ v8.2.4— check the latest release first). Prefer Concourse's own docs or test suite (atc/scheduler/algorithm/algorithm_test.go's GinkgoEntry(...)scenarios are precise, dozens of them) over reading implementation source directly; when you do have to read source, say so in the citation and treat it as a finding, not an official guarantee. - Transcribe the scenario, not the implementation — a
stepsfixture (Go-constructedconfig.Config, or a YAML pipeline written to a temp file and run viarun()/mustRun(), matching whichever pattern the surrounding test file already uses) reproducing the same input/output shape the Concourse doc/test describes, scaled to whatstepsactually models. - Name it
TestConformance...(or, if annotating a pre-existing test that already covers the behavior, add a// TestConformance note:comment to it instead of duplicating) sogo test -run TestConformance ./...finds the whole set. - Cite: the Concourse doc URL/section, or source file + symbol + pinned ref; and which of steps's own "mirrors Concourse" comments it verifies.
- If it fails against current
stepsbehavior, that's a real divergence — fix the divergence, not the test, unless it's a deliberate, already-documented one (see thedocs/infra.md/docs/workspace.mdprecedent above), in which case document the divergence next to the claim it partially contradicts instead of leaving the claim overbroad.
Known gaps
passed:has no conformance testjob_versionsnow carries abuild_idand the lookup isHasPassedVersionSet— "is there one build of that job where all of these were green at once", which is the question Concourse's scheduler answers. Pinned byTestConformancePassedRequiresVersionsToPassTogether(internal/trigger). Two limits worth knowing: rows predating the column cannot correlate, so a multi-resource fan-in waits one more upstream run after upgrading (TestConformancePassedRowsWithoutABuildCannotCorrelate); anda job using— closed by input sets: a fan-out is one build per set, and each records the versions IT fetched under its own id, so a fan-out correlates exactly as tightly as any other build. Fixing it also fixed a straightforward data loss the one-id shape was hiding — the record was keyed per resource, so of a multi-set run only the last set's versions were ever written, and the versions of a run that failed at any set were lost entirely (get: version: everyrecords its whole fan-out under one build id, which is exactly the old uncorrelated behaviour for that one shapeTestRunPassedSeesEveryGreenSet,TestRunPassedKeepsGreenSetsFromAFailedRun).serial:/serial_groups:have no conformance testTestConformanceSerialForcesOneInFlight(serial forces one build at a time, taking precedence overmax_in_flight),TestConformanceSerialGroupsBlockAcrossJobs(a shared group blocks across jobs), andTestConformanceMaxInFlightAdmitsUpToTheLimit(allinternal/store/storetest). Rows 29–30 above carry the citations. This entry previously said they were unimplemented, then implemented-but-untested — each time the stale line was how the "add its test as part of implementing it" instruction quietly went unfollowed.Job-level— closed. It is a job field now, spelledmax_in_flightis not implementedmax_in_flight:exactly as Concourse spells it. The feared name collision withacross:'smax_in_flight:turned out to be Concourse's own overload rather than one invented here, and the two live on different things: a job field and a step field. Conforming meant keeping the word in both places.No fixture for— closed.on_error/on_aborton_erroris pinned byTestEndToEndAgentInfraErrorFiresOnError(an unreachable provider endpoint — hermetic, no docker; it was originally pinned by thedeadlinedoc fixture, until the timeout-classifies-errored divergence that fixture leaned on was itself closed for Concourse parity);on_abortbyTestConformanceAbortFiresOnAbortHook, which cancels the job context mid-step because no pipeline file can ask for one. All five hook modifiers now have a deterministic trigger.version: everyfans out over everythingcheckjust returned, not over the versions this job has yet to buildput:or anagent:, whichroute.go'sunskippableReasonnever skips: the cache avoids recomputing a value, and was never a mechanism for not repeating an effect. There is now a per-(job, resource) cursor (job_version_cursor,internal/pipeline/cursor.go), applied at the one seam both the planner and the executor read through (resource.WithConsumed), so plan and run cannot disagree. Pinned byTestConformanceGetVersionEveryTakesEachVersionOnce.One divergence remains, and it is structural rather than chosen:— closed.stepsstores no version historyresource_versionsrecords every version a check reports, in discovery order (check_order, the column Concourse's ownNextEveryVersionwalks), so a version that scrolled out of a check's window is still there and a job that was down can still build what it missed. Two things bound it, and both are stated rather than implied: history is capped per resource (defaults.version_history:,--version-history, default 1000) and a pruned version takes its green record with it by foreign key, so a cap set below what a slow downstream job needs will hold that job back; and nothing remembers a version steps never asked about, so the very first check of a resource still starts from whatever it reports. A second, briefly-shipped divergence was reverted: taking a version only on SUCCESS, so failures retried. Concourse takes it when the build is created and never looks at status again, which is what this now does — the interpretation was wrong, and the source (build_resource_config_version_inputs, no status filter) settled it.A structural divergence, now stated rather than silent: Concourse resolves an input SET per build and gives EACH input its own cursor, so several gets in one job may each be— closed.every;stepsfans out at a single point — the first get in a planstepsnow resolves input sets the wayatc/scheduler/algorithm'sindividualResolverdoes — oneNextEveryVersion-style cursor per input, each advancing one step per set, an exhausted input holding at the newest version at-or-below its mark (Concourse'scheck_order <=fallback) — and runs one build per set, so any top-level get may beevery(resolveInputSets,internal/pipeline/inputsets.go). The planner and the executor consume the SAME resolved sets, which is what makes the relaxation safe where the old load error was hiding a planner/executor disagreement (the planner would have cross-producted what the executor pinned stale). Pinned by theTestWatchLockstep*scenarios (streaming interleave, the diagonal, hold-at-mark) andTestPlanChainsSetsMatchRecursion, the golden test proving per-set planning is byte-identical to the old recursion for zero- and single-every plans — no hashes changed, nothing re-ran on upgrade. What still stays a load error, deliberately:everyon a non-top-level get (its build's set is already bound, so it would fetch one version forever) and twoeverygets aliasing one resource (they would share one cursor).- A plan cannot consume its own put's version in the same run. Versions are resolved once at plan time, so the explicit get-after-put spelling (row 37) fetches the pre-put version — a divergence beyond the implicit-get removal itself, since Concourse's implicit get fetches exactly the version the put created. Found empirically 2026-08-17 while auditing docs/resources.md's put example (whose prose used to claim the opposite). Undecided whether a same-run get should re-resolve after a put or whether "publish here, consume next run" is the intended model; until decided, the docs state what the code does.
input_mapping/output_mapping's two undocumented sub-behaviors (an omitted key defaults to binding by its own declared name; a mapping naming a nonexistent plan artifact is a load-time error) aren't in Concourse's docs — they'resteps's own reasonable design choices, not Concourse claims, and don't need a conformance test (there's nothing upstream to conform to).