Skip to content

ci: verify what we publish from the consumer side - #1

Open
MattJackson wants to merge 2 commits into
devfrom
ci/consumer-verification
Open

ci: verify what we publish from the consumer side#1
MattJackson wants to merge 2 commits into
devfrom
ci/consumer-verification

Conversation

@MattJackson

@MattJackson MattJackson commented Aug 8, 2026

Copy link
Copy Markdown
Contributor

What this is

Nothing in this repo has ever checked the artifact a user downloads. ci.yml proves the code builds and its tests pass. release.yml's verify-assets proves the upload step believed itself, asserted from inside the run that did the uploading. Neither is evidence about the published thing.

That gap shipped twice in this fleet:

  • webrequest-hook v1.0.4 published as a zero-asset phantom. Tag, Release object, nothing to download.
  • headroom-hook's published Docker bundle could not boot the gateway. Its shipped docker/bundle/config.yaml still uses config shapes busbar 1.5.3 retired (auth.admin_auth: with inline module entries), so the container exits 1 during config load and never binds a port. Verified against the current published image, getbusbar/busbar-headroom:2.0.4, while writing this.

Both were green everywhere. Nothing anywhere noticed.

What it adds

.github/workflows/consumer-verify.yml, which calls the fleet's shared reusable workflow GetBusbar/busbar/.github/workflows/plugin-consumer-verify.yml@dev - one file for all ten repos, the same way every repo already calls plugin-ci.yml, so a fix reaches the whole fleet instead of being copied ten times and drifting nine.

It checks out nothing, so a fix that is committed but never published still fails it. Per release it asserts:

  1. the Release is published, not a still-draft one that no user can see;
  2. /releases/latest resolves to it;
  3. every platform archive it owes is present, plausibly sized, and downloadable through /releases/latest/download/. Not a count - a count can never see a missing platform, only a name can;
  4. one archive really is a plugin busbar would accept: manifest name/alias/kind/version, the sha256 actually binding the cdylib beside it, the packed file really being a shared library, and a non-empty signature (every release workflow here silently falls back to --allow-unsigned when BUSBAR_SIGN_KEY is unset, and busbar then refuses the plugin on the user's machine);
  5. where a runnable bundle is published, the image is pulled fresh and the container must boot and serve.

The same job is appended to release.yml as its final step, so a broken publish turns the release red rather than leaving a green release beside a red run nobody correlates.

Consumer verification is post-publication by nature - you cannot download an asset that was never uploaded - so it cannot block the publish and does not pretend to. The value is the verdict being impossible to miss: red on the release, and an auto-filed issue. On failure the shared workflow opens or updates one labelled issue naming the failing check, its expected and observed values, and the run URL, and closes it again when a run passes.

Note on inputs

They are read off the published manifest, not guessed from the crate name. This fleet genuinely disagrees with itself about naming: the store repos drop the trailing -plugin that the auth repos keep, and store-valkey publishes as busbar-store-redis.

Merge order

Depends on GetBusbar/busbar#54, which adds the shared reusable workflow. Merge that to busbar's dev first; until then this workflow's uses: cannot resolve.

Matthew added 2 commits August 8, 2026 12:57
Nothing in this repo has ever checked the artifact a user downloads. ci.yml proves
the code builds and its tests pass; release.yml's verify-assets proves the upload
step believed itself, asserted from inside the run that did the uploading. Neither
is evidence about the published thing.

That gap shipped twice in this fleet. webrequest-hook v1.0.4 published as a
zero-asset phantom: tag, Release object, nothing to download. headroom-hook's
published Docker bundle could not boot the gateway, because its shipped
docker/bundle/config.yaml still used config shapes busbar 1.5.3 retired, so the
container exits 1 during config load and never binds a port. Both were green
everywhere.

Adds consumer-verify.yml, which calls the fleet's shared
GetBusbar/busbar/.github/workflows/plugin-consumer-verify.yml. It checks out
nothing, so a fix that is committed but never published still fails it. It asserts
the Release is published rather than a still-draft, that every platform archive it
owes is present and downloadable THROUGH /releases/latest/download/ (a count can
never see a missing platform, only a name can), that one archive really is a plugin
busbar would accept - manifest name/alias/kind/version, the sha256 binding the
cdylib beside it, a real shared library, a non-empty signature - and, where a
runnable bundle is published, that the image BOOTS AND SERVES.

The same job is appended to release.yml as its final step, so a broken publish
turns the RELEASE red rather than leaving a green release beside a red run nobody
correlates. On failure the shared workflow opens or updates ONE labelled issue
naming the failing check, its expected and observed values and the run URL, and
closes it again when a run passes.

The inputs are read off the PUBLISHED manifest rather than guessed from the crate
name: this fleet genuinely disagrees with itself about naming (the store repos drop
the trailing -plugin the auth repos keep, and store-valkey publishes as
busbar-store-redis).
On a pull_request event `github.ref_name` is '<number>/merge', not a branch name,
so plugin-ci asked GetBusbar/busbar for a branch called '5/merge' and the sibling
checkout died with 'The process /usr/bin/git failed with exit code 1', naming
neither the ref nor the reason. Every pull request to this repo has been red on
that before running a line of the change it was testing.

`github.base_ref` is the branch the PR targets and is empty on a push, so this
means dev-against-dev and main-against-main as intended on both events.
plugin-ci.yml also now falls back with a loud warning when a ref cannot be
resolved, so this is belt and braces - but a caller should say what it means
rather than lean on a fallback.
@MattJackson
MattJackson force-pushed the ci/consumer-verification branch from 5a1a593 to 0ab8165 Compare August 8, 2026 20:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant