Exit 0 with every vendor unreachable

What the audit found

LaunchOps is the part of our platform that reads launch-day configuration from vendors such as Vercel and Stripe, diffs it against a declared manifest, and gates the changes. During an end-to-end verification this August we ran the whole surface from a clean install, before any vendor credentials had been configured. Every vendor was unreachable, and the commands still exited 0.

Each unreachable vendor printed an [err] line, so a human watching the terminal saw the problem immediately. The process exit code said otherwise. CI does not watch terminals; it reads codes. Any pipeline built as 'scan, then proceed if it passed' would have proceeded.

# before the fix                    # since 4.16.0
$ es launch scan                    $ es launch scan
  [err] vercel Missing credentials    [err] vercel Missing credentials
  [err] github Missing credentials    [err] github Missing credentials
$ echo $?                           $ echo $?
0                                   1

The plan command was worse. Building a plan from a scan that had read nothing produced a syntactically perfect plan file with an empty action list, persisted to disk, declaring that no gates were required. Downstream of that file, 'could not read' and 'read, and found nothing' are the same artifact. That conversion is the dangerous part, more than any single wrong exit code.

The exit-code contract

Since 4.16.0 the exit codes are the interface: scan and diff exit 1 when any vendor was unreadable, and plan refuses outright to build from a partial read. The refusal names the vendors it could not reach and the env vars each one was missing, so the operator can fix credentials and rerun.

One case needed care. An operator who excludes a vendor with --skip-vendors did it on purpose, so deliberate skips render as [skip] and stay exit-neutral. If skips counted as failures the gate would be red on most runs, and a check that is red on most runs eventually gets disabled.

Where fail-open defaults come from

Nobody writes 'return success on failure' on purpose. Ours accumulated from four ordinary steps: the scan loop catches per-vendor errors so one bad vendor cannot abort the rest; the caught errors become outcome objects; the outcomes render as [err] lines; then execution reaches the end of main(), where the default exit status is 0. Nothing on that path converts a vendor error into a nonzero exit.

We had even been living with the consequence in our own automation. The weekly vendor smoke could not trust the exit code, so its workflow grepped the captured output for failure markers instead, and the grep carried a comment explaining why. In hindsight, that comment was a bug report against our own exit codes; nobody had filed it as one.

What we took away

• CI reads exit codes, not terminal output. If a tool prints [err] and still exits 0, the error is invisible to every pipeline built on top of it.
• 'Could not read' must stay distinguishable from 'read, and found nothing'. Any artifact that erases the distinction, like an empty plan file, will eventually be trusted by something.
• Keep deliberate skips exit-neutral. If routine operator choices turn a gate red, people stop treating red as a signal.

Sources: the fix shipped 2026-08-05 as @enterprise-skills/launchops-core · how our release gates work · The production-ready guide