Artifact provenance, not artifact integrity

Shipped ≠ sourced.

Your lockfile proves the tarball is the tarball. It says nothing about whether that tarball is the code at the git tag it claims to come from. Sourcewitness fetches both, normalizes the file sets, and returns one of three answers: IN_SYNC, DIVERGED, or an honest UNVERIFIABLE.

How the diff is computed
01 — Live check

Ask the registry what it actually shipped.

This runs on the server, right now: fetch the npm metadata, download the tarball, list it with tar, download the GitHub tag archive, extract it, and compare the two file sets. No stored snapshots, no third-party scanner, no trust in a badge.

Try left-pad, debug, chalk, or a scoped package like @babel/core. The 20s budget covers registry + GitHub.

IDLE —

Nothing checked yet.

Enter a package above. The verdict is computed from the live registry and the live git tag, not from a curated list.

02 — Curated evidence

Two cases where "same version" was not the same code.

These are documented cases, not live checks. They are the reason the live check compares file sets instead of trusting a version string.

documented case, not a live check

xz / liblzma

5.6.0 · CVE-2024-3094 · disclosed March 2024

The release tarball was generated from the attacker's branch, not from the public tag. Distros built the tarball. The git tag did not contain what the tarball contained.

shipped only  m4/build-to-host.m4   (modified)
shipped only  tests/files/bad-3-corrupt_lzma2.xz
shipped only  tests/files/good-large_compressed.lzma
at tag        same paths, different content / absent

During ./configure, the m4 macro decoded a payload from the test fixtures and patched liblzma so sshd would accept a hidden key. The repository's own history never showed the payload.

Why the lockfile can't see it: a lockfile records the artifact's version and hash. The hash of the malicious tarball is correct — it is the hash of the thing that was served. Integrity is not provenance.
documented case, not a live check

tj-actions/changed-files

tags v35–v45 · CVE-2025-30066 · disclosed March 2025

A stolen token let an attacker push a malicious commit and then re-point existing tags at it. Consumers kept asking for v45; they got different code. The version string never changed.

tag v45 → 0e58ed8  benign commit   (original)
tag v45 → 0e58ed8  malicious commit (re-pointed)
effect   runner memory dumped to workflow logs,
         base64-encoded, on every job using it

The compromise was detected by comparing the commit a tag resolved to against the commit it used to resolve to. Nothing in the workflow file changed.

Why the lockfile can't see it: Dependabot and Renovate pin owner/action@v45 — a mutable name. A lockfile that records a tag records a pointer, and pointers can be rewritten. Only a commit SHA is content-addressed.
03 — Ref ledger

A tag is a pointer. Pointers move.

Git itself has no memory of what a tag used to point at. If you never recorded v45 → commit A, then v45 → commit B looks exactly like a tag that was always B.

01

Record the mapping

On every publish, resolve the tag to a commit and append {package, version, tag, commit, first_seen} to a ledger you control.

02

Append, never rewrite

The ledger is append-only. A correction is a new row that supersedes an old one, with both visible. History you can edit is not evidence.

03

Alert on a re-point

Same tag, different commit, no new version: that is the tj-actions signal. A new version with a new commit is normal. A moved tag is not.

seentagcommit / note
2025-01-08v45a1b2c3d first observed
2025-02-02v45a1b2c3d unchanged ✓
2025-03-14v450e58ed8 RE-POINTED, same version
2025-03-15v45a1b2c3d restored upstream

The ledger is not a scanner and not a signature. It is a memory. Most supply-chain attacks are only visible against a memory.

04 — Why this is not an SCA tool

Dependency scanners ask "is this version vulnerable?" This asks "is this the code?"

Those are different questions with different failure modes. A vulnerability database maps versions to advisories; it assumes the version identifies the code. The xz and tj-actions incidents broke that assumption.

/ 01

File sets, not signatures

The check compares paths that exist in the tarball with paths that exist at the tag. No malware signature, no allowlist, no model. A file that ships but is not in source is a fact.

/ 02

Deterministic and replayable

The same package and tag produce the same verdict. There is no score to tune and no confidence interval to argue about. The refs compared are printed with the answer.

/ 03

Honest about coverage

No repository link, no tag, a rate limit, a 20s timeout: you get UNVERIFIABLE with the reason, never a green badge invented from missing data.

05 — Method

Five steps. All of them observable.

Every step is a network fetch or a local tar call. You can reproduce the result by hand with curl and tar.

01

Resolve the registry ref

Fetch registry.npmjs.org/<pkg>, take dist-tags.latest, its tarball URL, and its repository URL.

02

List the artifact

Download the tarball to a temp dir and run tar -tzf. Directories are dropped; the package/ prefix is stripped.

03

Extract the tag

For GitHub repos, fetch codeload/.../tar.gz/refs/tags/v<version>, falling back to <version>. Extract, walk the tree, strip the archive root.

04

Normalize, and count what you drop

Ignore manifests, lockfiles, README/LICENSE, and CI metadata. Count them. dist/ is not ignored — generated bundles are exactly where a split hides, so they are compared and flagged.

05

Diff and classify

Shipped-only paths are the serious signal. Source-only paths are usually tests or a files allowlist and are reported as a coverage gap, not an attack. Both lists are capped at 40 paths.

// the whole product, essentially registry = GET registry.npmjs.org/{pkg} artifact = tar -tzf {dist.tarball} source = tar -xzf codeload/{owner}/{repo} refs/tags/v{version} // set comparison after normalization S = shipped paths − ignored metadata G = source paths − ignored metadata IN_SYNC S = G DIVERGED S ≠ G shippedOnly = S \ G ← the xz signal sourceOnly = G \ S ← usually benign UNVERIFIABLE no repo, no tag, timeout, rate limit, unreadable archive // bounds max 40 paths per side · cache 10 min hard budget 20s per check · never throw

Extraction is confined to a fresh temp directory; archives with absolute paths or .. entries are rejected. Symlinks are skipped.

06 — API

One endpoint. One verdict.

No key, no account. The response always includes the refs compared and the elapsed time, so a verdict can be audited after the fact.

# ask the witness
curl -s 'https://sourcewitness.toyou.gr/api/check?pkg=left-pad'

# scoped names are fine too
curl -s '.../api/check?pkg=%40babel%2Fcore'
// what comes back
{
  "pkg": "left-pad",
  "version": "1.3.0",
  "verdict": "IN_SYNC",
  "counts": {
    "shipped": 9, "source": 9,
    "shippedOnly": 0, "sourceOnly": 0
  },
  "refs": {
    "npm": "left-pad@1.3.0",
    "tag": "v1.3.0",
    "github": "stevemao/left-pad"
  },
  "elapsedMs": 812
}
07 — Limits, stated up front

What this cannot do.

A provenance check that hides its blind spots is selling confidence. These are the ones that matter.

Generated files create false positives

Many packages build dist/ before publishing. Those files legitimately exist in the tarball and not in the tag. The diff flags them; whether that is acceptable is your call, and the paths are printed so you can judge.

No repo link means no comparison

If the published metadata has no GitHub repository field, there is no tag to compare against. You get UNVERIFIABLE — a coverage gap, not a clean bill of health.

GitHub rate limits and tag naming

Tag archives come from codeload. Heavy use gets 429s; only v<version> and <version> are tried. A package tagged some other way will read UNVERIFIABLE, with the tags attempted in the response.

File sets, not file contents

Two files with the same path can differ byte-for-byte. This check catches the xz shape — a file that should not exist — but not a same-path content swap. That is a deliberate scope boundary, not an oversight.

Monorepos and subdirectories

The repository URL's subdirectory is not resolved; the whole tag archive is compared. Expect source-only noise for packages published from a monorepo subfolder.

One version at a time

Only dist-tags.latest is checked. A malicious old version stays invisible unless you ask for a package that resolves to it.