Skip to content

The whole is not the sum of its files

Most static analysis is myopic by construction. A linter opens one file, walks its syntax tree, closes the file, and moves on; it can tell you that a function is too long, but not that the interface it just parsed is the third file in the project to redeclare the same three fields under a different name. That kind of smell only shows up once you can see the whole codebase at once — and until recently, cofferdam couldn't.

Six checks added in this release change that. Each one waits until every file has been parsed, then asks a question that only makes sense at the level of the whole project: is this file unusually central? Does this barrel re-export everything in sight? Does the project even agree on how it handles errors? The answers come from statistics computed over the codebase, not from a rule written against a single file.

Cheap thresholds are false thresholds

The obvious way to write "flag a file that imports too much" is to pick a number — more than 15 imports, say — and flag anything over it. This is also the least defensible number a check will ever produce. Fifteen is fine for a hundred-file project and absurd for a ten-thousand-file monorepo where every substantial module legitimately touches a dozen others. A hardcoded threshold either nags small projects into submission or lets a genuinely bloated one sail through untouched, depending on which side of an arbitrary line it happens to sit.

Design.ImportFanOutOutlier and Design.BarrelReexportBloat avoid the arbitrary number by computing their own. For fan-in and fan-out, that means the project's mean import count and its standard deviation, with a file flagged only once it clears the mean by three standard deviations — a bar that adapts to whatever "normal" looks like in that particular codebase. For barrels, it's the fraction of a directory's exports funneled through one index.ts, again judged against the spread of the same figure across every other barrel in the project. Nobody has to tune a number per project; the project supplies its own baseline.

Statistics of this kind come with a hazard that a fixed threshold doesn't: outliers that are supposed to be there can poison the sample they're being measured against. A barrel file is, by design, a file with an enormous fan-in — that's the whole point of a barrel. Left in the population, one legitimate index.ts inflates the mean enough to mask a real god-module hiding two files over, or shifts the standard deviation until nothing clears the bar at all. Both checks solve this the same way: recognized aggregator files (index.*, types.*) are stripped out before the mean and standard deviation are computed, not merely excused afterward from being flagged. The population has to be clean before the statistic means anything.

That principle was not academic. Running the fan-out check against a real Next.js codebase during validation surfaced a second, subtler version of the same problem: a bare import specifier that resolves into node_modules still carries a resolved path in cofferdam's import graph (the resolver traces it there deliberately, to distinguish a genuinely external package from one that merely looks like one). The check hadn't been told to ignore that case, so a single vendor bundle — lucide-react's dist file, imported everywhere a lucide icon was used — was quietly acting as an uncredentialed barrel, dragging the project's mean fan-in up for every file in it. The fix excludes node_modules-resolved edges from the count entirely; the project's real mean dropped from 2.9 to 2.1 once the vendor noise was gone, and the false positive disappeared. The lesson generalizes: any check that computes a population statistic over a codebase has to be as careful about what it excludes from the sample as about what it looks for in it.

Consistency has a majority, not a rule

Consistency.ErrorHandlingIdiom asks a different kind of question: not "is this file unusual" but "does this file match everyone else." A codebase that throws for errors in ninety percent of its functions and returns { error } objects in the other ten doesn't have two valid idioms — it has one convention and one set of outliers, and the check's job is to name the minority. It does this by tallying throw statements against error-shaped returns across the whole project, in a two-pass design: the first pass over every file just counts, the second pass reads the completed tally and flags whichever occurrences sit on the losing side of a strict majority. A close split, or an exact tie, decides nothing — the check stays silent rather than guess, and a genuinely mixed-idiom project simply won't be told what to do about it. That is a deliberate absence, not an oversight: teaching a linter to decide which idiom a team should prefer is a different, much harder problem than teaching it to notice they haven't picked one.

Structural echoes: Design.DuplicateTypeShape and Design.EffectLeakage

Two more checks look for a different kind of project-wide inconsistency — not statistical, but structural. Design.DuplicateTypeShape compares every interface and object-literal type alias in the project against every other one, scoring how much of their field structure overlaps, and flags pairs that are similar enough to suggest one type was invented twice under different names. A floor on how many fields must match, and a similarity threshold below full agreement, keep it from flagging two-field coincidences or interfaces that only vaguely resemble each other.

Design.EffectLeakage instead polices a promise. A file marked @pure is telling its readers — and any tooling built on that annotation — that it has no side effects. The check follows the file's import chain outward, hop by hop, until it either runs out of imports or lands on a module known to touch the filesystem, the network, or a database. A pure-annotated file whose own code never touches fs can still be lying, if the disk access is one import away rather than zero. Finding that requires walking the graph the same way ImportFanOutOutlier and BarrelReexportBloat do — it just asks the graph a different question.

The missing negative: Design.MissingTestFile

The other five checks all look for something present that shouldn't be — a duplicate, an outlier, a minority idiom, a broken promise. Design.MissingTestFile looks for an absence: a source file with no plausible corresponding test anywhere the project's naming conventions would put one. That inversion makes it the most configuration-hungry check of the six, because "plausible" is doing a lot of work — a project's test-naming convention, its framework-entry-point exemptions (a Next.js page.tsx isn't unit-tested the way a utility module is), and its file-extension habits all have to be supplied or guessed correctly before an absence is meaningful rather than noisy.

What validation is for

None of this is trustworthy on the strength of its own fixtures. A hand-built example proves a check fires on the shape it was written to catch; it says nothing about what the check does to a hundred thousand lines of code nobody wrote with that check in mind. The node_modules fan-out bug above didn't exist in any fixture — cofferdam's own test suite for ImportFanOutOutlier was, and remains, green throughout. It only existed in a real codebase's import graph, where a UI library happened to be imported often enough to look, statistically, exactly like a god module. Running each of these six checks against a real Next.js project and reading the findings by hand — not just counting them — is what turned that resemblance into a fix instead of a standing false positive. The other five checks came through the same process with nothing to show for it beyond confirmation: real findings, on real code, that looked exactly as intended. That is itself worth recording, if only because it's the least interesting possible outcome and the one every check should be aiming for.

MIT License