Programmatic Page System Design for Regulated Data

A programmatic page set almost always fails in the same place. The thinnest third of it drags down the rest, and the drag arrives four to seven months after launch, once enough of the set has been crawled for a site-wide judgement to form. So the first thing we produce is not a template. It is the publish threshold: the data density below which a row does not earn a URL, written as a rule your build can enforce. The entity model: We work out what the page is before deciding what it looks like. One entity per URL, one query intent per entity, and an explicit statement of which attribute combinations are entities in their own right and which are filters on one. Most sets we are asked to review have this backwards. They have a page for every facet pair and no page for the thing people actually search for. URL grammar: Facet order fixed, canonical direction declared, and a written rule for which combinations are indexable, which are noindex and followed, and which should never render. We specify parameter handling, pagination, trailing-slash behaviour and the redirect map for the day a source record changes its identifier, which it will. The data contract: Which fields must be non-null for a page to publish. The freshness window in days, per field, after which the page reverts to noindex until the source updates. The source of record for each field and what happens when two sources disagree. This is where the design earns its fee. A set built on a feed nobody owns will decay whatever the template looks like. Index gating and the withdrawal path: New rows publish noindex and are promoted by a scheduled job only once they clear the threshold. Demotion runs on the same job. We also specify how the set comes down: quickly and cleanly, without soft 404s or redirect chains, on the day a regulator, a data partner or your own legal team asks. In regulated categories that path is designed before launch, not improvised during it. Internal linking: Sibling and parent relationships derived from the entity model rather than from a related-items widget, with a cap on links per hub and a rule that stops the hub becoming a sitemap. We specify how the set connects to the editorial part of the site in both directions, because a page set that no human-written page links to reads as a bolt-on. Sample-based quality review: We read thirty rendered pages from across the distribution, best, median and worst, and mark up each one. Reading the template tells you nothing about what the data does to it. Work we have designed: Facility and practitioner directories where advertising rules govern what may appear beside a name. Instrument and fund pages where mandatory disclosure text sits inside the template and shifts the ratio of unique text to boilerplate past the point where the page reads as unique at all. Event and result pages in endurance sport, where data is fresh for a week and stale for a year, so the staleness policy is the design. What we refuse: We do not write the generator or supply content at volume. The build is your engineering team's and we specify against it. We will not design a set targeting prescription medicine terms aimed at the Polish public, because Article 57 of the Pharmaceutical Law forbids the advertising and the pages would be unpublishable however well they were built. We decline sets whose underlying data is licensed against public display, and we check the licence before the kick-off call rather than after. Ten thousand pages built on a design nobody wrote down is not a page system. It is a liability with a sitemap.