Marketplace index control for volatile listing inventory

On a marketplace with 60,000 live listings we typically find Google crawling three to six times that many URLs. The extra ones are facet combinations, sold items, sort orders and session parameters, and they are not harmless. They consume the crawl that should be reaching today's inventory, which is why a new listing waits nine days for its first visit instead of two. This engagement closes that gap and keeps it closed through your release cycle. Where we start: Raw logs, thirty days minimum, from Cloudflare Logpush or nginx, loaded into BigQuery. We separate verified Googlebot by reverse DNS, then answer four questions. What share of crawl goes to URLs that should never be indexed. How long a new listing waits for first crawl. Which templates get recrawled and which are visited once and abandoned. And whether the pattern changed after your last release. Sampling crawlers guess at this. Logs do not. Facets and parameters: Every marketplace ends up with a facet grid capable of producing millions of combinations, and each one has a defensible argument for existing. We make the call instead of listing the options. Facets with genuine standalone demand get a clean static path, an indexable page and a content requirement. The rest are noindex-follow, blocked in robots, or moved behind a POST or a fragment so they never become URLs at all. Our threshold is live inventory, not search volume: under three live listings a facet page is noindex regardless of how many people search for it, because a page that lands a buyer on an empty grid costs more than the click is worth. Expired inventory: This is the decision most marketplaces get wrong, and in pets, game keys and second-hand sports equipment it is most of the catalogue. A sold item that earned links and rankings should not 404 and should not be blanket-redirected to its category. We usually keep the URL, mark availability honestly in Product and Offer markup, say plainly that the item is gone, and put live alternatives above the fold. Items that never earned anything get a 410 so the index clears rather than rotting. The rule is written per template with the numbers that justify it. Then sitemaps, split by freshness rather than by category, so the file holding the last 48 hours of listings stays small and lastmod means something. IndexNow for Bing and Yandex, which is one endpoint and works. Hreflang across sv, no, da and fi if you run separate Nordic storefronts, plus a hard look at whether you should be running four. Not included: We do not write category descriptions and we are sceptical of the 300-word block under the grid. We do not implement. The output is a specification precise enough for your engineers to build from, with acceptance criteria, and we review the pull requests. We do not manage paid listings, marketplace ads or Google Shopping feeds. Who should not hire us: Marketplaces under roughly 5,000 listings. That is a two-day fix, not an engagement, and we will point you at what to read. Teams on a hosted platform they cannot modify — if response codes and URL structure are locked, our diagnosis will be accurate and useless. Anyone who needs the traffic curve to bend before the next release train, since nothing here ships without engineering time. What lands: A crawl allocation model with before numbers, the facet and parameter ruleset, the expired-listing policy per template, sitemap and lastmod specification, and monitoring so that the next release does not quietly undo the work.