LLM Brand Mention Tracking on a Fixed Prompt Panel

Ask an assistant the same question thirty times in one afternoon and it will not name the same set of brands thirty times. For a brand sitting mid-pack in our panels, the ordinary run-to-run spread is fifteen to twenty percentage points, which means a report built from one screenshot per prompt per month is measuring weather and calling it climate. We run each prompt thirty times per engine per cycle and report a proportion with an interval around it, so that when a number moves you can tell whether anything happened. The panel: Sixty to a hundred and twenty prompts, written in the words your buyers use, held fixed for the quarter. They are grouped by what the person is actually doing: is this treatment covered, who does this in this suburb, is this agency or that one better for an off-market sale, is this business legitimate. Prompts added mid-quarter go into a separate cohort so the trend line stays comparable. If we quietly swapped prompts every month, every chart would slope upward and none of it would mean anything. Two answer modes, kept apart: Every prompt runs twice where the engine allows it, once with browsing and once without. The distinction decides your remedy. A mention that appears without retrieval comes from training data, moves on model release cycles, lags your reality by months, and responds only to what the open web said about you a year ago. A mention that appears only with retrieval is live and responds to work done this quarter. Blending the two into one figure is the most common way these reports mislead the people paying for them. What we extract per run: Whether you are named and in what order, which competitors are named alongside you, the attributes attached to you in the sentence itself, bulk billing, after hours, no auction, fixed fee, sentiment where it is not neutral, and every domain cited in the answer. Third-party sources that keep recurring are ranked, because they are the lever. In Australian health and property the answer is usually a portal, a directory, a government page and two forum threads, and no amount of work on your own site substitutes for what those say. What arrives each month: One page on what changed and why we think so. The raw runs as JSONL, with prompt, engine, timestamp, browsing state and full answer text, so you can recount our figures and tell us we are wrong. A call, which is the part that matters. What we refuse: No composite AI visibility score. A single number blending five engines, two answer modes and a mixed prompt panel cannot go down for a reason anybody can act on, and its only real function is to look like progress in a board pack. No guarantees of movement, because the largest input is what third parties publish about you. No astroturfing, no review solicitation for AHPRA-regulated practitioners, no accounts created to seed threads. Who this is not for: Anyone who wants a dashboard rather than a decision. If nobody reads the monthly call, the retainer is a subscription to a PDF and we would rather end it, which we have done. Also not for brands in a category where every assistant answers from training data with a two-year lag and nothing we measure will move for three quarters. We can usually tell within the first cycle, and we will say so and recommend pausing rather than billing you to watch a flat line. Agencies: Reports ship unbranded and we will join client calls as your analyst. We do not take the client relationship, and we will not run a panel for a competing brand in the same city while your retainer is live.