A reverse LLM query starts with ChatGPT’s answer. You feed a model the questions your buyers actually ask, then treat the response as data that shows which businesses it named, which topics it tied to the category, which sources it cited, and how it framed each one. That gives you a visibility map of the AI layer of search, not a guess.
Most brands still measure visibility the same way they did five years ago, tracking keyword rankings, impressions, and a Search Console graph. Meanwhile a growing share of buyers never see a results page at all. They ask an assistant who to hire, where to buy, or which company to choose, and they act on the shortlist it hands back. If your business is not in that shortlist, you are invisible in the exact moment the decision gets made.
The fix is not another subscription tool. It is a repeatable audit you can run yourself for a few cents, using an agent, a short Python script, and an API key. Here is exactly how to build it.
Core Concepts on reverse LLM audits.
- What a reverse LLM query is, and why running it backward from the answer is the point.
- How to generate real buyer-intent questions from a site audit instead of inventing them.
- How the
llm_visibility_audit.pyscript turns raw AI answers into structured, comparable data. - What the report actually tells you about competitor mentions, recurring topics, cited sources, and sentiment.
- How to convert visibility gaps into a content plan, and how to schedule the whole thing so it re-runs itself.
Who does this apply to
Marketers, agency owners, and business owners who already care about search and want to know whether AI assistants recommend them. It is equally useful for a brand-new site trying to find its footing and for an established brand that wants to confirm it still owns its category when the answer comes from a model instead of a results page.
What a Reverse LLM Query Actually Is
A normal SEO audit runs forward. You start with your site, pull your keywords, check your rankings, and try to reverse-engineer intent from there.
A reverse LLM query flips the direction. You start with the finished AI recommendation and trace backward to the signals that shaped it. You give the model buyer questions such as who it recommends for a service, where to buy a specific product, or what to look for when choosing a company. Then you stop reading the answer like a human and start parsing it like a dataset.
Three things matter in every response:
- Which businesses got named. That is your real competitive set in the AI layer, and it is often not the same list you see in organic search.
- Which topics and subtopics the model attached to the category. That is the language the model associates with buying decisions, which is a direct content brief.
- Which sources it pulled from. When the model has web search enabled, the citations show you which publications, directories, and pages are feeding the recommendation.
One rule makes or breaks the exercise. Do not put your own brand name in the prompt. If you type your company into ChatGPT and ask about yourself, the model will happily describe you. That proves nothing. The whole value comes from asking a neutral buyer question and seeing whether you surface on your own.
From experience: I ran this against my own site knowing full well I would not show up. The domain is roughly three months old. The audit confirmed it, and that was still worth doing. Confirming a gap with data is what turns “I should probably write more” into a specific list of topics I am missing and the exact questions where competitors are winning.
Search Did Not Die, It Moved
There is a loud narrative that search is dead. It is not. If search were dead every time discovery changed shape, it would have died when the Yellow Pages did.
What changed is how people search. Buyers still have the same intent and still find what they are looking for. They just increasingly get there through an AI Overview, an assistant, or a chat thread instead of ten blue links. That means you now have to win on three surfaces. Classic organic results, AI-generated summaries inside search, and direct assistant recommendations each behave a little differently. Google’s own guidance on AI features in Search frames AI surfaces as an extension of search, not a replacement for it, which is exactly why abandoning fundamentals is the wrong reaction.
A reverse LLM query is how you measure that third surface. It is closer to a share-of-voice study than a rank check, and it is the same instinct behind how agentic search is reshaping discovery for anyone selling to a buyer who now delegates research to a model.
Step 1: Generate Buyer Questions From a Real Site Audit
Skip the brainstorm. Have your agent run a standard SEO audit of the site first, then ask it to produce five phrases worth testing in a reverse LLM query based on what it found.
That sequencing matters. Questions pulled from an audit map to documented visibility gaps, so the output is actionable instead of arbitrary. In practice the five questions land in predictable shapes:
- “Who are the best agencies for
serviceinmarket?” - “What should I look for in a white label
servicepartner?” - “Who are the best
categoryconsultants?” - “What does a full funnel
serviceengagement include?” - “How do I choose a
servicecompany?”
Five to ten questions is the right starting range. Save them to a plain text or Markdown file, one per line, because the script reads that file as an input.
Step 2: Run the Audit Script
The agent runs llm_visibility_audit.py with three inputs. It needs the business being checked, the file of buyer questions, and the model to query. The script itself is about a hundred lines. There is no magic in it, which is the point, because you can read it, change it, and trust the output.
Here is the loop:
- Send each buyer question to the model through the API.
- Capture the normal, human-readable answer.
- Ask for a second, structured pass on that same answer that extracts businesses, topics, subtopics, sources, and sentiment.
- Normalize duplicate business names so one company listed three different ways counts once.
- Aggregate everything across all questions into a single report, with the raw answers appended.
That last step is what keeps this defensible. Structured extraction can misread an answer, so keeping the original responses in the file means you can spot-check any line item instead of taking the summary on faith.
Authentication is standard. You create a key on the API platform and the script calls it directly using the patterns in the OpenAI API reference documentation. If you already pay for a Codex subscription, you can route the calls through that instead of billing raw API usage, the same way running Codex inside Hermes agent lets an existing plan do the work.
Because the whole thing is just API calls plus a prompt, it is not locked to one provider. Add a key for another assistant, another frontier model, or an open-weight model and run the same question set across all of them in parallel. Different models return different shortlists, and those differences are useful signal on their own.
Use Cheap Models to Audit and Strong Models to Write
The most common mistake is spending frontier-model money on a task that does not need it. Asking whether a brand appears in an answer is a retrieval and formatting job, not a reasoning showcase.
| Stage | Model class to use | Why it fits |
|---|---|---|
| Site audit and question generation | Fast mid-tier model, high reasoning effort | Needs speed and breadth, not creativity |
| Running the reverse LLM query | Cheapest capable model with web search | You are collecting answers, not writing them |
| Parsing a long results file | Long-context speed model | Chews through a large Markdown report in seconds |
| Drafting and polishing the content | Strongest creative model, plus a human pass | Where voice, nuance, and judgment actually pay off |
In practice, five to ten questions on a cheap model costs a couple of cents. I have run the same workflow with a hundred questions in a single batch, and it still lands in pocket-change territory. Compare that to a tool subscription that bills monthly whether you run an audit or not.
Step 3: Read the Report Like a Marketer
The raw report is long. It repeats the question, lists the brands mentioned, tags sentiment, and includes the full response for every prompt. Nobody needs to read it end to end. Hand it to a fast long-context model, like the new GLM 5.3, and ask for the specific decisions you need.
The prompt that produces usable output asks for three blog ideas and one new page to build this month, based on the findings, with the target topics and phrases named and an explanation of how each one gets worked into the content. Two additions make it much better:
- Ask it to explain its reasoning. Knowing why a topic was chosen tells you whether the recommendation is grounded or a guess.
- Tell it not to name competitor brands in the output. You want the topic map, not a competitor teardown pasted into your own content.
What you are hunting for in the summary:
- Buyer questions where competitors appear consistently and you never do.
- Topics and subtopics that show up across multiple answers, since repetition signals what the model treats as decision criteria.
- Sources the model leaned on, which doubles as a placement and digital PR target list.
- Sentiment patterns, because being mentioned dismissively is a different problem than not being mentioned at all.
Then pick the single strongest missing topic and build the asset. Not ten. One good, sourced, genuinely useful page beats a batch of thin posts, and it gives you a clean before-and-after when you re-run the audit.
Close the Loop With Schema, FAQs, and a Re-Run
A gap list is only half the job. The second half is making the page easy for a model to quote:
- Answer the primary question in the first sentence of each section so an extractive system can lift it cleanly.
- Add structured data and an FAQ block that mirrors the buyer questions you tested.
- Keep declarative headers for most sections and save question-style headers for the FAQ.
- Publish the page, then run the identical question set again in two to four weeks.
That re-run is the measurement. Same questions, same model, different date. If you start appearing, the content worked. If you do not, you have an authority problem rather than a topic problem, and the cited-sources column tells you where to go earn it.
Automate It So You Stop Prompting It
Once the workflow is stable, there is no reason to babysit it. Schedule it to run every two weeks or once a month and deliver the report to email or a chat channel. Chain the steps into a queued multi-step job so the agent runs the site audit, generates the phrases, executes the reverse LLM query, and drafts the content plan while you are doing something else.
Two practical notes from running this repeatedly. First, keep the output in the project folder rather than a chat session, so the report, the questions file, and the drafts stay together and can be revisited in a fresh session later. Second, being able to swap models mid-workflow matters more than it sounds. Start on a cheap model for the audit, switch to a long-context model to parse, then switch again to draft, all inside the same project.
The Skill Stack Behind the Workflow
This is not one prompt. It is a set of skills the agent selects on its own, which is why it holds up across clients:
- Website and SEO audit: establishes the baseline and surfaces the initial gaps.
- LLM visibility audit: buyer-intent questions, model answers, brand mentions, topics, content gaps.
- Answer engine optimization playbook: converts those gaps into schema, FAQs, and on-page fixes.
- Content research and writing: blog plan, evidence gathering, drafting, and editorial review.
- Ranking page and text enhancement skills: applied afterward to turn findings into new pages and tightened existing copy.
The underlying agent is the open-source Hermes agent from Nous Research, which is what makes the skill-selection behavior possible. You describe the outcome and it retrieves the right skill instead of waiting for you to name it. If you want to see the writing half of this in practice, it is the same foundation as the Hermes blog and content workflow walkthrough. Build the skills once, customize them to your process, and the marginal cost of every future audit is a few cents and a scheduled job. The marketing strategist behind these workflows runs them on client accounts and on this site the same way.
Reverse LLM Query vs. Traditional Rank Tracking vs. Brand Prompting
| What you are comparing | Reverse LLM query | Traditional rank tracking | Typing your brand into ChatGPT |
|---|---|---|---|
| Starting point | The AI answer to a buyer question | Your keyword list | Your brand name |
| What it measures | Whether you are recommended, and to whom | Position on a results page | Whether the model can describe you |
| Competitive insight | Names the brands the model actually surfaces | Ranks domains you already track | None |
| Content output | Topic and subtopic gaps tied to real questions | Keyword gaps | None |
| Source visibility | Shows which citations shaped the answer | Not applicable | Rarely useful |
| Cost | Cents per run | Monthly subscription | Free |
| Bias risk | Low, brand is never mentioned in the prompt | Low | High, the prompt hands over the answer |
| Best used for | Measuring and closing AI visibility gaps | Tracking classic organic performance | A quick sanity check, nothing more |
Frequently Asked Questions
What does a reverse LLM query cost to run?
A batch of five to ten buyer questions on an inexpensive model costs a couple of cents. Even a hundred-question run stays in pocket-change territory. The larger cost is the one-time work of building the script and skills, which you then reuse indefinitely.
Do I need an API key, or can I just use ChatGPT?
You need programmatic access to collect structured, comparable data at scale, so an API key or a routed coding subscription is required. You can absolutely spot-check a single question in a chat window, but you cannot aggregate, deduplicate, or track it over time that way.
Should I run this against more than one model?
Yes, once the workflow is stable. Different assistants return different shortlists and cite different sources, so running the same question set across several providers shows you whether a gap is universal or specific to one platform. Start with one, then expand.
Why should the prompt avoid my brand name?
Because naming your brand guarantees the model talks about it, which destroys the measurement. The only meaningful result is whether you appear unprompted in the answer to a neutral buyer question.
How often should this be re-run?
Every two to four weeks is enough to see movement without chasing noise. Keep the question set, the model, and the wording identical between runs, since changing the inputs makes the comparison meaningless.