Automating an Internal Linking Audit with Screaming Frog and the ChatGPT API

The exact Screaming Frog crawl and ChatGPT API prompt that surfaces real internal-linking gaps, using this site's own real numbers as the worked example, including the trap of trusting the default Orphan Pages report alone.

Most "internal linking audit" guides describe the concept and skip the part where you'd actually find the gaps. Here's the exact Screaming Frog crawl, the exact ChatGPT API prompt, and the real numbers from auditing this site's own internal links, including the finding that would have looked clean if we'd stopped at Screaming Frog's default report.

What this workflow actually does

Screaming Frog crawls the live site and exports every page's inlink count. The ChatGPT API then reads a sample of thin-on-links pages alongside a sample of existing content and suggests specific, editorially sensible places to add a contextual link, with anchor text. It does not touch the site. The output is a reviewable list of suggested links for a human to add, because a model that's never read your full site can suggest a technically-relevant but editorially awkward pairing, and the fix for that is a human scanning the list, not a smarter prompt.

Inputs you need before you build this

• Screaming Frog SEO Spider, the licensed version. The free tier caps a crawl at 500 URLs, and a site with a tools directory, a workflow library, and a blog clears that fast. Licensing also unlocks the CLI mode this workflow runs on.
• The site's sitemap.xml (this site has one, generated dynamically, so it always reflects the current page count).
• An API key for the model you'll use for the suggestion step. This uses ChatGPT's API; any capable model works.
• A spreadsheet or CSV to hold Screaming Frog's export and the model's suggestions side by side.

Map the logic: four steps, in order

1. Crawl the site headless via Screaming Frog's CLI. Screaming Frog's command-line interface can run a full crawl and export specific tabs without opening the desktop UI, which is what makes this repeatable instead of a one-time manual pull. Export the "Internal:All" tab and the "Links: All Inlinks" bulk export, the two together give you every page and exactly which other pages link to it.
2. Run the built-in Orphan Pages report, then don't stop there. Screaming Frog's Orphan Pages report finds pages with zero crawled inlinks. On a directory site like this one, that report will likely come back clean, or close to it, because every tool, workflow, and post is already linked from its own index page. That's a false sense of security: an index-page link and pagination link carry far less topical relevance than a link from a paragraph of actual content about that exact page. The real audit filters for pages whose ONLY inlinks are from list/index pages, with zero inlinks from editorial content, since those are functionally invisible to a reader browsing the blog and weaker signals to a crawler weighing topical relevance.
3. Cross-reference against your own content-relations data, if you have it. This site already tags every blog post with which tools and workflows it relates to. Pulling that data directly, instead of re-deriving it from the crawl, turned a fuzzy "which pages feel under-linked" question into an exact count. If your site doesn't track this explicitly, approximate it by grep-ing your CMS content for internal link targets instead of guessing from the crawl alone.
4. Feed the candidate list to the ChatGPT API for suggestions. For each under-linked page, pass the model that page's title and a short description, plus a sample of your existing content that could plausibly link to it (titles and excerpts, not full text, to keep the prompt a reasonable size). Ask for a small number of realistic single-sentence link suggestions with anchor text, not a rewrite of the source content.

The real audit: this site's own numbers

Running that cross-reference against The Workflow Finder's own data (384 tools, 192 workflows, 33 blog posts, all pulled live, not from memory) surfaced a real gap: 155 of 192 workflows have never been linked from a single blog post. The most-linked workflows (build-personal-knowledge-base-with-ai and multi-agent-research-assistant, 4 blog links each) sit next to dozens with zero, including straightforward, useful ones like "Build a Lead Generation System" and "Automate Customer Support," both Advanced-tier workflows that are exactly the kind of practical topic a blog post would naturally reference in passing.

The tool side shows the same pattern at a larger scale: 313 of 384 tools have never appeared in a blog post's related-tools list, and 210 of 384 tools appear in neither a blog post NOR a workflow's recommended-tools list, meaning their only inbound links on the entire site come from the /tools directory index itself. Meanwhile a small handful of tools (Claude, ChatGPT, Gemini) are referenced across more than half the posts published, a natural but lopsided result of writing about generative AI most often.

None of this shows up as a Screaming Frog "orphan page." Every one of those 155 workflows and 313 tools is fully crawlable and indexed, linked from its category page like every other entry. The gap is contextual link density, not crawlability, and it's invisible unless you specifically compare crawler inlinks against editorial content relations rather than trusting the orphan report alone.

The exact prompt

This is the message content for the suggestion step, run once per under-linked page against a batch of candidate source pages.

You are a content editor looking for natural places to add an internal link to an under-linked page on an AI tools and workflows site.

Under-linked page: "{{ page_title }}" -- {{ page_description }}

Candidate source pages (existing blog posts that could plausibly reference it):
{{ candidate_pages_titles_and_excerpts }}

For each candidate source page where a link to the under-linked page would be a genuinely natural fit for a reader, not a forced insertion, return:
{
  "source_page": "string, the exact title of the candidate page",
  "anchor_text": "string, 2-6 words, the exact phrase that would become the link",
  "sentence_context": "string, the one sentence this anchor text would sit inside, written in the source page's existing voice",
  "confidence": "high" | "medium" | "low"
}

Rules:
- If none of the candidate pages are a natural fit, return an empty array. Do not force a link into a page just because it was in the candidate list.
- Anchor text must describe what the linked page IS, not a generic phrase like "click here" or "this workflow."
- Never suggest more than one new link per source page in a single pass, a paragraph that suddenly links three things in a row reads as SEO manipulation, not editorial judgment.

The "return an empty array rather than force it" rule is the load-bearing line. Without it, the model will confidently propose a link for every candidate you feed it, whether or not it actually makes sense, and you'll spend more time rejecting bad suggestions than you saved by not writing them yourself.

Where this breaks

• The orphan-page trap, if you skip step 2's filter. Trusting Screaming Frog's default Orphan Pages report alone on a directory-style site tells you almost nothing is wrong, because almost nothing is a literal zero-inlink orphan. The actual gap only shows up once you separate index-page links from editorial-content links.
• Model suggestions still need a human sanity check. "High confidence" from the model isn't the same as editorially correct; a suggestion can be topically related and still read as awkward or promotional in the actual sentence. Review before adding, every time.
• This list goes stale the moment you publish. A one-time audit is a snapshot. New posts change the under-linked list immediately; treat this as a recurring check, run monthly at minimum, not a task you close out once.

The verdict

Screaming Frog is the right tool for the crawl and inlink-counting half of this because that's precisely what a crawler is built to do accurately at scale, and its CLI mode is what makes the crawl repeatable instead of a manual click-through every time. The ChatGPT API is the right tool for the suggestion half because judging whether a link reads as natural or forced is an editorial call, not a graph-traversal problem. Run this against your own site's real numbers, not a hypothetical, and you'll likely find the same shape of gap this audit found here: nothing crawlable is missing, but most of the catalog is still waiting for its first real editorial link.