What is Screaming Frog and how to use it

Screaming Frog is a local program for technical website auditing that scans your site just like a search engine bot does. It visits every page, “reads” the code, analyzes tags, headings, meta data, statuses, canonicals, links, and dozens of other parameters. It is one of the most powerful and accurate tools for professional SEO diagnostics. It is used by specialists when they need to thoroughly check the technical condition of a website: from its structure to minor errors that are invisible to the eye.

The main advantage of Screaming Frog is that it works locally. Unlike cloud services, the program is installed on your computer and works directly through your connection. This gives you complete flexibility: you can configure the depth of the crawl, exclude or include certain types of URLs, process sites with authorization, and work with huge projects without any page limits (in the paid version). At the same time, the analysis is faster and more in-depth than many web services.

Screaming Frog is like an X-ray: it shows what the user, CMS editor, or even a web developer cannot see. With it, you can identify:

• hidden redirects and redirect chains,
• broken links leading to non-existent pages,
• duplicate H1, Title, and Description tags,
• missing alt tags for images,
• canonical and hreflang errors,
• pages closed from indexing,
• problems with architecture and nesting depth,
• unnecessary 301 and 302 redirects within the site.

If you offer SEO solutions for businesses, Screaming Frog is a must-have for your audit. With it, you don’t just state that “something is wrong,” you pinpoint exactly where the error is, what it is, and how it affects indexing and ranking. The tool is especially relevant for large websites (500+ pages), online stores, media platforms, corporate portals, and projects with frequent updates. It is used both during the development stage and for regular checks of existing resources. And if you are engaged in SEO consulting, Screaming Frog will allow you to generate a map of all technical risks that prevent your website from growing in just an hour.

Read also: What is the Ahrefs tool.

Installing and launching Screaming Frog: step-by-step guide

Screaming Frog SEO Spider is a desktop application available for Windows, macOS, and Linux. To get started, go to the official website screamingfrog.co.uk/seo-spider/ and download the distribution package. The program is free for up to 500 URLs, which is enough for small websites or a trial run. For large projects and a full audit, you will need a paid license (around £259/year), which removes limits and unlocks advanced features: integration with GSC, GA, configuration saving, JavaScript rendering, Custom Extraction, and more.

After installation, a minimalistic interface opens: an address bar at the top, a tab bar in the center, and settings and filters on the side. To start, simply enter the website URL (starting with https://) and click the Start button. The program begins scanning: it goes to the home page, follows internal links, and reads all parameters from the HTML code. Within a minute, the first data appears: a list of URLs, titles, meta tags, statuses, page size, depth, etc.

Before a full scan, it is worth configuring the basic parameters. Through “Configuration → Spider” you can set:

• the depth of the crawl (by default — unlimited),
• the type of files to analyze (HTML, JavaScript, PDF, images),
• ignoring external links,
• taking into account redirects and URL parameters,
• including/excluding pages by template (for example, all ?page= or /cart),
• JavaScript rendering (if the site runs on React/Vue — enable).

It is also important to specify the user-agent: Googlebot, Googlebot Smartphone, or Screaming Frog. This affects the behavior of some sites, especially those where access to content depends on the user-agent. In the “Speed” section, you can limit the number of threads to avoid overloading the server (especially important when scanning client sites).

If you are conducting SEO consulting for your website, project setup is not just about “running a scan.” It is about controlling what you will see: what errors will be recorded, what will be included in the report, and how the data will be interpreted. Incorrectly set parameters will give a distorted picture, and even the most powerful tool will not help. After starting the scan, it is important not to touch the interface — the program works with a high load on the network and disk. When the scan is complete (100%), you can save the project, export reports, analyze by tabs, or build visual diagrams of the site.

Screaming Frog interface overview: key tabs and their meaning

After the scan is complete, you will see a table with many URLs and dozens of columns. By default, the Internal tab is open — this is all the pages that are inside the domain you specified. The following information is displayed here: URL path, content type, server response status, headings (Title, H1, H2), description length, canonicals, indexability, nesting depth, presence of hreflang, alt for images, and other parameters. This is the central area where you find the main errors: empty tags, duplicates, pages without H1 or with duplicate descriptions.

The External tab shows all external links found on your site. This is important for monitoring: you can see where buttons, images, affiliate blocks, and links in the footer lead to. Sometimes outdated or broken links to external resources are found here, which harm the quality of the site. The types of links are also displayed here — HTML, JS, PDF, images, etc.

The next important tab is Response Codes. This is where you will see which pages return a 200 (OK) status, which return a 301/302 (redirects), which return a 404 (not found), 500 (internal server errors), or 503 (site unavailable). Use the filters at the top of the window to quickly highlight problem areas. For example, you can filter only 404s, and you will immediately see which pages link to non-existent content. This is critically important for technical audits.

The Page Titles tab shows the title tags on all pages. Here you can quickly find duplicates, tags that are too long or too short, missing titles, or over-optimization. Similarly, the Meta Description tab shows the page description that is used to form the snippet. Screaming Frog shows the length, presence of duplicates, and formatting issues. You can sort the table by length or uniqueness and immediately identify areas for editing.

H1 and H2 are first and second level headings. The program will show where they are missing, where there are duplicates, and where there are multiple H1s on a page. Important: for SEO, a correct heading structure is not only convenient, but also signals the relevance of the page to key topics.

Canonicals is a section where canonical URLs are recorded. Screaming Frog will show which pages link to themselves (this is normal), which link to other pages (this may be an error or part of the logic), and where canonicals are missing altogether. You can also track conflicts between canonicals and actual URLs here — for example, if a canonical points to a non-indexable page.

If you work with a website that has language versions, the Hreflang tab will be indispensable. It will show all hreflang attributes, their correspondence, the presence of errors, the absence of backlinks, incorrect country codes, and duplicates. This is an important element in an international strategy.

Additionally, the Images, Directives, AMP, and Structured Data tabs provide advanced control, especially if the site uses microdata, mobile adaptation, AMP pages, or a complex link structure. Understanding these tabs allows you to turn a site scan into a systematic audit: not just finding errors, but building a strategy for improvement. And if you provide SEO solutions for businesses, this means that you can not only point out the problem, but also explain why it is hindering growth and how fixing it will affect the site’s visibility.

How Screaming Frog helps with technical SEO audits

When the scan is complete, you are presented with more than just a list of pages — it is a technical picture of the entire site that no CMS or visual interface can provide. And this is where the real use of Screaming Frog SEO Spider begins. It is important not just to see the errors, but to understand their nature, filter out the insignificant ones, and identify critical areas. In this section, I will show you what data to look for, where to find it, and how to use it for a real audit, especially if you are conducting SEO consulting for your website.

It is always worth starting with an analysis of server response statuses. The Response Codes tab allows you to filter out pages with 4xx errors — these are broken links that either lead nowhere or no longer exist. It is important to fix them: search engines perceive such errors as an indicator of poor architecture. Next, analyze redirects. If you see 301 → 301 → 200 or 302 on pages where there should be a permanent address, this is a signal for optimization. Such chains slow down bot crawling, lose weight, and can interfere with the indexing of new pages.

The next step is to check the titles. In the Page Titles tab, look for duplicate or missing titles. If you have 15 pages with the same title, it means that the bot will not understand the differences and some of them may be dropped from the index. The same applies to Meta Description — here, it is important not only that it is present, but also that it is unique. Screaming Frog shows the length of descriptions and highlights those that are truncated, which affects the clickability of the snippet. Next comes H1 — if it is duplicated, missing, or used more than once, this is a reason for audit notes.

Special attention should be paid to the canonical tag. In the Canonicals tab, you can see which pages point to themselves, which point to other URLs, and which do not contain a canonical tag at all. An error occurs when the canonical tag leads to a page with parameters or a non-existent version. There are also conflicts: for example, a page is open for indexing, but the canonical tag points to a noindex page. Or vice versa — the canonical is correct, but the page itself is closed in robots. All this confuses the bot and can lead to partial or complete loss of indexing.

It is also important to look at the Directives tab. This tab contains all signals for search engines: noindex, nofollow, canonical, hreflang, robots.txt. Analyze which pages are actually accessible and which only appear to be so. Very often, sections that are open in the navigation are closed by a random noindex meta tag — Screaming Frog shows this, and it is not always noticeable to the eye.

And if all this is accompanied by exporting reports, prioritizing errors, and adding them to checklists, you turn a technical audit into a systematic task. This is especially important if you offer SEO solutions for businesses — after all, technical barriers are often what prevent a website from growing, even if the content and structure seem fine at first glance.

Read also: What is keyword growth.

How to filter, export, and format SEO reports in Screaming Frog

Once you have scanned the site and figured out the key tabs, the next task is to organize everything you have found. It is one thing to see duplicates and errors within the program, but it is another to prepare a clear, structured, and useful report. Screaming Frog is well suited for this: you can flexibly filter data, export it to tables, highlight blocks by priority, and generate a visual map of the site. This is especially important if you are preparing an audit as part of SEO solutions for business, where you need to explain the essence to the client, not just show them numbers. Working with filters is the first step towards clarity. Within each tab, you can select one of the system filters: for example, in the Page Titles tab, you can select “Missing,” “Duplicate,” “Over 60 Characters,” or “Below 30 Characters.” These filters instantly narrow down the data array to a specific error.

If you filter out pages without a title, you immediately get a list of URLs that require attention. It works the same way for H1, Canonical, Alt for images, Description, and other elements. This allows you to proceed systematically, focusing on areas where there are actual issues. Once the desired group of pages has been filtered, the next step is to export them. Right-click and select Export or Bulk Export from the main menu. You can export to CSV, Excel, or TSV, which is ideal for further work in Google Sheets, analytics, or transfer to the development department. You can export:

• all pages with 404 errors,
• redirect chains,
• duplicate H1 or Title tags,
• pages without Description or Canonical,
• all URLs closed from indexing.

Exports can be grouped by error type, which makes it easier to prioritize tasks. For example, when moving a site, you first export the old URLs with canonicals, then the new ones with redirects, and check that the redirection worked correctly. All this is done in one click and does not require manual data collection from CMS or external tools.

Additionally, Screaming Frog SEO Spider offers visualization through the “Site Structure” tab or a separate “Visualizations” section. Here you can build interactive diagrams:

• internal linking map (Force-Directed Diagram),
• depth of nesting (Tree Graph),
• transitions between URLs (Crawl Path).
• Such diagrams are especially useful when analyzing site architecture.

You can immediately see which pages are “hanging” without incoming links, where there are isolated clusters, and which nesting levels are overloaded. This helps not only to identify errors, but also to suggest logical improvements: for example, add interlinking, simplify navigation, or merge low-level pages. If you work with clients, it is convenient to insert Screaming Frog exports into custom audit reports. You can compare: error — its impact — recommended solution — result. And then even a large file with 5,000 lines becomes a readable and understandable document, especially when it comes to SEO consulting for your website in Kyiv, where trust and transparency are more important than the number of terms.

Scenarios for using Screaming Frog in SEO: from diagnostics to strategy

To get the most out of the tool, it needs to be integrated into real-world processes. Screaming Frog SEO Spider is not limited to one-time checks — it works great as a system assistant: during migrations, indexing analysis, content checks, link control, and even regular automation. Below are key scenarios where Screaming Frog becomes an integral part of an SEO specialist’s methodology.

Site migration and redirect check

The most common case is migration to a new domain or URL structure. First, scan the old version of the site and export all URLs. Then scan the new one and use the “List Mode” feature to load the old addresses to check how they redirect. Screaming Frog will show which old pages are redirected correctly (301), which return a 404, and where an incorrect redirect is triggered (for example, to the home page or with a 302). This allows you to avoid losing traffic and link weight when moving. This is especially important to consider if you are responsible for SEO solutions for businesses, where every lost page means lost rankings and money.

Indexing and closed page audit

Regularly use the Canonical and Directives tabs to check which pages are actually being indexed. This is especially important on sites with dynamic URLs, parameters (?sort=, ?page=), and pagination. Screaming Frog allows you to identify pages with conflicts: when there is a noindex meta tag but there are external links; when the canonical tag points to a URL without an index; when the sitemap includes pages that are not on the site. This provides not just a list of errors, but a complete picture of what the search engine actually sees and what only the user sees.

Content audit: from duplicates to empty tags

The data collected in Crawled Data for the Title, Description, H1, and H2 tags helps you quickly identify duplicates and empty spaces. This is especially useful for online stores, where descriptions are often template-based and meta information generation does not always work. Screaming Frog shows where content needs to be unique, where titles do not match queries, and where structure needs to be added. After exporting, you can send tasks to copywriters or the content department with a ready-made list and priorities.

Internal linking analysis

Visualizations and the Inlinks tab allow you to understand how pages are linked to each other. It often turns out that important pages (such as services or categories) are located at a depth of 4–5 levels and have no incoming links. Screaming Frog shows:

• where internal links are missing,
• which pages link to each other,
• where there are orphan pages,
• which URLs have too many outgoing links.

This allows you to improve the internal architecture, redistribute weight, and strengthen the necessary pages, especially if your goal is SEO consulting for your website in Kyiv with a focus on the growth of specific clusters.

Integration and automation

In the paid version of Screaming Frog, you can connect Google Analytics and Search Console. This allows you to:

• see traffic for each URL directly within the program,
• mark pages with no traffic for deletion or update,
• compare pages that are indexed but do not generate clicks.

Integration with PageSpeed Insights API is also available — you can see speed metrics right in the interface. And with Custom Extraction, you can extract any code snippets: microdata, JSON-LD, Open Graph structure, template schema, UTM parameters — and even custom elements such as the presence of a call-to-action block. The workflow becomes clear: scan, filter, export, transfer to tasks, or visualize immediately. If everything is set up correctly, Screaming Frog ceases to be just a “link analyzer” and becomes the engine that powers your SEO system.