Duplicate Content Checker: Compare Your Pages

Two URLs can rank for nothing useful when they say the same thing with different slugs. Marketing calls it "syndication." Search engines call it a pick-one problem. You do not need a lecture on duplicate content theory to fix it on your own site. You need to know which pairs are too similar and what text they share. This guide walks through a practical order: pick candidates, compare body copy, decide consolidate vs differentiate, then wire canonicals or redirects. The matching tool is the Duplicate Content Checker on Scrawl: 2–5 URLs or pasted text blocks, main-body extraction, pairwise similarity, shared passages, and an 80%+ near-duplicate flag. It compares only what you enter , not the web. Start with a short list, not a full crawl Duplicate panic often starts with someone exporting every URL from analytics. That is the wrong first move. Start from: Pairs you already suspect (two service area pages, two SKU variants with the same spec sheet) URLs that swapped positions in Search Console after a template change Drafts from writers that "sound familiar" Put those URLs in the checker. If the list grows past five, run multiple batches grouped by template. Screaming Frog includes near-duplicate detection in its paid licence when you need sitewide coverage. This browser tool is a quick first pass on a handful of pages. What the checker actually compares For URL mode the tool: Fetches each page (initial HTML, no JavaScript rendering) Removes nav, header, footer, scripts, and similar chrome Extracts visible body text from article , main , or body Tokenizes words and builds shingles for Jaccard similarity Surfaces the longest shared word sequences between each pair Paste mode skips the fetch and compares text you supply. Use it when staging is blocked, when you only care about the article field…