Bot & Crawler Simulator

Spider View - What Spiders See

See exactly what search engine crawlers (Googlebot, Bingbot, and AI discovery scrapers) receive when fetching your raw server HTML—before client-side JavaScript executes.

Viewing Spider Inspection for: tonyherman.com SP97275201 View Scan History (31)
Run 144-Point Checkup ↗
🛡️

Security Firewall / Bot Verification Intercepted Request (Firewall Challenge: Checking search engine crawler)

This website's server security firewall (such as Cloudflare, Wordfence, or an anti-bot WAF) intercepted the simulated crawler request and returned a verification/challenge page instead of the real website content.

Why this happens: When simulated crawlers visit with a bot User-Agent (like Googlebot or Bingbot), strict firewalls check if the request comes from Google's official IP ranges (Reverse DNS lookup). Because requests from external test servers are not Google IPs, the firewall flags it as a potential "spoofed bot" or automated scraper.

💡 Tip: Change the dropdown at the top to Standard Desktop Browser (Chrome) or SiteAnalyzerFree Bot to view how the site responds to regular visitors.

32 / 100
Grade F

Spider Crawlability Score: 32 / 100

Critical Crawl Blockers

Crawlers are severely obstructed by client-side JavaScript rendering, firewall challenges, or server errors.

Status: 0/25
Content: 0/20
Headings: 7/15
Links: 0/15
Speed: 15/15
Payload: 10/10
🔒 tonyherman.com
403
HTTP Status
Firewall Challenge
Cloudflare / WAF challenge intercepted crawler request.
4
Words in Raw HTML
Minimal Text
Search bots receive an empty HTML shell requiring JavaScript.
1
Headings Discovered
Empty Headings (1)
Found 1 empty heading tag(s) in source markup.
0
Links in Raw HTML
No Links (Dead End)
Zero links found. Crawler cannot discover other site pages.
240.6ms
Fetch Response Time
Lightning Fast
Rapid response time maximizes crawler budget efficiency.
6.7 KB
HTML Payload Size
Lightweight
Very lean DOM payload. Quick for crawlers to download and parse.
Spider Essentials:

Raw Text Reading View (Lynx Simulator)

This is the exact linear text reading order extracted by search engines, screen readers, and AI discovery agents.

Checking search engine crawler...

Markdown Content View

0 words · 1 chars

Structured Markdown formatted from raw server HTML—ready for Large Language Models (LLMs), AI agents, documentation, and content workflows.

#

DOM Tree & Structure Inspector

16 elements · Max Depth: 4

Interactive tree explorer of the parsed Document Object Model (DOM). Inspect semantic landmarks, tag nesting, element attributes, and CSS selectors.

Total Elements 16 Optimal size
Max DOM Depth 4 Clean hierarchy
Landmarks / Structure 0 header, nav, main, section...
Interactive Controls 0 a, button, input, form...
Media & Embeds 0 img, svg, iframe, video...
Headings & Text 2 h1-h6, p, blockquote, lists
Loading DOM Tree...

🔍 Node Inspector

Select any node

Click any element tag in the DOM hierarchy tree on the left to inspect its complete CSS selector path, XPath, attributes, text snippet, and DOM geometry.

No element selected

Discovered Headings Structure

Showing all 1 headings in page hierarchy

1 Primary <h1> Tag (Optimal)
Heading Architecture & SEO Best Practices
1. Exactly One Primary <h1> Tag Every page should have exactly one <h1> heading defining its primary subject. Having multiple or missing <h1> tags dilutes topical clarity for search engines.
2. Topical Relevance in Subheadings Click on <h2> and <h3> filters to audit your subtopics. Ensure each heading clearly reflects what your website is about and supports your core keywords.
3. Logical Heading Hierarchy Structure content hierarchically (<h1> → <h2> → <h3>). Avoid skipping levels so web crawlers and screen readers navigate smoothly.
4. Capitalization & Case Consistency Maintain uniform casing across headings in each group (e.g., all <h2>s in Title Case where important words/proper nouns are capitalized, or all in Sentence case). Mixing casing styles looks unpolished.
Heading Capitalization Consistency Audit
✓ 100% Uniform Styling

Evaluates whether headings within each group (H1, H2, H3...) consistently capitalize important words and proper nouns (Title Case) or follow Sentence case.

H1 [Empty Heading Tag]

Discovered Images & Alt Tags (0)

Search engine spiders and accessibility screen readers rely on descriptive alt text to understand images.

No <img> tags were found in the raw HTML response.

Search Engine Directives & Metadata

Page Title (<title>) Checking search engine crawler...
Meta Description [Missing Meta Description]
Canonical URL [No Canonical Tag Declared]
Meta Robots Tag [None - Index/Follow by Default]
X-Robots-Tag Header [None]

Structured Data (JSON-LD Blocks: 0)

No <script type="application/ld+json"> schema markup was found in the initial HTML.

Raw HTTP Response Headers

HTTP/2 403 
server: nginx
date: Sat, 19 Sep 2026 21:12:39 GMT
content-type: text/html
strict-transport-security: max-age=31536000
set-cookie: _hcc=NTZiYTI0YTc4OTFkMjY3ZTRkNTlmODU5OTVhYmJhYzU0NDVlYWEyNzE5ZDlmYTRkZTU5NjZhZWZhMTA5Njc1NA==:YTozMzA3MHwxNzg5ODUyMzU5fENocUl2SXlMcFJ7c2xZdGBdUFJ2S2tRSkNQYltqd1l6fDE=; Max-Age=30; Path=/
alt-svc: clear
server-timing: a8c-cdn, dc;desc=dfw, cache;desc=BYPASS;dur=0.0

Raw Server Response HTML

<!DOCTYPE html><title>Checking search engine crawler...</title><noscript><p>Javascript required</p></noscript><meta content="width=device-width"name=viewport><link href="data:;base64,="rel=icon><script>(()=>{const e=2;let n,t,o,i,a,l,r,c,s=-1,u="",d="",h={},f=e=>document.getElementById(e);function p(e,n,t){let o=document.createElement(e);if(n)for(let e in n)o[e]=n[e];return t&&S(o,t),o}const m=new Uint32Array([1116352408,1899447441,3049323471,3921009573,961987163,1508970993,2453635748,2870763221,3624381080,310598401,607225278,1426881987,1925078388,2162078206,2614888103,3248222580,3835390401,4022224774,264347078,604807628,770255983,1249150122,1555081692,1996064986,2554220882,2821834349,2952996808,3210313671,3336571891,3584528711,113926993,338241895,666307205,773529912,1294757372,1396182291,1695183700,1986661051,2177026350,2456956037,2730485921,2820302411,3259730800,3345764771,3516065817,3600352804,4094571909,275423344,430227734,506948616,659060556,883997877,958139571,1322822218,1537002063,1747873779,1955562222,2024104815,2227730452,2361852424,2428436474,2756734187,3204031479,3329325298]),g=new Uint32Array(64),w=new Uint32Array(8);function y(e,n){let t=n+9+63>>6,o=t<<6;e[n]=128,e.fill(0,n+1,o);let i=8*n;e[o-4]=i>>>24&255,e[o-3]=i>>>16&255,e[o-2]=i>>>8&255,e[o-1]=255&i,w[0]=1779033703,w[1]=3144134277,w[2]=1013904242,w[3]=2773480762,w[4]=1359893119,w[5]=2600822924,w[6]=528734635,w[7]=1541459225;for(let n=0;n<t;n++){let t=n<<6;for(let n=0;n<16;n++){let o=t+(n<<2);g[n]=e[o]<<24|e[o+1]<<16|e[o+2]<<8|e[o+3]}for(let e=16;e<64;e++){let n=g[e-15],t=g[e-2],o=(n>>>7|n<<25)^(n>>>18|n<<14)^n>>>3,i=(t>>>17|t<<15)^(t>>>19|t<<13)^t>>>10;g[e]=g[e-16]+o+g[e-7]+i|0}let o=w[0],i=w[1],a=w[2],l=w[3],r=w[4],c=w[5],s=w[6],u=w[7];for(let e=0;e<64;e++){let n=u+((r>>>6|r<<26)^(r>>>11|r<<21)^(r>>>25|r<<7))+(r&c^~r&s)+m[e]+g[e]|0,t=o&i^o&a^i&a;u=s,s=c,c=r,r=l+n|0,l=a,a=i,i=o,o=n+(((o>>>2|o<<30)^(o>>>13|o<<19)^(o>>>22|o<<10))+t|0)|0}w[0]=w[0]+o|0,w[1]=w[1]+i|0,w[2]=w[2]+a|0,w[3]=w[3]+l|0,w[4]=w[4]+r|0,w[5]=w[5]+c|0,w[6]=w[6]+s|0,w[7]=w[7]+u|0}}let v;async function _(e){let t=performance.now(),o=await async function(e,n,t,o){let i=(new TextEncoder).encode(e),a=new Uint8Array(64+(i.length+10+9+63>>6<<6));a.set(i);let l=[];for(let e=0;e<n.length;e++)l.push(parseInt(n[e],16));let r=performance.now();for(let n=0;n<t;n++){let t=""+n;for(let e=0;e<t.length;e++)a[i.length+e]=t.charCodeAt(e);y(a,i.length+t.length);let c=!0;for(let e=0;e<l.length;e++)if((w[e>>3]>>>28-((7&e)<<2)&15)!==l[e]){c=!1;break}if(c)return e+t;1023==(1023&n)&&performance.now()-r>=o&&(await new Promise((e=>{if(void 0===v)try{v=new MessageChannel}catch(e){v=null}null!==v?(v.port1.onmessage=e,v.port2.postMessage(0)):setTimeout(e,0)})),r=performance.now())}return null}(n,e,2e8,10);if(s=Math.round(performance.now()-t),null!==o)return l=o,void b();setTimeout((()=>window.location.reload()),5e3)}function b(){a||!l||r||i&&!u||(a=!0,fetch("/__challenge",{method:"POST",headers:{"X-Hashcash-Solution":btoa(l),"X-Hashcash-Solve-Ms":s,"X-Interactive":u,"X-Hashcash-Host":window.location.hostname,"X-Hashcash-Env":d}}).then((e=>{c&&e.ok&&(c.innerText="Thanks, human!",c.prepend(p("input",{type:"checkbox",checked:!0}))),window.location.reload()})))}function H(e){let n;e=Math.max(e,3);do{for(n="";n.length<e;)n+=Math.random().toString(36).substring(2);n=n.substring(0,e)}while(h[n]);return h[n]=1,n}function T(e){S(e,{display:"none"})}function A(e){S(e,{visibility:"hidden"})}function C(e){S(e,{opacity:0,"pointer-events":"none"}),e.querySelectorAll("input").forEach((e=>e.tabIndex=-1))}function x(e,n){return n=n||0,Math.floor(Math.random()*(e-n))+n}function I(e,n){let t=H(8),o=p("div",{className:"check-wrap"},{top:x(8,1)+"ex"}),i=p("input",{type:"checkbox",id:t}),a=p("label",{htmlFor:t,innerText:"I am human"});return i.addEventListener("change",(()=>{!function(e,n){c=e,e.innerText="Verifying";let t=p("div",{className:"spinner small"});e.prepend(t),u+=n,b()}(o,n)})),e.appendChild(o),o.appendChild(i),o.appendChild(a),o}function S(e,n){let t=document.styleSheets[0],o="c"+H(8),i=Object.keys(n).map((e=>e+":"+n[e]+" !important")).join(";");e.classList.add(o),t.addRule("."+o,i)}function k(e,n){f("head").innerText=e,f("text").innerText=n}function E(){if(i){!function(){let e=f("active"),n=x(10),t=[T,A,C][x(3)];for(let i=0;i<10;i++){const a=n===i,l=I(e,a?o:H(o.length));a?S(l,{opacity:1}):t(l)}}(),k("Confirm you are human","We need to check you're not a robot before you can enter this site."),f("active").style.display="block"}else k("Checking search engine crawler","This will only take a few seconds..."),f("main").appendChild(p("div",{className:"spinner big"}))}window.addEventListener("load",(()=>{!function(a){n=atob(a.split(":")[1]);let l=n.split("|");t=l[3]||0,o=l[4]||"",i=t>=e}(function(e){let n=e+"=",t=decodeURIComponent(document.cookie).split(";");for(let e=0;e<t.length;e++){let o=t[e];for(;" "==o.charAt(0);)o=o.substring(1);if(0==o.indexOf(n))return o.substring(n.length,o.length)}return""}("_hcc")),d=function(){let e={ua:()=>navigator.userAgent,lang:()=>(navigator.languages||[navigator.language||""]).join(","),wd:()=>navigator.webdriver?1:0,tz:()=>Intl.DateTimeFormat().resolvedOptions().timeZone,dpr:()=>window.devicePixelRatio,hc:()=>navigator.hardwareConcurrency},n=[];for(let t in e){let o="";try{o=e[t]()}catch(e){}n.push(t+"="+encodeURIComponent(null==o?"":o))}return n.join(";")}();let a=1500,l=10+x(100);setTimeout(E,l),r=setTimeout((()=>{r=null,b()}),a),i&&setTimeout((()=>window.location.reload()),12e4),_("0000")}))})();</script><style>*{font-family:Helvetica Neue,HelveticaNeue,Helvetica,Arial,sans-serif;color:#111516;margin:0;padding:0;user-select:none;box-sizing:border-box}body{min-height:100vh;min-height:100dvh;padding:8px}#main,body{display:flex;flex-direction:column;align-items:center}#main{flex:1;justify-content:center}h1{font-weight:500;padding-bottom:16px;text-align:center}#active{display:none;position:relative;margin-bottom:calc(48px + 1em + 8ex)}#body{text-align:center}input[type=checkbox]{margin-right:10px}div.check-wrap{position:absolute;white-space:nowrap;background-color:#f7f8fe;border-radius:4px;padding:24px;left:50%;line-height:1em;transform:translateX(-50%)}.spinner{border:3px solid #3f58e0;border-top-color:transparent;border-right-color:transparent;border-radius:50%;animation:rotate .6s linear infinite;aspect-ratio:1}.small{border-width:2px;display:inline-block;vertical-align:bottom;width:1em;margin-right:16px}.big{margin:32px;height:40px;width:40px}#brand{white-space:nowrap;font-weight:700;margin:40px;margin-top:10px}svg{vertical-align:bottom;display:inline-block;fill:#3F58E0;height:1.3em;margin-right:-2px}a{color:#3f58e0;text-decoration:none}@keyframes rotate{0%{transform:rotate(0)}100%{transform:rotate(360deg)}}</style><div id=main><h1 id=head></h1><div id=body><div id=text></div><div id=active></div></div></div><div id=brand></div>
🗺️ XML Sitemap Ready to Inspect

Visual Sitemap Explorer

Explore, search, filter, and inspect all URLs declared in this website's XML sitemap structure.

📍
Discovered Sitemaps:
🗺️
Total URLs
0
Single level structure
🕒
Last Modified
Not Specified
0% of URLs include lastmod dates
Priority Coverage
0%
0 URLs specify priority score
🔄
Change Frequency
None
0 URLs specify update cadence
🖼️
Rich Media & Locales
0 Images
0 alternate language links
Showing 0 of 0 URLs
# Page URL & Canonical Path Directory Last Modified Frequency Priority Media Actions
Rows per page:

Want a Full 144-Point Website Audit?

Check your website for SEO, page speed, technical health, conversion signals, security headers, and AI readiness in under 60 seconds.

Fix Your Website Help Guide Book Cover
Report Companion Guide

This Report Found the Problems. This Guide Helps You Fix Them.

A website analysis is only useful if you know what to do next.

The Fix Your Website Help Guide explains all 144 checks used by SiteAnalyzerFree.com in straightforward language. You’ll learn what each result means, why it matters, and how to correct it.

🔒 Instant Digital Access & Companion Guide - Just $17