AuditLamp // crawler

The AuditLampAudit crawler

Identifies itself · obeys robots.txt · rate-limits

AuditLampAudit is the crawler AuditLamp uses to read a website and audit it. It runs only when that site's owner asks us to, by submitting the URL. An audit reads a small, bounded set of public pages plus robots.txt, sitemap.xml and llms.txt, and scores what it finds against 73 documented SEO, AEO and GEO checks.

How it identifies itself

Every request AuditLamp makes carries a user-agent that names us and links back to this page, so you can always tell it apart in your logs:

User-agent (default)Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/126.0 Safari/537.36 AuditLampAudit/1.0 (+https://auditlamp.com/bot)

The browser portion is kept so we can read sites that block unfamiliar clients outright; the AuditLampAudit/1.0 token is the part that identifies us. Operators can switch AuditLamp to a fully transparent, non-browser user-agent if they prefer.

How to allow or block it

AuditLampAudit obeys robots.txt. To control it by name, target the AuditLampAudit token. For example, to block it entirely:

robots.txtUser-agent: AuditLampAudit Disallow: /

We honour Disallow rules for any page we discover while crawling, and we honour a Crawl-delay you set. The one page we always read is the exact URL the owner submits to us for an audit, because that is a direct, owner-initiated request for that page.

How politely it crawls

Questions or problems

If AuditLampAudit is causing trouble, or you want it to stop, email hello@brimmapp.com and we will sort it out quickly.