About IntelGrabBot

The HTML crawl uses one worker at at most one request per second, with a limit of 100 public URLs by default or up to 500 for an agreed review. Page crawling respects robots.txt. Optional browser rendering can make additional requests for page resources. A crawl that cannot read the rules because of a server error or timeout does not proceed.

The crawler checks public links and page markup. It does not sign in, submit forms or run a vulnerability scan. Manual form tests are agreed separately with the website owner.

Email hello@antrimstudios.com with your domain and a request to stop. We add the domain and its subdomains to our do-not-crawl list before further checks.

You can also disallow IntelGrabBot in your robots.txt file. Public access to a page does not establish permission to reuse its content.