Fractle uses Mandelbot (our crawler) to find all the pages on your website, so we can create a search engine just for you.
If you have spotted Mandelbot in your website's logs, then either:
Mandelbot is Fractle's web crawler and it's used to index your website for inclusion in Fractle.
Make sure Mandelbot is allowed to crawl your website. If you have a robots.txt file, make sure you allow Mandelbot to crawl everything, and at least all your pages and images. You can do this by adding these two lines to your robots.txt file:
User-agent: Mandelbot Disallow:For more details, read about Mandelbot's support of the Robots Exclusion Protocol.
Fractle © 2024