BlueBandedBee is a respectful web crawler designed specifically for cache warming. Our bot visits your website's pages to ensure they're cached and ready for your real visitors, improving performance and user experience.
User Agent: BlueBandedBee/1.0 (+https://bluebandedbee.co/bot)
How It Works
- Respectful Crawling: We identify ourselves clearly and follow industry best practices
- Smart Rate Limiting: Default 3 requests/second with 10 concurrent connections
- Cache-Aware: We detect cache status (HIT/MISS) and warm pages intelligently
- Link Discovery: Optionally finds and warms linked pages within your domain
We Respect Your Site
- ✅ Honors robots.txt - We check your robots.txt for sitemaps
- ✅ Professional identification - Clear user agent with link to this page
- ✅ Browser-like behavior - Sends standard browser headers
- ✅ Handles rate limits - Detects 429/503 responses and backs off
- ✅ No personal data - We don't store page content, only performance metrics
Technical Details
Headers Sent
User-Agent: BlueBandedBee/1.0 (+https://bluebandedbee.co/bot)
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8
Accept-Language: en-US,en;q=0.9
Accept-Encoding: gzip, deflate, br
What We Collect
- Response times and cache status
- HTTP status codes
- Performance metrics (DNS, TCP, TLS, TTFB)
- Links found on pages (if enabled)
What We DON'T Collect
- Page content or personal data
- Form submissions
- Cookies or session data
- JavaScript execution results
Managing Our Bot
Allow Our Bot
No action needed - BlueBandedBee respects standard web practices by default.
Slow Down Our Bot
Add to your robots.txt:
User-agent: BlueBandedBee
Crawl-delay: 10
Block Our Bot
Add to your robots.txt:
User-agent: BlueBandedBee
Disallow: /
Contact
General enquiries: hello@bluebandedbee.co
Technical Support: support@bluebandedbee.co
Report Issues: https://github.com/Harvey-AU/blue-banded-bee/issues
About Blue Banded Bee
Blue Banded Bee is a cache warming service designed to improve website performance by pre-loading pages into CDN
caches. We're especially focused on helping Webflow sites achieve optimal performance.
Founded in Castlemaine, Victoria, Australia 🇦🇺
---
Last updated: July 2025