BustanBot
The verification crawler for Bustan.
What it does
Bustan helps UAE credit-card holders understand the benefits their cards already include, and find public dining and lifestyle offers those cards unlock. To keep that accurate, BustanBot periodically fetches public pages: bank marketing pages and linked terms & conditions PDFs describing card benefits, and public offer and promotion pages — restaurant and venue websites, public Instagram posts, and bank offer portals. Card-benefit changes are reviewed by a human before publication; offers are checked automatically and spot-checked.
How it identifies itself
BustanBot/1.0 (+https://getbustan.com/bot)
How it behaves
BustanBot reads public pages only — never behind logins or paywalls. It revisits card and benefit pages about once a month and offer and promotion pages every few days. Our own fetcher sends at most one request every ten seconds per site, identified with the user agent above. Some offer pages are loaded through third-party rendering services, which use their own infrastructure and request patterns.
How to block it
Add this to your site’s robots.txt to stop BustanBot visiting
your card and benefit pages:
User-agent: BustanBot Disallow: /
For public offer and promotion pages — which BustanBot reads regardless of robots.txt — email the address below and we will pause your site within two working days.
Contact
Questions, rate concerns, or removal requests: support@getbustan.com. We respond quickly and honor opt-outs.