Crawler

Publishing the Ti-bag DNS record verifies ownership and authorises scheduled recrawls of that domain. Revoking authorisation in the console stops further crawls.

Only verified domains are crawled. Unverified and third-party sites are not crawled.

Verified domains are recrawled periodically to keep commerce data current. On-demand refreshes are subject to a cooldown.

Developers can add the open-source Ti-bag tag to a site to signal the crawler and begin onboarding. Crawling starts once the domain owner completes DNS verification.

The crawler identifies itself with a dedicated user-agent, respects robots.txt and observes per-site rate limits. User-agent details and configuration guidance are provided in the console.

Only public pages are read — nothing behind a login. Business contact information is retained; personal data about staff or customers is discarded.

Block the crawler in robots.txt or revoke authorisation in the console. If you own a site you did not connect, contact inquiries@ti-bag.com to request removal and blocking.