Last updated: 2026-09-06
This page exists because our crawler names it in its User-Agent string. If you found
CatchspanBot in your server logs and want to know what it is, block it, or ask us to
remove something, everything you need is on this page.
Contact: bot@catchspan.com — a real mailbox, read by a person. Write in English or Japanese.
What it is
CatchspanBot/1.0 (+https://catchspan.com/bot; bot@catchspan.com)
CatchspanBot collects word-frequency statistics for catchspan.com,
a free listening test. We need to know how common each English word is in ordinary spoken
material, so we count words across public-domain transcripts and podcasts.
We are interested in counts of words, not in your content. We do not republish what we
fetch, and we do not build a search engine, a dataset for sale, or a training corpus for
sale.
How it behaves
| robots.txt | Respected. We do not work around it. A 403 is treated as “stay out”, not as an error to retry |
| Crawl-delay | Honored. If your robots.txt sets one, it overrides our own default |
| Rate | At most 1 request per second overall, and at most 1 request per second per host |
| Volume | Runs every 5 minutes, stops itself after 240 seconds, and fetches at most 400 items per run |
| Time of day | Sources we read in bulk are shifted to your local night where we know it |
| Media files | Audio and video bodies are not downloaded by default — we read transcripts |
| Login walls | Never. We do not create accounts, submit forms, or pass paywalls |
We also keep an internal do-not-crawl list of hosts we never contact at all, regardless of
what their robots.txt allows — sites whose licensing we judged unclear, and sites that ask
not to be mirrored.
How to block it
Add this to your robots.txt:
User-agent: CatchspanBot Disallow: /
We re-read robots.txt and will stop. If you would rather not wait, or you want us to
delete material we have already fetched, email bot@catchspan.com and say so — you do not
need to justify the request, and we do not ask for anything in return.
What we collect from
Only material that is in the public domain or carries an explicit licence permitting
this use. Every source is reviewed before it is added, and sources whose licensing we could
not confirm are excluded rather than assumed.
Reporting a problem
Email bot@catchspan.com for anything: excessive load, a source you believe we should not
be using, a request to be removed, or a mistake on this page.
For copyright takedown notices and privacy requests there is a dedicated form at
catchspan.com/takedown, which states a response time and
sends you a written receipt. You may also simply email this address instead — we will never
refuse a notice because you did not use the form.
日本語
CatchspanBot は catchspan.com（無料のリスニング測定）のための
語の頻度統計を集めるクローラです。パブリックドメイン、または明示的に許諾された資料だけを対象にし、
取得した内容そのものを再公開することはありません。
- robots.txt に従います。回避しません。
Crawl-delayの指定があればそちらを優先します - 全体で 毎秒 1 リクエストまで、ホストごとにも毎秒 1 リクエストまで
- 5 分ごとに起動し、240 秒で自主的に降ります（1 回あたり最大 400 件）
- 大量に読むソースは、先方の深夜に寄せます
- 音声・動画の本体は既定では取得しません（転写テキストを読みます）
- ログインの必要なページ・フォーム送信・ペイウォールの通過は行いません
止めたいときは robots.txt に上記の 2 行を書いてください。すぐ止めたい場合や、
すでに取得した資料の削除を求める場合は bot@catchspan.com にご連絡ください。
理由の説明は不要です。
著作権に基づく削除の申立て・プライバシーに関する請求には専用の窓口があります →
catchspan.com/takedown。応答期限を明示し、受付の控えを自動でお送りします。
このアドレスへのメールでも受け付けます。フォームを使わなかったことを理由にお断りすることはありません。