GetTheJobBot

GetTheJobBot is the crawler of bekommedenjob, a job-application tool. It reads public job ads once a night so users can find jobs that fit them — and always links to the original ad.

What it fetches

Only machine-readable, public job feeds: the interfaces of public employment services (Bundesagentur für Arbeit, job-room.ch, EURES) and the job feeds employers publish through their applicant-tracking system (such as Personio, Greenhouse, Lever, Recruitee, Ashby, softgarden, d.vinci). No pages behind a login, no application forms.

How it behaves

  • It reads robots.txt and follows the rules for “GetTheJobBot” and for “*”.
  • At most one request at a time per website, with a pause in between.
  • A fixed daily cap on requests per website.
  • When a website answers 403 or 429, it stops for that night and does not try harder.
  • It asks the employment services only for what users search for, and does not re-fetch unchanged ads.
  • Names, email addresses and phone numbers of contact persons are not stored.

Opting out

These lines in your website's robots.txt keep GetTheJobBot out entirely. It re-reads robots.txt at least every six hours.

User-agent: GetTheJobBot
Disallow: /