BalkanaberBot

BalkanaberBot is the crawler of Balkan’aber, a Turkish-language news site that follows the local press of nine Balkan countries. It identifies itself with this user agent:

BalkanaberBot/0.1 (+https://balkanaber.com/bot; haber derleme)

It connects from a single server, IP address 178.105.127.149.

What it reads and why

It reads RSS feeds and, where a site has none, news sitemaps, the home page and article pages. Each Balkan’aber story is a short Turkish synthesis, usually written from two or three sources. It is never a translation or copy of one article. Every source is named in the story, with its language and a link to the original article.

• It does not download or republish your photos.
• It does not log in, and it does not read paywalled or subscriber-only content.
• It does not try to get around bot protection. A 403 or a challenge page is treated as a “no”.

How often

• A few visits a day, not continuous crawling.
• At most 15 new article pages per source per visit, opened one at a time and at least 1.5 seconds apart.
• Before writing a story it may open the few articles that story is based on once more, again one at a time.
• When a site answers 429 (too many requests) while the bot is reading new articles, it stops for that visit. Sites that need it are read more slowly.
• After more than five failed visits in a row the site is paused and tried again only once a day.

How to limit or block it

BalkanaberBot obeys robots.txt and checks it before every feed, sitemap and article it requests. To block it completely:

User-agent: BalkanaberBot
Disallow: /

You can also write to editor@balkanaber.com. We will remove your site from our source list, lower the visit rate, or switch to a feed you prefer. We reply to every request. Our sources and how we use them are listed on our source policy page (in Turkish).