Bots

fivebar keeps bots out of your figures, and the Bots page shows what they were.

Crawlers, link previews, uptime monitors and scripts load pages too, and counted as people they would swell your figures. fivebar tells them apart as they arrive, with nothing to set up, and the Bots page is where to look when a visit of your own goes missing.

Only bots that run your pages’ JavaScript, or send the script’s counts themselves, reach fivebar at all. Most crawlers don’t, Googlebot’s everyday crawling and the AI crawlers among them, so they never show: see Bots that never show.

The Bots page

Open it from the Overview, Speed, Errors and Bots switch under the site’s name. It keeps the dashboard’s dates and filters, and shows:

• the pageviews taken for bots, beside the period before, and their share of every pageview in the range, bots’ and people’s
• a chart of them for each hour, day, week or month
• why each was taken for a bot, and which known bot it said it was
• the pages they asked for, where they said they came from, which a bot can make up, their networks and countries, and the browsers and operating systems their user agents name

Click any row to narrow the page to it. A reason or a bot narrows the Bots page alone, so it stays behind when you switch pages. Narrowed to one, the share is still of every pageview. Under a filter by a page, a source, a country, a network, a browser or an operating system, the page counts the bots that match. Filtered by anything else, such as a referrer, a city, a device or a goal, it has nothing to show.

A bot is never a visitor or a visit, so the page counts pageviews alone. A bot’s pageview answered with an error, such as a 404, isn’t counted at all, here or on the Errors page.

Bots are left out of everything else: the dashboard’s figures and panes, who is on the site now, email reports and the stats API. An app connected over MCP reads this page’s figures with get_site_bots, and no other tool’s figures count a bot.

What counts as a bot

fivebar takes a visit for a bot’s the same way on every site, with no setting to change it, and gives it one of these reasons:

• Crawler: a user agent, the name a browser sends with every request, with a word ending in bot, such as Googlebot, bingbot, GPTBot or ClaudeBot, or with crawl, spider, scrap or fetcher in it. A few more are caught by name, such as Yahoo’s Slurp, Yandex’s robots, Perplexity and Anthropic’s Claude-User, and so are Google’s other agents: GoogleOther, APIs-Google, Mediapartners-Google and Google-InspectionTool, which Search Console’s live test runs.
• Automated browser: a browser a script drives, which says so: its user agent says headless, as HeadlessChrome’s does, or it tells the page, as a browser run by Selenium, Playwright, Puppeteer or Cypress does unless told not to. A test you run against the live site is one, with its window open or not.
• Link preview: what an app fetches to show a link shared in it: Facebook’s, WhatsApp’s, Telegram’s, Discord’s and Slack’s, and anything else that says preview. The people who open the link are counted, in the app’s own browser too, and so is a site used as a Telegram Mini App or a Discord Activity.
• Monitor or speed test: anything that says monitor or uptime, Pingdom, and Lighthouse, which PageSpeed Insights runs.
• Program: curl or Wget, or code using Python’s requests, axios, OkHttp, Java, Go or Perl’s libwww, sending the script’s counts itself. One that only fetches a page runs none of its JavaScript, so it sends nothing and shows nowhere.
• No user agent: a request without one, which no browser sends.
• Hosting network: any other visit from a network for servers rather than people: see Hosting networks.

Known bots

The Bot pane names the bots fivebar knows, from a list of its own. The crawlers: Google, Bing, Apple, Yandex, Baidu, DuckDuckGo, Petal Search, Yahoo, OpenAI, Anthropic, Perplexity, Meta, Common Crawl, Ahrefs, Semrush, Majestic and Moz. The link previews: Facebook, WhatsApp, Slack, Telegram and Discord. The monitors: Lighthouse and Pingdom. The programs: curl, Wget, Python requests, axios, OkHttp, Java, Go and libwww-perl.

A bot is named by what its user agent says, which anyone can copy, so a name is a claim, not proof. Any other bot has no name, and isn’t in the Bot pane.

Hosting networks

A visit from a network classed as hosting is a bot’s whatever its user agent says, since most bots run on servers and few people browse from one. That’s 8,959 networks, from ipverse’s as-metadata, the list a site’s settings name networks from. Among them are Amazon Web Services (AS16509), Google Cloud (AS396982), Microsoft Azure (AS8075), DigitalOcean (AS14061), Hetzner (AS24940) and OVHcloud (AS16276). The list classes each network from several signals of its own, and says it isn’t always right.

Some networks classed as hosting carry people too, so a visit from one is taken for a bot’s by its user agent alone:

• iCloud Private Relay, which leaves through Cloudflare (AS13335), Akamai (AS36183) and Fastly (AS54113), and Cloudflare’s WARP
• Google’s own network (AS15169) and Apple’s (AS714), which carry their offices
• the gateways companies send their staff’s browsing through, such as Zscaler’s (AS53813) and Cisco Umbrella’s (AS36692)
• universities’ and schools’ networks, such as Jisc’s Janet in the UK (AS786)
• internet providers and mobile carriers classed as hosting, such as Charter’s in the US (AS22516)

Most VPNs and proxy services run on hosting networks, so a person browsing through one is taken for a bot, as is a check you make from a cloud server. To see which network you’re on, look under Excluded traffic in a site’s settings.

Behind a proxy

Through a proxy that passes on who the visitor is but not their network, fivebar can’t tell a hosting network from any other. Bots are then told apart by their user agent alone, so a visit through a VPN or from a cloud server is counted as a person’s. A proxy that passes the network on gets the same checks as a visit straight from a browser.

Bots that never show

A bot reaches fivebar only by running the script, as a browser does, or by sending its counts as the script would. Most crawlers, the AI ones among them, read a page’s HTML without running its scripts, so however often they come, they never show. An empty Bots page means no bot ran the script, not that none came: your server’s own logs see every request.

Googlebot reads most pages that way too. It shows only when Google renders a page, running its scripts as a browser would, and the renderer sends the script’s count. So the Bots page is no measure of how often Google, or any search engine or AI company, reads your site.

A bot that fits none of the reasons above is counted as a person, and so is an automated browser that hides what it is, as some scrapers do. Leave one out by its address, or by its network if the network is its own: see Excluding traffic. Traffic a site excludes is left out before anything else, so it’s never on the Bots page either.

What is kept

For each pageview taken for a bot, counted by the hour: why it was taken for one, which known bot it said it was, the page, where it said it came from, its country and network, and the browser and operating system its user agent names. Nothing else is kept: not its speed, time on the page, scroll depth, events or clicks, nor its user agent, its IP address or a visitor code made from them, so one bot is never told from another. See Privacy.