Book a Call
Support & Security

Your Website Traffic Is More Bot Than Human

And your architecture has to account for that.

August 16, 2026 · Ecropolis Team

Editorial illustration showing automated, helpful, and human traffic passing through layered website defenses including edge cache, verified bots, blocked probes, rate limiting, managed challenge, and protected origin capacity.

For years, website security advice treated bots as an occasional nuisance: a scraper here, a fake login attempt there, a burst of junk traffic during an attack. That view is now outdated.

Here is what it looks like on our own site. Over eight days, ecropolis.com took 167,259 requests. About 75% of them did not come from a browser. Roughly one in six was a request for something that does not exist, and nearly three quarters of those were probes for credentials, config files and admin panels.

A caveat on that number, because it matters: 75% is a user-agent measurement, not a bot-management verdict. Cloudflare’s bot scoring is not available on this plan, so the split counts requests whose user agent is not a browser. It understates the real figure if anything — a large share of the traffic that does claim to be Chrome turns out, on inspection, to be scanners walking a list of exploit paths.

The industry numbers say the same thing at scale. According to the 2026 Thales Bad Bot Report, bots made up 53% of internet traffic in 2025, while human traffic fell to 47%. Thales also reports that 40% of total internet traffic came from bad bots. The prior year’s Imperva 2025 Bad Bot Report had already found automated traffic at 51%, meaning the shift was not a one-month anomaly. It is becoming the operating environment for public websites.

That does not mean every automated request is hostile. Search crawlers help people find your site. Uptime monitors confirm that your site is reachable. Feed readers, link-preview services, accessibility tools, SEO crawlers, and some user-directed AI agents can all have legitimate uses. Cloudflare’s documentation for Verified Bots makes this distinction clearly: some bots are transparent about who they are, behave within reasonable limits, and provide useful functions.

The problem is that useful automation and abusive automation now share the same front door.

Bad bots scrape content, test stolen passwords, probe old software paths, hammer expensive search pages, distort analytics, attack APIs, and consume server resources that should be reserved for real users. Akamai’s 2025 research on AI bot activity found a 300% year-over-year increase in AI bot traffic, with bots generating billions of requests and skewing business metrics. OWASP’s automated-threat taxonomy also treats credential stuffing, denial of service, scraping, vulnerability scanning, and similar patterns as distinct application threats, not just generic “traffic.”

For businesses, the practical issue is not only security. It is cost.

Every request has a price. Sometimes that price is tiny: an edge cache returns a page without touching the origin server. Sometimes it is expensive: a request wakes PHP, queries a database, runs a search, loads a CMS theme, or triggers a login workflow. When a bot sends thousands of those requests, it can create real infrastructure cost, slower pages, degraded user experience, and operational noise.

Why WordPress sites take the worst of it

Attackers do not write bespoke tooling for your site. They write it once for whatever runs everywhere, and then point it at everything.

That makes WordPress the highest-value target on the web, and not because it is badly built. It is the opposite problem: it powers a very large share of the internet, and its real surface is not the core software but the ecosystem around it — tens of thousands of plugins and themes, written to wildly varying standards, installed by people who are not developers and updated on no particular schedule. Core WordPress is maintained by professionals. The plugin that adds a contact form to one page may not be maintained by anyone at all.

The scale of that surface is easy to underestimate. When we built vulnerability monitoring into ControlPlane, the WordPress intelligence feed alone carried more than 38,000 published vulnerability records — of which only 117 were relevant to plugins actually installed in our production fleet. The gap between those two numbers is the ecosystem, and scanners work through it indiscriminately.

You can watch it happen on a site that has nothing to do with WordPress. Ecropolis runs on Astro — static files, no PHP, no database, no admin panel. It still fields a constant stream of WordPress probes: 4,495 requests across 551 distinct WordPress paths in eight days, naming 39 different plugins and themes. /wp-login.php alone was requested 219 times. Every one of them was answered by a 404, because there is nothing there to find.

That is the point worth taking away. The probing is indiscriminate, so running WordPress is not what attracts it — but running WordPress is what makes it land. On a static site those 4,495 requests cost nothing. On a WordPress site, a meaningful share of them wake PHP, hit the database, and consume the same capacity your actual visitors need.

Which is why the hardest version of this problem is a busy WordPress site. Low traffic and you can absorb the waste; static and the requests never reach anything expensive. But a content-heavy WordPress platform with real human traffic has to serve genuine visitors well while absorbing automated load against the most heavily targeted software on the internet, on infrastructure sized for the humans. That combination is where tiered defense stops being good practice and starts being the thing that keeps the site up.

That is why modern website defense should not be a single wall. It should be a tiered system.

The wrong approach: block everything suspicious

A blunt “block bots” policy sounds appealing, but it usually fails in practice.

It can block search engines and legitimate monitoring. It can challenge real customers at the wrong time. It can hide the difference between harmless automation and dangerous abuse. It can also move load from one part of the system to another: for example, replacing cheap cached responses with expensive challenge pages or forcing too many requests back to origin.

The better approach is proportional defense: make each request cost as little as possible, as early as possible, using the least disruptive decision that is still correct.

That principle comes from our own architecture work on tiered traffic defense for high-traffic WordPress environments. The details below are not industry-wide statistics; they are practices drawn from running these systems.

Layer 1: Let cache absorb what should not reach the server

The cheapest request is the one the application never has to answer.

For public, anonymous content, a strong caching strategy can absorb a large share of normal human traffic and bot traffic alike. Edge caching, page caching, object caching, and targeted fragment caching all reduce the amount of work the origin server has to do.

This matters because bot defense is not only about identifying attackers. It is also about reducing the cost of routine, repetitive access. If a crawler requests a public article that is safely cacheable, the ideal outcome may be to serve it from the edge without waking the CMS at all.

We treat cache as the first defense layer, not merely a performance enhancement. Purge behavior is also handled carefully, because excessive cache purging can create its own traffic surge by forcing previously cheap requests back to origin.

Layer 2: Verify trust instead of trusting user-agent strings

Many bots claim to be Googlebot. That does not make them Googlebot.

A useful bot strategy should verify identity through network-level signals, not just self-reported headers. Cloudflare’s bot-management documentation describes verified bot fields that identify known good automated services through validation methods such as reverse DNS, ASN checks, public lists, and other verification methods.

The practical rule is simple: allow or exempt known-good bots because they are verified, not because they use a familiar name.

This distinction is especially important on expensive routes. A real search crawler may need access to public content. A spoofed crawler should not get a free pass to hammer search filters, pagination, login endpoints, or CMS internals.

Layer 3: Use cheap verdicts for obvious junk

Not every bad request deserves an expensive response.

If a request is looking for .env, .git, wp-config.php, Drupal paths on a non-Drupal site, direct CMS internals, or old exploit paths, there is no useful business interaction to preserve. These requests can often be blocked outright or answered with a cheap status such as 410 Gone when the route is permanently dead.

The goal is to avoid spending application resources on requests that can never become legitimate customer activity.

This is also where old-platform cleanup matters. Legacy URLs, stale CMS paths, and abandoned files can become magnets for scanners. A good defense posture includes knowing which routes are real, which are retired, and which should never reach the application.

Layer 4: Challenge suspicious behavior only where it is worth it

Challenges should be targeted. They should not be the default response to every unknown visitor.

Some routes are naturally more expensive than others: faceted search, filtered directories, deep pagination, login pages, XML-RPC endpoints, admin paths, and dynamic CMS queries. These are the surfaces where abusive automation can create outsized load.

A tiered defense can challenge anonymous, non-verified traffic on those expensive routes while continuing to allow normal public browsing, verified crawlers, authenticated editors, and known operational monitors. This keeps friction close to the risk instead of spreading it across the whole site.

Layer 5: Rate-limit abuse patterns, not normal users

Rate limiting is most useful when it matches the behavior being controlled.

A human may browse several pages quickly. A bot probing hundreds of 404s, repeatedly posting to login endpoints, or firing thousands of requests from one IP in a short window is a different pattern. Those cases can justify temporary blocks or managed challenges.

We scope rate limits by route type and abuse pattern: expensive public content, query-heavy searches, 404 sniffing, credential probing, and single-IP firehose behavior. The important point is not the exact threshold; every site’s thresholds should be tuned to its traffic. The important point is that rate limits should protect the most expensive and most abused surfaces without punishing ordinary visitors.

Layer 6: Escalate only when the origin is actually under strain

The most disruptive controls should be tied to measured server health.

If the origin pool is healthy, a website does not need to challenge broad public traffic just because the internet is noisy. If the origin pool is degraded, deadlocked, or running out of capacity, stronger protection may be justified for a short period.

This is where operational telemetry matters. A practical system looks at signals such as response latency, failed health checks, 5xx errors, memory pressure, and repeated probe failures. It should also use hysteresis so it does not flap between states during normal variation.

Broad protection mode is not a permanent setting. It is a temporary escalation used when multiple signals show real strain, and it should auto-expire or be manually reviewed. The system should also guard against self-inflicted load, such as cache purge storms or internal verification floods, so it does not mistake its own maintenance activity for an attack.

Most of this is available on a free plan

There is a fair objection to everything above: it sounds like enterprise tooling. Some of it is. Most of it is not.

We build on Cloudflare across the estate, from national platforms down to five-page brochure sites, and the reason is that the free tier is unusually generous about exactly the layers that matter most here. Caching and a global CDN, unmetered DDoS protection, TLS, a managed WAF ruleset, cache and redirect rules, and basic bot filtering are all available before you pay anything. A small business site can put four of the six layers in place for the cost of pointing its DNS at Cloudflare.

What you actually pay for is the sophisticated end: bot scoring, verified-bot identification as a field you can write rules against, advanced rate limiting, and higher rule counts.

That distinction is worth being precise about, because it changes the advice. Layer 2 above — allow bots because they are verified, not because they claim a familiar name — needs a plan tier that exposes bot management fields. We ran into this measuring our own traffic for this article: ecropolis.com is on a Pro plan, and Cloudflare’s bot score simply is not queryable there, which is why the 75% figure is a user-agent estimate rather than a verdict. On a free or Pro plan you can still verify Googlebot properly, it just takes a reverse DNS check rather than a one-line rule.

So the honest version of the recommendation is tiered too:

  • Any site, free plan. Turn caching on and mean it. Add the managed ruleset. Block the obvious junk paths. Retire dead URLs instead of letting them 404 forever.
  • Growing site, paid plan. Scope rate limits to your expensive routes. Challenge non-verified traffic on search, login and pagination rather than everywhere.
  • Platform scale. Verified-bot gating as a first-class rule, health-based escalation, and the telemetry to know which state you are in.

Exact plan limits move around, so check Cloudflare’s current pricing rather than this page. The shape of the split — cheap layers free, precise identification paid — has been stable for years.

The business case: less waste, better availability, lower cost

Bot defense is often sold as a security feature. It is also an infrastructure discipline.

Reducing unwanted automated traffic can lower origin load, preserve PHP and database capacity, improve page speed during traffic spikes, reduce hosting pressure, and make analytics cleaner. It can also delay or avoid expensive infrastructure upgrades that would otherwise be needed only to serve waste.

That does not mean every business needs enterprise-grade bot management on day one. It does mean every public website should answer a few practical questions:

  • Which traffic should be cacheable?
  • Which bots are useful enough to allow?
  • Which routes are expensive enough to protect?
  • Which old paths should be blocked or retired?
  • Which behaviors should trigger rate limits?
  • Which server-health signals justify temporary escalation?
  • How do we avoid blocking real users during our own maintenance events?

The modern internet is no longer mostly human traffic with a few automated exceptions. Automation is now part of the baseline. Some of it creates value. Some of it steals value. Some of it simply wastes capacity.

Good architecture does not panic about that reality. It sorts traffic by trust, intent, cost, and current server health. It lets useful automation do useful work. It keeps obvious abuse away from the application. And it protects the budget by making sure the most expensive systems are reserved for the people and workflows that actually matter.

Sources

The tiered model, cache-first posture, verified-bot gating, scoped rate limits and health-based escalation described above come from our own operational practice rather than published research. They are architecture-specific and should not be read as industry data.

Keep reading