The Bot Traffic Hiding in Your Analytics Reports

Why is Direct Traffic Spiking While Engagement Drops?

Over the past few months, I’ve watched the same pattern show up across several of the sites we manage. Direct traffic spikes. Engagement metrics drop. Some drop in engagement is normal when a lot of new visitors arrive, but these numbers were falling faster than that explains.

We were seeing average visits of around four seconds on pages that take minutes to read. Nobody lands on a detailed product or content page and leaves in four seconds at that volume. Something was hitting these sites, and it was showing up in our reports as people.

Most of it, it turns out, is bot traffic. A large share comes from AWS and AI companies crawling the web to train their models, and the biggest source we’ve found is ByteDance, the company behind TikTok.

How to Identify Bot Traffic in GA4

The clearest example came this July, from one of our clients. I started with the AI assistant built into GA4 and asked a simple question, “Is there direct traffic with very short visit times, and where is it landing?”

It pointed me to a handful of pages. When I asked for the median and average time spent on those pages from direct traffic, it came back at about two seconds, and there was a lot of it. If you picture the distribution of visit lengths, the peak, where most visits cluster, sat way down near zero.

I refined the prompt, then went into an Exploration report myself to check. Sure enough, a large volume of Linux and Windows machines were hitting one specific page for about two seconds each. Digging further, most of those visits weren’t coming from the United States. They traced back to major AWS server locations, often masked, and landed overwhelmingly on the contact page.

To measure it precisely, I added a flag to the client’s analytics that identifies automated browsers. Browsers driven by automation tools like Selenium report a property called navigator.webdriver as true. I capture that value in a Google Tag Manager variable, pass it to GA4 as an event parameter on the configuration tag, and register it as a custom dimension. From there, any Exploration report can split sessions into automated and human.

function() {
  // Returns true if automated, false if not, or undefined if the browser doesn't support it
  return navigator.webdriver;
}

Within two days of turning the flag on, the numbers were hard to argue with. Around 69% of the client’s direct traffic was automated bot traffic, and those sessions averaged 2.1 seconds. The real visitors in that same channel stayed for 4 minutes and 8 seconds.

Why Does ByteDance’s Bytespider Crawler Show Up in GA4?

ByteDance does a lot more than TikTok. It runs large server operations in Hong Kong and mainland China, and its crawler, Bytespider, hits websites across the internet to gather training data for its AI systems. It also doesn’t respect robots.txt, the file sites use to tell crawlers what to leave alone.

Plenty of other AI crawlers visit your site too. You’ll see OpenAI’s GPTBot, Anthropic’s crawler, and Googlebot in your server logs. What keeps most of them out of your analytics is JavaScript. GA4 only records a visit when its tag runs, and the tag is JavaScript. Most crawlers don’t execute JavaScript by default. Google’s does, but GA4 knows how to recognize and exclude Googlebot.

My read, and I want to be clear it’s my read from what we’re seeing, is that ByteDance’s crawler does render JavaScript. Its user agent presents itself as a Chrome browser on Linux, which is consistent with a headless browser. When the page renders, the GA4 tag fires and records a session as if someone had arrived.

The broader driver is AI itself. Companies are racing to train bigger models, and that means crawling more of the web more often. AI also lowers the bar for bad actors. Someone who has never written a line of code can ask an unrestricted LLM to set up a scraper, and that’s a real change in who can generate this kind of traffic. I also think the sheer volume of AI crawler traffic has desensitized site owners, and bad actors are taking advantage of it. When every report already looks noisy, one more spike doesn’t get investigated.

If you’re wondering whether your site is too niche to attract this, it isn’t. We’ve seen it on hardware manufacturers and B2B retailers. It’s even hitting my personal website, which gets three or four visits a day from search.

How Does Bot Traffic Skew Analytics and Business Decisions?

When bots inflate your sessions, every per-session metric goes down with them. For an E-commerce client, revenue per session and conversions per session start to slide. For a content-focused site, page views climb while time on page collapses. Either way, the report tells a story about your marketing that isn’t true.

If 69% of your direct traffic is bots and you’re making budget, content, or channel decisions off that data, ask yourself whether those decisions are really backed by data.

Most of this traffic lands in direct, because bots typically arrive without a referrer. Not all of it, though. Some bots come through a search result or another website and get counted as organic or referral. Direct is the place to start looking, but it shouldn’t be the only place.

Should You Filter or Block Bot Traffic?

Once you know the traffic is there, you have two options. You can filter it out of your reporting, or you can block it from reaching your site at all.

Filtering keeps the bots out of your numbers without touching your site. For one of our clients, we used the automated-browser flag to clean bot sessions out of the data and excluded direct visits to the support section from the reporting dashboard. Because that traffic ran through AWS, filtering was the safer choice. You can also exclude very short direct sessions; a cutoff around three seconds removes most of what we saw.

Blocking stops the traffic before it reaches your site, which is what we did for a US E-commerce client; we blocked at the ASN level. Every device online has an IP address, and IP addresses belong to networks. Each network has an Autonomous System Number, or ASN. I think of it as the address of the house where all the little IP addresses live. My home internet runs through Xfinity, for example, and Xfinity has its own ASN.

We block the house instead of individual IPs because IPs rotate constantly, and a large operator can cycle through an enormous number of them. A determined bad actor could rotate networks too. A company like ByteDance likely operates from a small number of ASNs, and blocking those cuts off most of what it sends.

The easiest way to find the right ASNs is to ask your AI assistant. Every site has a different analytics and server setup, so describe yours to ChatGPT or Claude and ask, “I think we’re getting hammered by bots. Can you help me find where they’re coming from and how to block them, preferably by ASN instead of IP address?”

Which option is the right fit depends on your business. Many of our clients will never sell into China, so blocking ByteDance is low-risk for them. Our E-commerce client sells only in the US and doesn’t advertise on TikTok, so blocking was an easy call. But if TikTok is a meaningful ad channel for you, have that conversation first. A reporting filter may be the safer choice.

When is Blocking an ASN a Bad Idea?

An ASN is a house, and a lot of people can live in one house. For the big offenders like ByteDance, and another network we found in Hong Kong, nearly everything coming from the house is automated. Shared infrastructure is a different story.

Take Amazon Web Services. We run our own work on AWS, and so do thousands of legitimate businesses. Anyone can rent server space there and build an application that reads content from other websites, including something malicious. If you blocked the AWS ASN to stop that one bad actor, you’d block everything routed through Amazon, including traffic you want. I wouldn’t recommend it. A middle option is to challenge traffic from a suspicious ASN instead of blocking it outright. Real visitors pass the check, and automated ones get stopped. That’s what I recommended for our client experiencing an influx of AWS bot traffic.

For scattered, generalized bad activity, accept some of it as a cost of running a website, and let CAPTCHAs and bot management tools handle it. Save ASN blocks for the networks that hit you over and over, in volume, and do nothing but distort your analytics or strain your servers.

How to Remove Bot Traffic From Your Analytics

To remove bot traffic from your analytics, you have to start by measuring. Pull your direct traffic in GA4, look at how long those sessions last, and see what share of them end within a few seconds. That number will tell you whether you have a problem worth solving.

1. Ask your AI assistant to help you diagnose it. An LLM is a good partner for finding where the traffic comes from and weighing the pros and cons of filtering against blocking.
2. Add an automated-browser flag in GA4 using navigator.webdriver, then confirm what you find in Explorations before acting on it. Look at the flag alongside operating system, country, landing page, and session duration.
3. Pick the response that fits your business. Filter in reporting if the source overlaps with markets or channels you care about. Block at the ASN level if it doesn’t.
4. Add rate limiting where it helps. CAPTCHAs and Cloudflare’s bot tools can slow down bad actors without a permanent block.

One caution above everything else: if you’re not confident in a blocking decision, don’t make it. Block the wrong ASN and you can cost yourself real traffic and revenue. If I accidentally blocked Google from crawling my site, I’d disappear from Google search. Be certain the block is one you want before you put it in place.

What Changes Once Your Analytics Data is Clean?

Once the filter is in place, go back to the reports that shaped your last few quarters of decisions. Budget shifts, content priorities, and channel bets made on bot-inflated data deserve a second look. Note the date your filter went live, too, so a year-over-year comparison doesn’t read as a traffic drop.

It’s also important to tell your leadership team before they notice the change. Sessions will fall when the automated traffic comes out, and engagement metrics will rise. Explaining that ahead of time keeps the conversation on what your real visitors are doing, like the 4 minutes and 8 seconds our one client’s human visitors spent on site.

You should also treat this as a recurring check. AI companies are still scaling up their crawling, and new sources will keep appearing. Once a quarter, pull your direct traffic, check what share the automated-browser flag catches, and look for new networks or countries showing up in volume.

If you’ve been looking at a direct traffic spike and a falling engagement rate you couldn’t explain, start with the automated-browser flag. It’s a small change in Google Tag Manager, and in our case two days of data was enough to see the pattern.