PostHog's Bot Filter Worked Fine. Our QA Check Was the Bot.

July 2026

We almost shipped a fix for a site-wide analytics outage. The outage didn't exist. What we'd actually found was PostHog's bot filter doing exactly what it's designed to do — to a headless browser we were using to check our own instrumentation.

This is the story of how we got there, what it cost us before we caught it, and the test we built so it can't happen twice.

Short answer, if you're here from a search: a headless QA browser tripped posthog-js's built-in bot filter — the same filter that keeps Googlebot and other crawlers out of your analytics — which drops events client-side before any network request is made. A zero-capture result from a headless test tells you the filter is working, not that production is broken. The mechanism is below.

The Alarm: Numbers That Didn't Add Up

Our dashboard threw three signals at once. Homepage bounce rate jumped to roughly 95%. Clicks on our primary call-to-action dropped sharply. And one specific click event had gone completely silent for 19 days.

Normally any one of those is worth a look and nothing more. All three together, on the same day, looked like an instrumentation failure — the kind where the site works fine for visitors but stops telling us about it. The part that made it feel urgent: our server-side traffic logs showed no corresponding drop. People were still showing up. Our analytics said they weren't doing anything once they got there.

That combination — stable server traffic, collapsing client-side analytics — points at exactly one place: the JavaScript snippet that reports events from the browser. So that's where we looked.

The QA Check That Sealed It

The natural next step was to reproduce it directly: load the production site in a headless browser and watch the network tab for PostHog's capture requests.

We did exactly that, using headless Chromium. The results looked damning. posthog-js initialized without any errors — the config request succeeded, the feature-flag request succeeded, the project token was correct. But across every page we loaded, including after client-side navigation on our single-page routes, not one request ever reached PostHog's capture endpoint (/i/v0/e/). No console errors. No failed network requests. Do Not Track was off. Consent defaults were fine.

Everything about the SDK looked healthy except the one thing that actually matters: it never sent an event. That reads like "capture is broken site-wide," and we were about to write it up as a P0.

The Deminified Answer: Meet _is_bot()

Before shipping a fix, we wanted to understand the mechanism, not just the symptom. The filter itself isn't hidden — PostHog documents an opt_out_useragent_filter config option for exactly this behavior — but the precise check order and edge cases aren't spelled out in the docs, so we read the actual source: the _is_bot() instance method in posthog-js, and the isLikelyBot() helper it delegates to in the shared @posthog/core bot-detection module.

Here's that gate, in the real order, running entirely client-side before any network activity:

// posthog-js: PostHog.prototype._is_bot() calls this helper
function isLikelyBot(navigator, customBlockedUserAgents = []) {
  // 1. Check the UA string against a maintained list of 80+ known bots
  //    and crawlers — googlebot, headlesschrome, cypress, gptbot, and
  //    more — plus anything you've added via custom_blocked_useragents
  const ua = navigator.userAgent;
  if (ua && isBlockedUA(ua, customBlockedUserAgents)) return true;

  // 2. Chromium's Client Hints API exposes browser "brands." Headless
  //    Chromium reports "HeadlessChrome" here, checked against the SAME
  //    list above — even if the plain UA string has been spoofed
  try {
    const brands = navigator.userAgentData?.brands ?? [];
    if (brands.some((b) => isBlockedUA(b.brand, customBlockedUserAgents))) return true;
  } catch {
    // experimental API — ignore if unavailable
  }

  // 3. Last resort: the flag automation frameworks set directly
  return !!navigator.webdriver;
}

If isLikelyBot() returns true, _is_bot() drops the event before it ever builds a request — unless you've explicitly set opt_out_useragent_filter to true. There's no network call to fail, no error to catch, no log line to find. From the outside, it looks identical to a silent capture failure.

Our headless QA browser tripped this on more than one check: Playwright's default launch UA string already contains "HeadlessChrome", which fails check 1 on its own, and navigator.webdriver is true by definition in an automated session, which would have failed check 3 regardless. We hadn't found a bug in production capture. We'd found the bot filter working correctly on the one browser in the story that actually was a bot.

Proving It Both Ways

We didn't want to take the source-reading as gospel, so we tested the mechanism directly in both directions.

First, with opt_out_useragent_filter temporarily set to true in our headless test, capture fired immediately — same site, same browser, same everything else, just the filter disabled. Second, we left the filter active but made the browser look human across all three checks: a UA string with no blocklist match, userAgentData brands with no HeadlessChrome entry, and navigator.webdriver forced to false. With all three spoofed, isLikelyBot() returned false and events reached the real endpoint normally.

One more detail worth knowing if you're setting up your own headless checks: Playwright's default chromium.launch() context ships a UA string that already contains "HeadlessChrome" — enough on its own to fail the very first check, independent of navigator.webdriver. Playwright's built-in devices['Desktop Chrome'] profile supplies a normal-looking UA instead, which is why it's part of the fix below, not just the navigator.webdriver override.

Both directions of the test pointed at the same conclusion: production capture for real visitors had been healthy the entire time. Only our automated check was ever affected.

What Was Actually Happening to Our Traffic

Ruling out a capture failure didn't explain the original alarm, so we went back to the three original signals with that piece removed.

The bounce-rate spike and CTA-click drop turned out to be a composition artifact from a www → apex domain redirect. Our analytics had been reporting www and apex traffic as separate series. The www series had been declining on its own for a while — a small, real trend — and once the redirect consolidated all www traffic onto the apex domain, the combined numbers looked like a sudden collapse rather than what they were: two trends merging into one series at the same moment.

The timing made it look worse than it was. The day the numbers looked worst happened to land on a weekend, when baseline traffic on a developer-tools site normally dips anyway — so the "onset" of the apparent problem lined up with ordinary seasonality, not a new event.

We also owe an honest word about small numbers: several of these metrics were sitting on single-digit event counts per day. A 95% bounce rate sounds catastrophic. On a low-traffic page, it can mean 19 bounces out of 20 sessions — a swing that a couple of extra sessions in either direction would erase. Percentages built on small denominators move a lot more than the underlying reality does.

That leaves one piece we're not going to round off into a tidy ending: the click event that went silent for 19 days. Its typical rate is low — about 0.65 events per day — so a multi-week gap isn't as statistically alarming as it first sounds, but it's also not something the bot-filter explanation accounts for. We're calling it an open anomaly, not a solved one. If it recurs, it goes back on the board.

What We Shipped Instead of a Patch

The actual defect here wasn't in our analytics pipeline. It was in how we validated our analytics pipeline — a headless-browser check that can't observe the thing it's supposed to verify, because the tool being checked is specifically designed to ignore headless browsers.

Patching that one check would have fixed today's false alarm and left the next one waiting. So instead we built a capture-invariant test: an automated check that spoofs a human fingerprint on purpose, verifies an event actually reaches PostHog's capture format, and runs in CI on every deploy. Here's the whole thing, consolidated into one runnable file:

import { test, expect, devices } from "@playwright/test";

// A normal desktop Chrome UA — not Playwright's headless-launch default,
// which already contains "HeadlessChrome" and fails the bot check on its own
test.use({ ...devices["Desktop Chrome"] });

test.beforeEach(async ({ context }) => {
  // Present a human fingerprint so posthog-js's bot check returns false
  await context.addInitScript(() => {
    Object.defineProperty(Object.getPrototypeOf(navigator), "webdriver", {
      get: () => false,
      configurable: true,
    });

    try {
      Object.defineProperty(navigator, "userAgentData", {
        get: () => ({
          brands: [
            { brand: "Chromium", version: "125" },
            { brand: "Google Chrome", version: "125" },
            { brand: "Not/A)Brand", version: "24" },
          ],
          mobile: false,
          platform: "Windows",
        }),
        configurable: true,
      });
    } catch {
      // userAgentData isn't defined on every platform — the webdriver
      // spoof plus a clean UA is usually enough on its own
    }
  });
});

test("homepage fires a PostHog pageview", async ({ page }) => {
  const captured: string[] = [];

  // Match PostHog's ingestion path regardless of host — point this at
  // wherever your environment sends capture calls
  await page.route(/\/(i\/v0\/e|e|batch|capture)\/?(\?|$)/, async (route) => {
    captured.push(route.request().url());
    await route.fulfill({ status: 200, contentType: "application/json", body: '{"status":1}' });
  });

  // Run this against a local/dev URL, not the production domain — page.route
  // only intercepts requests your own browser session makes, so it can't
  // fulfill a live production page's real capture calls
  await page.goto("http://localhost:3000");

  // posthog-js batches events client-side; forcing a pagehide flushes the
  // batch instead of guessing at an arbitrary wait
  await expect
    .poll(async () => {
      if (captured.length === 0) {
        await page.evaluate(() => window.dispatchEvent(new Event("pagehide")));
      }
      return captured.length;
    }, { timeout: 15000 })
    .toBeGreaterThan(0);
});

One thing this test still can't see: server-side capture calls — the ones your backend or edge functions fire directly — run outside the browser entirely, so page.route never observes them. Verifying those needs a separate local capture sink your server points at during tests, not a browser-level intercept.

That test is now a deploy gate. If a real change to the site ever stops events from reaching PostHog, this fails loudly before it reaches production — instead of surfacing weeks later as a dashboard anomaly that looks exactly like the one that started this whole investigation.

What We're Taking Into the Next Investigation

A few things are worth carrying forward, and none of them are specific to PostHog.

Headless browsers can't observe client-side analytics capture by design, if the tool has a bot filter — a zero-capture result from a headless check proves the filter is working, not that anything is broken. If you're debugging a similar "silent tracking" alarm, check whether your reproduction method is itself a bot before you check anything else.

Read the mechanism before you declare an outage. We had a plausible, symptom-matching story — SDK initializes fine, zero capture requests, no errors — that was completely wrong. That's the kind of mistake that's expensive precisely because it's plausible: it burns real diagnostic time before anyone thinks to question the reproduction method itself, not just the system under test. Low-volume percentage metrics distort easily; a small denominator turns a normal fluctuation into what looks like a cliff. And the durable response to a misdiagnosis isn't a patch for today's alarm — it's a test that makes the same category of alarm self-diagnosing next time.

One more thing worth mentioning, since it's part of how we actually work: our engineers, working with our own agent team, run this kind of investigation on our own product, and we save what we find as we go — the isLikelyBot() check order, the redirect composition artifact, the still-open 19-day anomaly — into shared memory in MemNexus. The next session that touches analytics starts from those findings instead of re-running the same headless-browser test and drawing the same wrong conclusion. That's the whole point of building a memory layer for agent work: institutional knowledge that survives past a single conversation.

Try This on Your Own Stack

If you've ever seen a PostHog (or similar) integration look "silently broken" in an automated check — zero capture requests, no errors, SDK initializes fine — check your UA string, userAgentData brands, and navigator.webdriver before you file the incident. There's a decent chance you've found the bot filter, not a bug.

Related Reading

• Wiring Shared Memory Into a Multi-Agent Claude Code Team