Welcome to WP Defense Lab — your weekly brief on the WordPress news that impacts your business.
This week: European publishers absorb four times the AI bot traffic of North American sites while their robots.txt files get ignored three times more often, proof that opt-out signals don't survive contact with scale.
In WP Radar: We track the same risk-shifting pattern across a WordPress LMS reshuffle, an AI Overview click-loss playbook, a startup paying creators for AI training footage, and Apple's new pay-per-use deal with publishers.
🎥 Did you know we have a YouTube channel focused on WordPress security?
We share step-by-step tutorials on how to secure your site and protect your content.
👉 Explore the full channel:
youtube/@wpdefenselab
European Publishers Are Paying the AI Scraping Tax Nobody Else Sees
Robots.txt was supposed to be a control. TollBit's data shows it functions more like a suggestion, and Europe is absorbing the difference.
European sites get scraped by AI bots four times as often as North American sites and get almost nothing back for it.
The report tracked 40 scraping vendors across 3,906 publishers, including 456 European publishers. The gap is not a rounding error. It is a structural cost falling disproportionately on European publishers, with rising demand for multilingual content offering one possible explanation.
The Math Nobody Budgeted For
European publishers get one human visitor from an AI app for every 179 bot scrapes. North American sites see roughly three times better odds.
In the first half of 2026, AI apps sent just 0.05% of external referrals to European sites, versus 0.16% in North America. The trend line is worse, not better: scrapes were nearly 20% higher in June than in January, and the scrape-to-referral ratio widened from 150:1 in Q1 to 227:1 in Q2.
Local news and sports sites, the publishers with the thinnest margins, saw the sharpest growth.
Robots.txt Was Never a Contract
Here is the part that should worry every site owner who thinks a disallow line is protection: European publishers' robots.txt instructions get ignored almost three times more often than North American publishers' do.
The file was built as a request, not an enforcement mechanism, and AI crawlers are treating it exactly that way.
INMA researcher Grzegorz Piechota's read on the underlying cause reinforces the point: European country domains made up just 4.75% of pages in Common Crawl back in 2009.
By 2026, that share hit almost 30%, tracking rising demand for non-English training data.
Data cited from Anthropic's economic index shows U.S. users generate only 21.6% of Claude traffic.
The demand for multilingual content is real. The publishers supplying it are not being compensated for supplying it.
Who Actually Bears This Cost
This is risk-shifting in its purest form.
AI labs get the training data. Publishers get the bandwidth bill, the server load, and a robots.txt file that functions more as a suggestion than a control.
Cloudflare's own data complicates the regional story. It shows North American sites carry more total bot traffic in absolute terms, even though European sites see a higher share of their total requests come from bots.
Different datasets, different methodologies, same underlying fact: nobody scraping your site asked permission, and nobody is required to explain why they ignored the file that said no.
What This Means If You Publish From Outside the US
If your site serves a non-English audience or sits on European infrastructure, assume your actual scrape rate is worse than whatever average gets quoted to you.
A disallow rule in robots.txt is a data point for compliant crawlers, not a wall.
The only real levers are the ones you control at the infrastructure layer: bot-management rules at the CDN or WAF level that can block by behavior rather than by self-identification, and server-side monitoring that tells you which agents are actually hitting your pages regardless of what they claim to be.
Pull your own server logs this week. Compare declared bot traffic against your robots.txt rules. If the gap between what you disallowed and what still got through looks anything like TollBit's numbers, the file did not do the job you assumed it was doing.
WP RADAR
This week in WP Radar: WordPress course tools reshuffle, AI Overviews chip away at organic clicks, and two different companies bet that paying creators for their content beats scraping it for free.
Is AI Going to Kill WordPress? Not Yet. WordPress still powers roughly 41.5% of the web, and the counterargument to hand-wavy AI-killer takes is that WordPress is absorbing AI at every layer instead of losing to it: content generation, core APIs, page building, and maintenance tooling. That absorption is slower than hype cycles want. Site owners betting on WordPress dying before it adapts are making the wrong bet twice.
LearnDash's Owner Just Laid Off 25% of Staff. Liquid Web cut roughly a quarter of the StellarWP team, retired the StellarWP brand, and moved LearnDash to new three-tier pricing starting at $259 a year. Nothing forces existing customers out today, but a lapsed subscription now requires buying into the new plan to reinstate access. Course creators who never inventoried their content are the ones who will get stuck mid-move.
AI Overviews Can Cut Your Clicks by 38%. Researchers at the Indian School of Business and Carnegie Mellon found AI Overviews can reduce organic clicks by as much as 38%, but the fix isn't panic, it's triage. Sort affected keywords into ones worth defending with unique tools and gated content, ones worth adapting with deeper follow-up answers, and ones not worth the effort at all. Not every lost click was a click you needed.
A Startup Wants to Pay Creators for AI Footage. Diffraction, the company behind StreamGenie, is building contracts and infrastructure so creators get paid when their archived video trains AI robotics models, generating an estimated 40,000 to 50,000 hours of source footage a month from its own network alone. The pitch to AI labs is provenance: content that is cleared, traceable, and consented to. Whether that pricing model scales past one company remains in question.
Apple Will Pay Publishers Per Use, Not Per Deal. Apple is reportedly negotiating a nine-figure budget to pay publishers only when Siri actually uses their content for current news, breaking from the industry norm of flat licensing fees for broad access. Variable, usage-based payment is a meaningfully different economic model than the flat-fee deals publishers signed with other AI companies. It ties publisher revenue directly to how often their work gets used instead of a one-time buyout.
42% of negative WordPress plugin reviews blame reliability problems. Only 2.5% blame missing features.
Matt Cromwell analyzed 3,354 one-star and five-star reviews from 100 popular WordPress plugins. Broken functionality, disruptive updates, poor performance, and unmet expectations generated far more dissatisfaction than a limited feature list.
The positive reviews reinforced the same lesson: 21% praised ease of use, 20% praised dependable functionality, and 17% highlighted helpful human support.
Plugin teams keep competing through longer roadmaps. Users appear to care much more about whether the product works, whether they can understand it, and whether someone can help when it breaks.
That’s all for this week!
Michael - Operator @WP Folio - now WP Defense Lab. Same Plugins. Different Name.