Update, 18 October: After a backlash from publishers, Meta removed the sentence naming "train" from the US-English edition of these terms on 16–17 October. The revised wording still lets it crawl linked websites and use what it finds to "develop, improve, research" its products, it doesn't exclude training, and Meta says nothing about how its crawlers work has changed. The UK-English preview checked on 17 October still contained the training wording. The advice below stands.
If your club has a Facebook Page and a website, you probably have a decision to make before 30 October, and nobody on your committee has been told about it.
That's the day Meta's new Commercial Terms take effect. Buried in them is permission for Meta to send automated crawlers to the websites linked to your business accounts and use what they find to train its products, including Meta AI. There's no opt-out switch in the terms. Keep running the Page and you've agreed.
For a newsroom that's a licensing fight. For a grassroots club it's simpler: your fixtures, match reports, junior team pages and volunteer contacts sit on a website Meta now has written permission to read. Here's what changed, what you can do in twenty minutes, and what you can't control at all.
What Meta's new terms actually say
Notices started appearing in Meta Business Suite on 2 October. Nieman Lab put three legal experts on the wording on 15 October, and the story has run everywhere since. In plain English:
- Who it covers: anyone using Facebook, Instagram, Messenger or WhatsApp "for business or commercial purposes". That includes managing Pages, running ads, selling products and using Meta's business tools. A club Page that boosts posts, promotes sponsors or sells raffle and presentation-night tickets is clearly in scope. Even a plain club Page is run through Meta's business tools.
- What it allows: Meta, "or anyone acting for it", can use scrapers, bots and crawlers on the domains and pages "associated with" your products or services. It can come back periodically for updates and can create "test credentials", which one law professor read as permission to sign up as a fake user to get past a registration wall.
- What the content is for: Meta may use it to develop, improve and train its products. Meta AI is one of those products. (The US edition has since dropped the word "train" but kept "develop, improve, research". See the update above.)
- The opt-out: there isn't one for crawling or training. The only opt-out in the new terms is for the arbitration clause. If you don't agree, Meta's notice says, stop using its products for business.
"Associated with" isn't defined. Assume it means any website you've linked from your Page, your Instagram bio, your ads or your WhatsApp Business profile.
Meta's spokesperson told The Verge that suggestions its use of crawlers is changing are false, and that site owners can still use robots.txt to say what they want. That's the opening clubs should use.
The twenty-minute fix: tell Meta's training crawler no
Every website has (or can have) a small file called robots.txt that tells automated crawlers which parts of the site they may visit. Meta runs several crawlers, and its own developer documentation separates them by job:
- meta-externalagent: training AI models (and some indexing). Respects robots.txt.
- meta-webindexer: Meta AI search answers, citing and linking to sites. Respects robots.txt.
- meta-externalads: ads and business products. Respects robots.txt.
- meta-externalfetcher: fetching a single link when a user asks. May bypass robots.txt.
- facebookexternalhit: link previews when someone shares your page. May bypass robots.txt.
The one you care about is meta-externalagent. Blocking it takes two lines added to your robots.txt:
User-agent: meta-externalagent
Disallow: /
Leave facebookexternalhit alone. Block that and every link you share to your own Page loses its preview picture and headline, and your match reports stop looking clickable. Whether to keep meta-webindexer is a judgement call. It's the crawler that could get your club cited when someone asks Meta AI "which club near me has a girls' under-10s team?" If you want that visibility, leave it in. We covered that trade-off in our AI search visibility guide.
You won't be alone. Around 19% of top websites already block meta-externalagent, according to one public tracker, and Cloudflare's data shows Meta's share of AI and search crawler traffic rising from 0.9% to 7.5% between July 2024 and July 2025.
Where the file lives on your club's site
- WordPress: most SEO plugins have a robots.txt editor under their tools or settings menu. If you've got access to your hosting file manager, it's a plain text file in the site's root folder.
- Wix: there's a robots.txt editor in the site's SEO settings.
- Squarespace and other hosted builders: you usually can't edit the file directly. Look for an AI crawler setting under your SEO or crawler options and check whether its list covers Meta's training crawler.
- A club platform that hosts your site for you: you don't control robots.txt, the platform does. Email them and ask one question: "Do you block meta-externalagent, and will you before 30 October?" Get the answer in writing.
- Behind Cloudflare: its AI crawler controls can do the job without touching the file.
Once it's done, visit yourclubsite.co.uk/robots.txt in a browser and check the two lines are there.
What robots.txt can't do
Be honest with the committee about the limits.
It's a request, not a lock. robots.txt works because well-behaved crawlers choose to obey it. Meta says its training crawler does. Two of its crawlers say they may bypass it, though not for training.
The contract question is open. The legal experts Nieman Lab spoke to warned Meta could argue that accepting the terms overrides a robots.txt block. Nobody has tested that in court. One lawyer's advice to publishers was to assume the clause is enforceable. Meta also hasn't said the UK or Europe is excluded.
It doesn't reach backwards. Anything already copied is already copied.
So treat robots.txt as the seatbelt, not the brakes. The brakes are what you publish in the first place.
The real fix: don't publish what you wouldn't want copied
This is the part that matters most for grassroots clubs, and it was true long before 30 October. Anything on a public page can be read by any crawler, scraped by anyone, and screenshotted forever. Meta's terms just make it obvious.
Walk through your site this week with three questions:
- Are any juniors identifiable? Full names next to photos, shirt names on team pages, match reports naming the under-12s' scorers. Our guide to blurring junior photos covers the FA's guidance and the free tools. First names only, no surnames with images, and no junior photos at all without consent on file.
- Is there personal contact data? Volunteers' mobile numbers and home email addresses on the "Contact us" page are a gift to scrapers and spammers alike. Use role-based addresses (secretary@, welfare@) and a contact form.
- Is anything members-only sitting on a public URL? Committee minutes, disciplinary notes, AGM accounts with individual names. If it needs a password, give it one. If it doesn't need to be online, take it down.
Write the outcome into your club policy pack as a short "what goes on our website" paragraph, so the next web volunteer doesn't undo it.
The decision your committee actually has to take
There are three realistic positions:
- Block training, keep visibility: block meta-externalagent, leave meta-webindexer and facebookexternalhit open. Most clubs will land here.
- Block everything Meta can be asked to stop: add meta-webindexer and meta-externalads too. Your previews still work, but you give up any chance of Meta AI citing you.
- Do nothing: a legitimate choice if your site holds nothing but fixtures and sponsor thank-yous. Just make it a decision rather than an accident.
Quitting Facebook isn't on the list. For most clubs the Page is where parents actually read things, and walking away from your own audience to protect a fixtures page is the wrong trade.
If you're already reviewing which AI tools touch club data, fold this into the same checklist as our AI agent safety guide. The question each time is: what can this see, and did we choose that?
Before 30 October: the checklist
- Find out who controls your club website's robots.txt (you, a volunteer, or your platform)
- Add the two-line block for meta-externalagent, or get your platform's answer in writing
- Check yourclubsite/robots.txt shows the change
- Confirm link previews still work when you share a match report to Facebook
- Audit public pages for identifiable juniors, personal phone numbers and members-only documents
- Minute the decision at the next committee meeting
None of it costs money. One volunteer, one evening, before the deadline. If your club is also using that Facebook audience to raise funds, run draws through a platform built for compliance, like PlayFundWin, so entrant data never sits on a public club page in the first place.
Up the volunteers. Non-league — making the difference.
More from the LeaguePages blogPrefer the interactive version? Read this post in the app at leaguepages.com/blog/meta-ai-crawler-club-website-30-october-guide — with related reads, series navigation and saved posts.