meta-webindexer
by Meta
Builds the index an AI assistant searches when it needs to find and cite a source.
Meta crawler used to improve Meta AI search result quality.
What it usually means
Meta is indexing this page so it can surface in Meta AI search answers.
How to identify it
meta-webindexer User-agent: meta-webindexer How it's verified
57.141.4.0/24157.240.238.0/2457.144.114.0/2357.145.8.0/232a03:2880:f013::/48157.240.12.0/24157.240.14.0/242a03:2880:f16d::/4857.144.178.0/232a03:2880:f158::/48 1 IP source configured
Sources
More from Meta
Meta fetcher used when a person requests or shares a specific URL.
facebookexternalhit is an AI-related crawler operated by Meta.
Meta crawler collecting public content to train its AI models and improve Meta products.
meta-externalads is an AI-related crawler operated by Meta.
FacebookBot is an AI-related crawler operated by Meta.
Similar bots
Anthropic crawler that discovers and refreshes content for Claude search.
Moonshot AI crawler that builds the index behind Kimi's web answers.
Mistral crawler that builds the index behind Le Chat's web answers.
OpenAI crawler that builds the search index behind ChatGPT's web results.
See if meta-webindexer has visited your site
Kitbase verifies and tracks bot and AI crawler traffic separately from human visitors, so you know exactly who's hitting your site.
Start Free Trial