Bővítmény címke: crawler
-
Robots.txt Extender
Dynamic robots.txt for Multisite! Change parameters, or don't, for each site of your network without losing the defaults from WordPress Includes.
-
SEObot Monitor for Googlebot, Bingbot and search engine spiders
With SEObot Monitor for Googlebot you will be able to dump the server logs to Google Analytics, easily and automatically. This will allow you to dete …
-
-
JaspreetChahal’s wordpress bot detector lite
This plugin detects a few bots (e.g. Google, Google ads, Alta vista etc) and sends an email based on interval set by you. you can choose what format e …
-
Valvai – AI Traffic Control Center
Block, log and monetise AI crawlers directly from your WordPress dashboard.
-
AI Bot Monitor
Tracks and logs AI, Search and Social bots including ChatGPT, Perplexity, Googlebot, Bing and Meta crawlers.
-
GoBookMart Robots Editor
A lightweight plugin to edit or create robots.txt directly from your WordPress dashboard. Developed by GoBookMart.
-
-
Azayem Bots Tracker – Bot Visits Logger
A custom plugin to log search engine bot visits and analyze their crawl budget directly inside WordPress (no external services required).
-
WEO LLMS & AI Crawler Guide
The professional architect for your llms.txt file. Control how AI agents (ChatGPT, Claude, Perplexity) consume your website content with a structured, …
-
OpenBotAuth – AI Crawler Access Control
Verify AI crawlers with HTTP Signatures and enforce allow/deny/teaser policies; also serves llms.txt and AI-ready feeds.
