Question
Google bot
- vbueschken
-
Topic Author
- Offline
- New Member
-
Please Log in or Create an account to join the conversation.
- bertkoor
-
- Offline
- Platinum Member
-
- Greetings from Utrecht, Holland
Yesterday I have submitted my sitemap again (had not done that in years) and some hours later the Search Console reported me it was crawling happily.
Just checking: you are running a recent version (I recall but cannot check at this moment: v2.2.6) of webtrees without any external / 3rd party modules and no modifications or patches?
stamboom.BertKoor.nl runs on webtrees v2.2.6
Please Log in or Create an account to join the conversation.
- vbueschken
-
Topic Author
- Offline
- New Member
-
Is it possible to generally disable the Bad Bot Blocker - just for testing?
Please Log in or Create an account to join the conversation.
- vbueschken
-
Topic Author
- Offline
- New Member
-
I have removed all lines with "Google" in the BadBotBlocker.php
Than I have checked the Page in Google Search Console again. now I was able to request the indexing.
I assume I dont need to delete ALL Google lines in the BadBotBocker.php....
Please Log in or Create an account to join the conversation.
- bertkoor
-
- Offline
- Platinum Member
-
- Greetings from Utrecht, Holland
I think it should suffice to remove the 'google' line within the array ROBOT_REV_FWD_DNS (line 1480)
Can you go to the webtrees Control Panel - click link Server Information - find "disable_functions".
If the value of that contains either 'gethostbyaddr' or 'gethostbyname' then that could explain why google is not recognised. Although I'm not sure... if those functions were disabled then I expect a concrete error about that.
stamboom.BertKoor.nl runs on webtrees v2.2.6
Please Log in or Create an account to join the conversation.
- vbueschken
-
Topic Author
- Offline
- New Member
-
In the fiels "disable_functions" is 'no value'
Please Log in or Create an account to join the conversation.
- bertkoor
-
- Offline
- Platinum Member
-
- Greetings from Utrecht, Holland
I guess something on your server is not acting as expected, but what...
stamboom.BertKoor.nl runs on webtrees v2.2.6
Please Log in or Create an account to join the conversation.
- vbueschken
-
Topic Author
- Offline
- New Member
-
I have diskussed it withe the support of my hosting provider - they pointet me to the BadBotBlocker.php based on their logfiles.
After deleting all Google bots from BadBotBlocker.php the issue seems to be resolved. Now I have to wait vor Google to index my page and display it in Google Search
Please Log in or Create an account to join the conversation.
- jbyvosges
-
- Offline
- New Member
-
- Français mais moi je comprendre l'anglois.
Je suis intéressé par ce sujet, obtenant les mêmes erreurs.
La commande
curl -A "Googlebot" -I lesbarbry.fr/tree/lesbarbry.ged/individu...-Anne-Josephe-BARBRY
me renvoye toujours HTTP/2 406. Je n'ai pas ce problème avec Slurp ou Bingbot , mais je l'ai avec DuckDuckBot .
J'ai essayé sans succès d'ajouter
RewriteCond %{HTTP_USER_AGENT} ^(Googlebot|Bingbot|Slurp|DuckDuckBot|Baiduspider|YandexBot|facebot|ia_archiver|Twitterbot|Pinterest) [NC]
dans mon fichier .htaccess ainsi que supprimer toutes les lignes Google dans BadBotBlocker.php
Je ne trouve pas de solution.
PHP 8.4 MariaDB 11.8.3
webtrees 2.2.6
O.S. : Linux Ubuntu 26.04 64 bits
[url] dolman.fr/ [/url] [url] lesbarbry.fr/ [/url]
Hégergés par hostinger.com
Please Log in or Create an account to join the conversation.
- bertkoor
-
- Offline
- Platinum Member
-
- Greetings from Utrecht, Holland
github.com/fisharebest/webtrees/blob/23b...BotBlocker.php#L1480
github.com/fisharebest/webtrees/blob/23b...BotBlocker.php#L1647
I bet you did not do the request from a server in a google datacentre, so these requests get blocked.
May I ask why you went investigating this? Are you having specific errors with google?
stamboom.BertKoor.nl runs on webtrees v2.2.6
Please Log in or Create an account to join the conversation.