Question Increase Access denied
- braulio_holtz
- Topic Author
- Visitor
10 years 5 months ago #1
by braulio_holtz
Increase Access denied was created by braulio_holtz
Increase Access denied by Google Webmaster Tools would check the user_agent?
Example, holtzgen.com/famlist.php?alpha=F goes normally, but in Webmaster Tools shows as access denied.
Robots.txt
The table Wt_site_access_rule has over 1000 records.
user_agent as SAMSUNG-SGH-E250/1.0 Profile/MIDP-2.0 Configuration/CLDC-1.1 UP.Browser/6.2.3.3.c.1.101 (GUI) MMP/2.0 (compatible; Googlebot-Mobile/2.1; +http://www.google.com/bot.html) is unknown
Example, holtzgen.com/famlist.php?alpha=F goes normally, but in Webmaster Tools shows as access denied.
Robots.txt
Code:
User-Agent: *
Allow: /family.php
Allow: /index.php
Allow: /indilist.php
Allow: /individual.php
Allow: /mediafirewall.php
Allow: /mediaviewer.php
Allow: /module.php?mod=sitemap
Allow: /note.php
Allow: /repo.php
Allow: /source.php
Disallow: /site-offline.php
Disallow: /site-unavailable.php
Disallow: /login.php
Disallow: /module.php?mod=*&mod_action=ajax&sb_action=*
Disallow: /reportengine.php
Disallow: /index.php/install/*
Disallow: /login.php
The table Wt_site_access_rule has over 1000 records.
user_agent as SAMSUNG-SGH-E250/1.0 Profile/MIDP-2.0 Configuration/CLDC-1.1 UP.Browser/6.2.3.3.c.1.101 (GUI) MMP/2.0 (compatible; Googlebot-Mobile/2.1; +http://www.google.com/bot.html) is unknown
Please Log in or Create an account to join the conversation.
- fisharebest
- Away
- Administrator
Less
More
10 years 5 months ago #2
by fisharebest
Greg Roach - greg@subaqua.co.uk - @fisharebest@phpc.social - fisharebest.webtrees.net
Replied by fisharebest on topic Increase Access denied
We do not show famlist.php to search engines because
(a) these pages can consume a lot of CPU/memory
(b) it is possible for a search engine to find all the individuals from indilist.php
If you enable the sitemap module (you should!), then you should also block indilist.php in robots.txt
(a) these pages can consume a lot of CPU/memory
(b) it is possible for a search engine to find all the individuals from indilist.php
If you enable the sitemap module (you should!), then you should also block indilist.php in robots.txt
Greg Roach - greg@subaqua.co.uk - @fisharebest@phpc.social - fisharebest.webtrees.net
Please Log in or Create an account to join the conversation.
- braulio_holtz
- Topic Author
- Visitor
10 years 5 months ago #3
by braulio_holtz
Replied by braulio_holtz on topic Increase Access denied
Understand, funny that the problem occurs with a constant number of days during tracking
I will create a sitemap and I will put as allow: famlist.php the robots.txt to see if it improves
I will create a sitemap and I will put as allow: famlist.php the robots.txt to see if it improves
Please Log in or Create an account to join the conversation.
- ToyGuy
- Offline
- Moderator
- Live like it's Christmas every day - Santa Stephen
10 years 5 months ago #4
by ToyGuy
Santa Stephen the Fabled Santa
Latest webtrees at MyArnolds.com
Hosted by webtreesonline.com , a division of GeneHosts LLC
MacOS 10.6.8, Apache 2.2+, PHP 5.4.16, MySQL 5.5.28
Replied by ToyGuy on topic Increase Access denied
Think you may have misunderstood. We do NOT recommend giving the bots access to famlist.php. On any site with a substantial family tree, the burden of bandwidth and processing time can be exorbitant. A sitemap will direct the bots to all INDI records and that is much more efficient. Little can be gained from adding the FAM records. Of course, the choice is yours, but don't complain if you suffer performance issues from the abuse, or complaints on being a resource hog from your ISP .
Santa Stephen the Fabled Santa
Latest webtrees at MyArnolds.com
Hosted by webtreesonline.com , a division of GeneHosts LLC
MacOS 10.6.8, Apache 2.2+, PHP 5.4.16, MySQL 5.5.28
Please Log in or Create an account to join the conversation.