Question Google bot

vbueschken wrote:

I tried also to change to "Pretty URLs) by adding:
rewrite_urls="1" to my data/config.ini.php

I got: Internal Server Error
The server encountered an internal error or misconfiguration and was unable to complete your request.

Oh sorry, I overlooked that. This was posted while I was trying some things out.

It is impossible to fix an error if no further details are known. Is there perhaps something logged in an error log file?
But I also notice that your site gives this error when an unknown page is requested. The usual response is then '404 - Page not found'. I suppose something in your .htaccess file causes this.

My guess is that after setting rewrite_urls="1" you requested the old URL 'index.php?tree=tree1', but the main entry URL of webtrees is simply '/' which will redirect to the first public tree.

Yes, DuckDuckGo and Bing are working fine - its a Google Search specific issue.

I agree - its difficult to find the issue withaout access to the server and logfiles - my hoster does'nt provide this access. may be I need to move to another hosting provider

In the Crawling Statisticts from Google Search Console I found following crawling errors (Status 4xx):
 

Yes, the calendar is forbidden for bots. There is a calendar page for each day and webtrees supports also the Jewish calendar plus some other odd calendars, this results in millions of pages. Letting a bot crawl all of them can be too much for a server.

I see your robots.txt file has now changed. It should list what pages are allowed and which not, so these kind of errors can be prevented.

While you still have "ugly" urls it is not possible to write a good robots.txt, because these urls work with query parameters like "route=calendar", and that is not how robots.txt works. It needs plain and "pretty" urls.

I have tried to set up a new environment:
I have changed the config.ini.php :

base_url=" www.example.com/webtrees "
rewrite_urls="1"

when I enter the new URL in the browser ( webtrees.bueschken.com/ ) Im routed to :
webtrees.bueschken.com/tree/tree1/

The Folder /tree/tree1 doesn't exist - so I get an "Internal Server Error".

did I miss something? anything else I have to change for the pretty URL's?

> The Folder /tree/tree1 doesn't exist - so I get an "Internal Server Error".

No, that should not be the case.

In your .htaccess file you should have some configuration that when a request comes in for something which does not exists, then it routes the request to index.php which will handle it.

Did you take the example .htaccess (from the "pretty url" FAQ article? It assumes that webtrees is installed in a subdirectory. You and me have it in a subdomain. The .htaccess file on my own server is somewhat different:

Maybe it's a good idea to share your .htaccess file, so we can analyse it.

I'm now getting 404 instead of 500 when a page does not exist.
Except that index.php cannot be found. Did you delete it?

I have used code from this site:
webtrees.net/faq/urls/

thats my .htaccess:

<IfModule mod_rewrite.c>
RewriteEngine On
RewriteBase webtrees.bueschken.com

# GIT config files can contain credentials or other sensitive data.
RewriteRule \.git - [F]

# User data is stored here by default.
RewriteRule ^data(/|$) - [F]

# Nothing sensitive here, but there is no need to publish it.
RewriteRule ^app(/|$) - [F]
RewriteRule ^modules - [F]
RewriteRule ^resources(/|$) - [F]
RewriteRule ^vendor(/|$) - [F]

RewriteCond %{REQUEST_FILENAME} !-d
RewriteCond %{REQUEST_FILENAME} !-f
RewriteRule ^ index.php [L]
</IfModule>

GOT it... worked with Pretty URL's

I had to change in .htaccess :

RewriteBase webtrees.bueschken.com
to
RewriteBase /