Question Google index coverage

I would like to share your experiences with Google Index.

I currently have about 9,000 records in my Gedcom. Under V 1.7 I had 90% coverage in the Goolge Index. After migrating to rel. 2.0/2.1 and other changes (e.g. https) the coverage has dropped a lot and is now below 20%.

I've been trying to increase coverage for about 4 months, but the trend remains unbroken (beginning of September 36%, now 19%).
The average response time of my site ( wbt.warius.info ) is 300 ms and thanks to bot seokicks (now blocked) I know that it can deliver 17.5k requests per day without sacrificing performance.

The measure helped Bing and the daily crawl queries rose from 200 to 500, but Google continues to fluctuate between 120 and 200 queries per day. A very large part of this are redirects from old Rel. 1.7 links.

What is your experience and what actions have you taken?

You are right, it was a problem with the canonical URL. in june i developed a fix and the google coverage is increasing again.

While webtrees doesn't have duplicate URLs but over 15% of the Google crawler requests are old WT 1.7 URLs that Google puts through each other. Here the entry canonical helps. 
 

The fix can be found in the issue  SEO: add <link rel=canonical> to default.phtml · Issue #4873 · fisharebest/webtrees (github.com)
 

I have been analyzing the problem with the Google Index for over a year and am permanently optimizing the website. 

From over 5,000 indexed pages with webtrees 1.7, the Google index coverage with webtrees 2.0 / 2.1 decreased to under 700.

I believe the cause is that Google does not forget sites once crawled so quickly and keeps crawling them even after years. webtrees redirects the page to the current URL and Google then crawls it. So Google has 2 pages with almost identical content and the canonical link helps Google to recognize and index the current one and forget the old one.

Since I added this code to default.phtml on my site in June, the coverage increases by about 50 pages per week. So this old issue is done for me, but it would be nice if the fix in webtrees is adopted for all.

Documentation from Google suggests that this is not the correct solution.

See my comment at github.com/fisharebest/webtrees/issues/4...uecomment-1786152523

> From over 5,000 indexed pages with webtrees 1.7, the Google index coverage with webtrees 2.0 / 2.1 decreased to under 700.

I host a number of webtrees sites.  This includes

1) sites that have always been webtrees 1.x
2) sites that have always been webtrees 2.x with pretty URLs.

These sites cannot have this "duplicate URL" issue, because they only ever used the same URLs.

And yet I see the same thing with all of them.  Over recent years, the number of requests from GoogleBot is declining, and the number of requests from BingBot is increasing.