Please do NOT post requests for help here. Use the Help forum for that.
  • Page:
  • 1

TOPIC:

Google and Bing Indexing 2 months 2 weeks ago #1

  • makitso
  • makitso's Avatar Topic Author
  • Offline
  • Moderator
  • Moderator
  • Posts: 529
I have been trying to figure out how to influence both of the subject indexers to give more retentive information via search.
Right now, a google search on my domain name is showing the "For technical support or genealogy questions..." from my home page.
Using google.com, if I search the content of my Title tag I get "Search. Theme. clouds colors F.A.B. minimal webtrees xenea. Palette. Aqua Marine Ash Belgian Chocolate Blue Lagoon Blue Marine Coffee..."

Any suggestions?
Rob
www.skatekey.net ( webtrees beta GitHub)
PHP 7.4.14, MySql 7.4.14, Apache 2.4.43,
Hosted at tigertech.net

Please Log in or Create an account to join the conversation.

Google and Bing Indexing 2 months 2 weeks ago #2

  • eh215
  • eh215's Avatar
  • Offline
  • Junior Member
  • Junior Member
  • Posts: 237
Since you didn't mention what steps you have already taken, here are some basic ones that often help with search-engine visibility:
  • Update your website's robot.txt file to include a link to your sitemap.
    Sitemap: https://domain-name/wt/sitemap.xml
  • Go into the Google Search Console and Bing Webmaster Tools and complete the steps for authenticating your website to them. I believe both give you the option of specifying a sitemap file, I do it here also. Both will commonly give you a coded file to drop into the base directory of your website as a way of confirming you own the site.

I find that Google does a better job of indexing my site than Bing, but it is never 100%, particularly if you have pages that haven't been changed in a long time.

Eric
behunt.net/ft
PHP 7.4.16

Please Log in or Create an account to join the conversation.

Google and Bing Indexing 2 months 2 weeks ago #3

  • makitso
  • makitso's Avatar Topic Author
  • Offline
  • Moderator
  • Moderator
  • Posts: 529
Thanks Eric, I am setting up a sitemap as we speak.
Rob
www.skatekey.net ( webtrees beta GitHub)
PHP 7.4.14, MySql 7.4.14, Apache 2.4.43,
Hosted at tigertech.net

Please Log in or Create an account to join the conversation.

Google and Bing Indexing 2 months 2 weeks ago #4

  • fisharebest
  • fisharebest's Avatar
  • Offline
  • Administrator
  • Administrator
  • Posts: 14511
If you use "Pretty URLs", then webtrees will generate robots.txt automatically.
If you enable the sitemap module, then the robots.txt will automatically contain a link to the sitemap.

Do you use "Google webmaster tools"?
(TIP - webtrees has a module which sets the google-verification-id field automatically)
This will give you good feedback about your site's search indexes.
Greg Roach - This email address is being protected from spambots. You need JavaScript enabled to view it. - fisharebest.webtrees.net

Please Log in or Create an account to join the conversation.

Google and Bing Indexing 2 months 2 weeks ago #5

  • eh215
  • eh215's Avatar
  • Offline
  • Junior Member
  • Junior Member
  • Posts: 237
Good to know!

I couldn't remember if I had created the robots.txt entries or not
behunt.net/ft
PHP 7.4.16

Please Log in or Create an account to join the conversation.

Google and Bing Indexing 2 months 2 weeks ago #6

  • makitso
  • makitso's Avatar Topic Author
  • Offline
  • Moderator
  • Moderator
  • Posts: 529
I have created a sitemap with web-site-map.com. However, there are a lot of pages in the resulting sitemap.xml file should not be indexed. The page below has the meta tag to noindex.

<url><loc>skatekey.net/tree/peters.ged/report/indi...al_report?xref=I1642daily0.56

How can I prevent indexing of reports, calendar pages?
Rob
www.skatekey.net ( webtrees beta GitHub)
PHP 7.4.14, MySql 7.4.14, Apache 2.4.43,
Hosted at tigertech.net

Please Log in or Create an account to join the conversation.

Google and Bing Indexing 2 months 2 weeks ago #7

  • fisharebest
  • fisharebest's Avatar
  • Offline
  • Administrator
  • Administrator
  • Posts: 14511
webtrees has built-in sitemap functionality.

It uses the standard privacy logic to decide which pages to include and which pages to exclude.

Just enable one or more trees in the "Sitemap" module…

The module also provides links so that you can submit the sitemap directly to major search engines automatically.
Greg Roach - This email address is being protected from spambots. You need JavaScript enabled to view it. - fisharebest.webtrees.net

Please Log in or Create an account to join the conversation.

Google and Bing Indexing 2 months 2 weeks ago #8

  • makitso
  • makitso's Avatar Topic Author
  • Offline
  • Moderator
  • Moderator
  • Posts: 529
Thanks Greg. My point here is that web-site-map.com, that I am using to generate an sitemap is quiting after 5000 pages. It appears that its indexing a lot of calendar and report pages, that I don't need.
So, I am trying to stop that from happening......

I have added these entries to my robot.txt file
User-agent: *
Disallow: /tree/peters.ged/calendar
Disallow: /tree/Johannsen/calendar
Disallow: /tree/peters.ged/report
Disallow: /tree/Johannsen/report
Rob
www.skatekey.net ( webtrees beta GitHub)
PHP 7.4.14, MySql 7.4.14, Apache 2.4.43,
Hosted at tigertech.net

Please Log in or Create an account to join the conversation.

Last edit: by makitso.
Do you need a web hosting solution for your webtrees site?
If you prefer a host that specialises in webtrees, the following page lists some suppliers able to provide one for you: 
  • Page:
  • 1
Powered by Kunena Forum