Web based family history software

Important Search algorythm has changed ??

  • bertkoor
  • Offline
  • Platinum Member
  • Platinum Member
  • Greetings from Utrecht, Holland
More
3 years 8 months ago #21 by bertkoor
Andreas, could you test the following:
- make a GEDCOM export from your main tree.
- import the exported GEDCOM file in a new tree
- test in the new tree

Sometimes database indexes can become somewhat corrupted and need to be rebuilt, especially when the database is in use over several years.
I would actually advise to drop the database and start fresh with only a GEDCOM import, but that could have too much impact for you.
Data not stored in GEDCOM, such as users, would be lost.

stamboom.BertKoor.nl runs on webtrees v1.7.13

Please Log in or Create an account to join the conversation.

  • Andreas
  • Topic Author
  • Offline
  • Premium Member
  • Premium Member
More
3 years 8 months ago - 3 years 8 months ago #22 by Andreas
Thank you for the proposition.

In fact I use webtrees since the beginning of it's development from PhpGedView.
But I often restartet with a fresh database.

Last time very recently when migrating to V2.0.1

The users can be restored, too. That's really not the problem.
A 1:1 copy of the Gedcom works fine in a V1.7.17 environment.

So I will try to re-import the Gedcom to a new fresh tree and test it.
I created the new tree with the re-imported Gedcom and tried to repeat the search for "Andreas Hoffmann Schubert".

What a surprise. Here I receive either 14 FAM records or none. No importance whether I'm searching in the Test tree or Le Grand Arbre Commun

By the way, the same search in webtrees 1.7.17 gives the correct number of no results.
wt1.rauhut.eu/search.php?action=header&g...nn+Schubert+&x=0&y=0

Both MAUPILLÉ & RAUHUT families are using webtrees V2.1.18
Last edit: 3 years 8 months ago by Andreas.

Please Log in or Create an account to join the conversation.

  • Andreas
  • Topic Author
  • Offline
  • Premium Member
  • Premium Member
More
3 years 1 month ago - 3 years 1 month ago #23 by Andreas
Is there no one who has similar problems ?

I start to think that searching for persons or families is simply not possible with V2.x .

Let me give you an exemple:
When I am looking for families with the names LEHMANN and RAUHUT with simple search by entering the two names I get
1 Individual & 1554 families


When looking only for LEHMANN I get
24 individuals & 15 families

Within these 15 families I detected 4 couples of LEHMANN & RAUHUT as bride and groom.

Please try to explain this behavior.

Is there a limit in the numbers of records to be searched ? stats

By the way webtrees 1.7.17 is giving the good answer

Both MAUPILLÉ & RAUHUT families are using webtrees V2.1.18
Last edit: 3 years 1 month ago by Andreas.

Please Log in or Create an account to join the conversation.

  • fisharebest
  • Away
  • Administrator
  • Administrator
More
3 years 1 month ago #24 by fisharebest
If I search your site, I find 4 families where the husband/wife are LEHMANN & RAUHUT.

wt.rauhut.eu/tree/GAC.ged/search-general?query=LEHMANN%20RAUHUT&search_individuals=0&search_families=1&search_repositories=0&search_sources=0&search_notes=0&search_trees%5B0%5D=GAC.ged

You are saying that there are others?

Can you give the URL of one that is missing?

Greg Roach - greg@subaqua.co.uk - @fisharebest@phpc.social - fisharebest.webtrees.net

Please Log in or Create an account to join the conversation.

  • Andreas
  • Topic Author
  • Offline
  • Premium Member
  • Premium Member
More
3 years 1 month ago - 3 years 1 month ago #25 by Andreas
Fine. Now the result is OK for me, too.

I believe, that I have to repeat my searches at least two times and it might be that it is related to the errors in the web log.

I see very often this kind of error.
Code:
SQLSTATE[42000]: Syntax error or access violation: 1203 User rauhutbase already has more than 'max_user_connections' active connections …/vendor/illuminate/database/Concerns/ManagesTransactions.php:109 #0 …/vendor/illuminate/database/Concerns/ManagesTransactions.php(109): PDO->beginTransaction() #1 …/vendor/illuminate/database/Concerns/ManagesTransactions.php(93): Illuminate\Database\Connection->createTransaction() #2 …/vendor/illuminate/database/Concerns/ManagesTransactions.php(23): Illuminate\Database\Connection->beginTransaction() #3 …/app/Http/Middleware/UseTransaction.php(46): Illuminate\Database........................

What can I do against this ?

Both MAUPILLÉ & RAUHUT families are using webtrees V2.1.18
Last edit: 3 years 1 month ago by Andreas.

Please Log in or Create an account to join the conversation.

  • fisharebest
  • Away
  • Administrator
  • Administrator
More
3 years 1 month ago #26 by fisharebest
> What can I do against this ?

Your webhost has created a limit on the number of database connections that you can make.

If you have lots of visitors, then this will create lots of database requests.

Perhaps you have a lot of human visitors - or perhaps you have a lot of robot visitors.

Greg Roach - greg@subaqua.co.uk - @fisharebest@phpc.social - fisharebest.webtrees.net

Please Log in or Create an account to join the conversation.

  • Andreas
  • Topic Author
  • Offline
  • Premium Member
  • Premium Member
More
3 years 1 month ago #27 by Andreas
... and I suppose that I can't increase the limit, but I have to exclude unwanted robots.

And probably these errors are the reason for the unpredictable results of my searcxh queries.

Is there a way to see who or what is creating these requests ?

Both MAUPILLÉ & RAUHUT families are using webtrees V2.1.18

Please Log in or Create an account to join the conversation.

  • fisharebest
  • Away
  • Administrator
  • Administrator
More
3 years 1 month ago #28 by fisharebest
You will see them in your webserver access logs.

Greg Roach - greg@subaqua.co.uk - @fisharebest@phpc.social - fisharebest.webtrees.net

Please Log in or Create an account to join the conversation.

  • Andreas
  • Topic Author
  • Offline
  • Premium Member
  • Premium Member
More
3 years 1 month ago #29 by Andreas
OK. I see.

And how can I get rid of them ? Many IP adresses and search engin names appear really often.

Is there a list of undesired bots or IP adresses, where I can add manually all these unwanted guests ?

Both MAUPILLÉ & RAUHUT families are using webtrees V2.1.18

Please Log in or Create an account to join the conversation.

More
3 years 1 month ago #30 by ddrury
This is quite useful

Read all of it including the comments, scroll down until you hit the comment with

SetEnvIfNoCase User-Agent (libwww-perl|wget|LieBaoFast|Mb2345Browser|zh-CN) bad_bot

in it. for the best syntax to use

BTW you must be able to edit .htaccess

--
Dave

Local: Win 11 Pro, WSL2/Ubuntu20.04.4, Apache 2.4.51, PHP 7.4.26/8.1.7, MySQL 8.0.27
Production: Litespeed 8.0.1, PHP 8.1.9, MySQL 8.0.26

Please Log in or Create an account to join the conversation.

  • Andreas
  • Topic Author
  • Offline
  • Premium Member
  • Premium Member
More
3 years 1 month ago - 3 years 1 month ago #31 by Andreas
@Greg

fisharebest wrote:
1) delete any existing robots.txt
2) obtain the default by visiting your-site/robots.txt
3) copy/paste the text into your own robots.txt
4) edit the file to add your own rule.

Where can I find robots.txt in my installation?
There is none in the root that I can delete. :-((

Both MAUPILLÉ & RAUHUT families are using webtrees V2.1.18
Last edit: 3 years 1 month ago by Andreas.

Please Log in or Create an account to join the conversation.

  • Andreas
  • Topic Author
  • Offline
  • Premium Member
  • Premium Member
More
3 years 1 month ago #32 by Andreas
@ddrury

ddrury wrote: This is quite useful

Read all of it including the comments, scroll down until you hit the comment with

SetEnvIfNoCase User-Agent (libwww-perl|wget|LieBaoFast|Mb2345Browser|zh-CN) bad_bot

in it. for the best syntax to use

BTW you must be able to edit .htaccess

I'll read it carefully, trying to understand it.

Both MAUPILLÉ & RAUHUT families are using webtrees V2.1.18

Please Log in or Create an account to join the conversation.

  • Andreas
  • Topic Author
  • Offline
  • Premium Member
  • Premium Member
More
3 years 1 month ago - 3 years 1 month ago #33 by Andreas
I modified my .htaccess file and for the moment I have no error messages in the webtrees log.

But the search for lehmann rauhut gives my the same bad result.



Only when I restart the search a second time it gives me the correct result of 1 IND and 4 FAM records

Both MAUPILLÉ & RAUHUT families are using webtrees V2.1.18
Attachments:
Last edit: 3 years 1 month ago by Andreas.

Please Log in or Create an account to join the conversation.

  • Andreas
  • Topic Author
  • Offline
  • Premium Member
  • Premium Member
More
3 years 1 month ago - 3 years 1 month ago #34 by Andreas

Andreas wrote: I modified my .htaccess file and for the moment I have no error messages in the webtrees log.


And now the bot is angry. I had about 1500 error statements in less than half an hour (2021-02-08 12:39:04 - 2021-02-08 12:58:02) in the webtrees log

The access-log of the server told me that one IP 27.110.165.84 accessed all my websites for more than one hour between 08/Feb/2021:12:31:41 +0100 and 08/Feb/2021:13:28:11 +0100

Is there someone, who can help me ?

Both MAUPILLÉ & RAUHUT families are using webtrees V2.1.18
Last edit: 3 years 1 month ago by Andreas.

Please Log in or Create an account to join the conversation.

More
3 years 1 month ago #35 by Sir Peter
Replied by Sir Peter on topic Search algorythm has changed ??
I am not an expert for this, but you can try to block the complete network - at least for some time. Either follow htaccessbook.com/block-ip-address/ to block them via .htaccess or run a whois service like www.whois.com/whois/27.110.165.84 to find the ASN as explained by Greg in www.webtrees.net/index.php/en/forum/help...log-ver-2-0-11#82338 and block them via the config.ini.php

The attack seems to come from
Code:
% Information related to '27.110.160.0/19AS9299' route: 27.110.160.0/19 origin: AS9299 descr: Philippine Long Distance Telephone Co. 14/F Ramon Cojuangco Building Makati Avenue mnt-by: PHIX-NOC-AP last-modified: 2019-10-01T04:00:20Z source: APNIC

Then add a line like this to the config.ini.php and see whether it helps.
Code:
block_asn="AS9299"

Please let us know whether it worked.

Peter

Please Log in or Create an account to join the conversation.

  • Andreas
  • Topic Author
  • Offline
  • Premium Member
  • Premium Member
More
3 years 1 month ago - 3 years 1 month ago #36 by Andreas
Replied by Andreas on topic Search algorythm has changed ??
Thank you Peter,

for giving me that link to Gregs information how to block a network.

The string was added to my config.ini.php. I don't have too much active human users in Indonesia.

But I have to block them in the root of my server space.

From this adress came requests also to all of my other websites.

Both MAUPILLÉ & RAUHUT families are using webtrees V2.1.18
Last edit: 3 years 1 month ago by Andreas.

Please Log in or Create an account to join the conversation.

More
3 years 1 month ago #37 by Sir Peter
Replied by Sir Peter on topic Search algorythm has changed ??
The blocking via config.ini.php takes place on the application (PHP) level while blocking via .htaccess takes places on the webserver (Apache) level, but only for a specific domain. If you are running multiple subdomains together with your top level domain on the same webserver you will still have to maintain all .htaccess files unless you have access to the overall Apache configuration. If you are on a hosted server I assume you don’t have the necessary access rights.

Peter

Please Log in or Create an account to join the conversation.

  • Andreas
  • Topic Author
  • Offline
  • Premium Member
  • Premium Member
More
3 years 1 month ago #38 by Andreas
Replied by Andreas on topic Search algorythm has changed ??
Yes, indeed. I don’t have the necessary access rights.
I'm dealing with the .htaccess files in each root directory.

But I don't give up.

Both MAUPILLÉ & RAUHUT families are using webtrees V2.1.18

Please Log in or Create an account to join the conversation.

Powered by Kunena Forum
}