Question
High Resource Usage
- vdrumsroland
-
Topic Author
- Offline
- New Member
-
Less
More
4 months 1 week ago - 4 months 1 week ago #1
by vdrumsroland
High Resource Usage was created by vdrumsroland
I've ran into a issue with my current, new host. I was getting the 502 error when just visiting my install. Recently my old host company was purchased and my site was migrated over to their new servers. Since then I've been dealing with resource issues. This is what they have communicated to me with their updated resource list.
• Physical Memory - retained at 1 GB
• CPU - Tempo increased from 100% to 200%
• I/O - Tempo increased from 1 MB/s to 3MB/s
• Entry Processes - Tempo increased from 10 to 20
• NPROC - Tempo increased from 30 to 40
I'm sorry I don't have more info but does this seem normal?
• Physical Memory - retained at 1 GB
• CPU - Tempo increased from 100% to 200%
• I/O - Tempo increased from 1 MB/s to 3MB/s
• Entry Processes - Tempo increased from 10 to 20
• NPROC - Tempo increased from 30 to 40
I'm sorry I don't have more info but does this seem normal?
Last edit: 4 months 1 week ago by vdrumsroland.
Please Log in or Create an account to join the conversation.
- fisharebest
-
- Away
- Administrator
-
4 months 1 week ago #2
by fisharebest
Greg Roach - greg@subaqua.co.uk - @fisharebest@phpc.social - fisharebest.webtrees.net
Replied by fisharebest on topic High Resource Usage
Can you check your webserver logs?
It could be that robots are making many requests to your site.
It could be that robots are making many requests to your site.
Greg Roach - greg@subaqua.co.uk - @fisharebest@phpc.social - fisharebest.webtrees.net
Please Log in or Create an account to join the conversation.
- vdrumsroland
-
Topic Author
- Offline
- New Member
-
4 months 1 week ago #3
by vdrumsroland
Replied by vdrumsroland on topic High Resource Usage
I've checked the webtrees website logs for anything odd and nothing is listed that shows anything out of the ordinary.
I'm also behind Cloudflare's firewall where I'm filtering many things.
I'm also behind Cloudflare's firewall where I'm filtering many things.
Please Log in or Create an account to join the conversation.
- RBaker
-
- Offline
- New Member
-
4 months 1 week ago - 4 months 1 week ago #4
by RBaker
ibm-pc.org/family is my webtrees
Replied by RBaker on topic High Resource Usage
If you cannot connect at all the Cloudflare settings may need to be updated for your new server.
I had to update nameservers and IP on Cloudflare control panel when I changed servers.
I had to update nameservers and IP on Cloudflare control panel when I changed servers.
ibm-pc.org/family is my webtrees
Last edit: 4 months 1 week ago by RBaker.
Please Log in or Create an account to join the conversation.
- vdrumsroland
-
Topic Author
- Offline
- New Member
-
4 months 1 week ago #5
by vdrumsroland
Replied by vdrumsroland on topic High Resource Usage
Cloudflare is good. It's filtering as usual.
I was just curious if webtrees should be using as much resources as my hosting says it is. As mentioned before, I havn't had this problem until they migrated my site to a new server.
Here is a screenshot of the usage they sent me.
[url] prnt.sc/SN1__hz64MpF [/url]
I was just curious if webtrees should be using as much resources as my hosting says it is. As mentioned before, I havn't had this problem until they migrated my site to a new server.
Here is a screenshot of the usage they sent me.
[url] prnt.sc/SN1__hz64MpF [/url]
Please Log in or Create an account to join the conversation.
- vdrumsroland
-
Topic Author
- Offline
- New Member
-
4 months 1 week ago #6
by vdrumsroland
Replied by vdrumsroland on topic High Resource Usage
@fisharebest
Can you tell me the typical server requirements for Webtrees? I see these, webtrees.net/install/requirements/ which I'm covered there. What my hosting is asking are the resources needed, memory, etc.
Can you tell me the typical server requirements for Webtrees? I see these, webtrees.net/install/requirements/ which I'm covered there. What my hosting is asking are the resources needed, memory, etc.
Please Log in or Create an account to join the conversation.
- fisharebest
-
- Away
- Administrator
-
4 months 1 week ago #7
by fisharebest
Greg Roach - greg@subaqua.co.uk - @fisharebest@phpc.social - fisharebest.webtrees.net
Replied by fisharebest on topic High Resource Usage
You said:
> Since then I've been dealing with resource issues.
This is quite vague.
How do you know you have resource issues?
Do you have error messages saying "out of memory", "time limit exceeded", etc.?
> I've checked the webtrees website logs for anything odd and nothing is listed that shows anything out of the ordinary.
It's not clear what you are saying.
I asked about your webserver (apache/nginx/etc.) logs - not your webtrees logs.
If you are getting 502 responses, I would expect to see them in the apache/nginx logs.
> I was just curious if webtrees should be using as much resources as my hosting says it is
I don't understand what that screenshot is showing.
I haven't seen anything like that before.
What are "A", "L" and "F"? What do the various columns mean?
> What my hosting is asking are the resources needed, memory, etc.
The major things that affect this are:
1) Size of your site - i.e. number of individuals.
2) The number of concurrent users/visitors/robots etc.
> Since then I've been dealing with resource issues.
This is quite vague.
How do you know you have resource issues?
Do you have error messages saying "out of memory", "time limit exceeded", etc.?
> I've checked the webtrees website logs for anything odd and nothing is listed that shows anything out of the ordinary.
It's not clear what you are saying.
I asked about your webserver (apache/nginx/etc.) logs - not your webtrees logs.
If you are getting 502 responses, I would expect to see them in the apache/nginx logs.
> I was just curious if webtrees should be using as much resources as my hosting says it is
I don't understand what that screenshot is showing.
I haven't seen anything like that before.
What are "A", "L" and "F"? What do the various columns mean?
> What my hosting is asking are the resources needed, memory, etc.
The major things that affect this are:
1) Size of your site - i.e. number of individuals.
2) The number of concurrent users/visitors/robots etc.
Greg Roach - greg@subaqua.co.uk - @fisharebest@phpc.social - fisharebest.webtrees.net
Please Log in or Create an account to join the conversation.
- vdrumsroland
-
Topic Author
- Offline
- New Member
-
4 months 1 week ago - 4 months 1 week ago #8
by vdrumsroland
Replied by vdrumsroland on topic High Resource Usage
All I know is what my hosting is telling me. They state it's coming from my Webtree install. This is what they sent showing what they upgrade temporarily to solve the issue, which it did.
• Physical Memory - retained at 1 GB
• CPU - Tempo increased from 100% to 200%
• I/O - Tempo increased from 1 MB/s to 3MB/s
• Entry Processes - Tempo increased from 10 to 20
• NPROC - Tempo increased from 30 to 40
Since I'm using Cloudflare (free) as a security/firewall it shows their 502 Gateway error when the above resources are not in play. I'm blocking bad bots and countries. Cloudflare was setup and I've been using it prior to the issue.
There is nothing in the webtrees log that would indicate an issue. No errors or warnings, etc.
Here is a snippet of the server raw log file. The "*******" is when I got a 502 error.
The screenshot... I too have no idea what it means. I was hoping someone else might know. It's just what they sent to show the usage.
I would consider my tree to be on the small size: 11,628 Individuals, 2,947 media objects, 30 users (only 1 is active other than me).
• Physical Memory - retained at 1 GB
• CPU - Tempo increased from 100% to 200%
• I/O - Tempo increased from 1 MB/s to 3MB/s
• Entry Processes - Tempo increased from 10 to 20
• NPROC - Tempo increased from 30 to 40
Since I'm using Cloudflare (free) as a security/firewall it shows their 502 Gateway error when the above resources are not in play. I'm blocking bad bots and countries. Cloudflare was setup and I've been using it prior to the issue.
There is nothing in the webtrees log that would indicate an issue. No errors or warnings, etc.
Here is a snippet of the server raw log file. The "*******" is when I got a 502 error.
Code:
71.83.70.172 - - [17/Dec/2024:12:54:50 -0500] "GET /tree/module/_vesta_personal_facts_/Tab/BEST_2010-05-9.ged?xref=I5474 HTTP/1.1" 200 7783 "https://www.thebestgenealogy.com/tree/tree/BEST_2010-05-9.ged/individual/I5474/Kenneth-Dale-Dale-HUFFSTETLER-III" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_6) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/14.0.2 Safari/605.1.15"
71.83.70.172 - - [17/Dec/2024:12:54:50 -0500] "POST /tree/module/_vesta_personal_facts_/Vestals HTTP/1.1" 200 846 "https://www.thebestgenealogy.com/tree/tree/BEST_2010-05-9.ged/individual/I5474/Kenneth-Dale-Dale-HUFFSTETLER-III" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_6) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/14.0.2 Safari/605.1.15"
71.83.70.172 - - [17/Dec/2024:12:54:52 -0500] "GET /tree/tree/BEST_2010-05-9.ged/individual/I4602/Kenny-Dale-HUFFSTETLER-II HTTP/1.1" 200 25653 "https://www.thebestgenealogy.com/tree/tree/BEST_2010-05-9.ged/individual/I5474/Kenneth-Dale-Dale-HUFFSTETLER-III" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_6) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/14.0.2 Safari/605.1.15"
71.83.70.172 - - [17/Dec/2024:12:54:52 -0500] "GET /tree/module/_vesta_personal_facts_/Tab/BEST_2010-05-9.ged?xref=I4602 HTTP/1.1" 200 10390 "https://www.thebestgenealogy.com/tree/tree/BEST_2010-05-9.ged/individual/I4602/Kenny-Dale-HUFFSTETLER-II" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_6) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/14.0.2 Safari/605.1.15"
71.83.70.172 - - [17/Dec/2024:12:54:53 -0500] "POST /tree/module/_vesta_personal_facts_/Vestals HTTP/1.1" 200 1086 "https://www.thebestgenealogy.com/tree/tree/BEST_2010-05-9.ged/individual/I4602/Kenny-Dale-HUFFSTETLER-II" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_6) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/14.0.2 Safari/605.1.15"
71.83.70.172 - - [17/Dec/2024:12:54:54 -0500] "GET /tree/tree/BEST_2010-05-9.ged/individual/I2906/Luther-Kenneth-HUFFSTETLER HTTP/1.1" 200 33551 "https://www.thebestgenealogy.com/tree/tree/BEST_2010-05-9.ged/individual/I4602/Kenny-Dale-HUFFSTETLER-II" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_6) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/14.0.2 Safari/605.1.15"
71.83.70.172 - - [17/Dec/2024:12:54:55 -0500] "GET /tree/tree/BEST_2010-05-9.ged/media-thumbnail?xref=M313&fact_id=62dd04cc12e500f814c7149f11c6f3ed&w=80&h=100&fit=contain&mark=0&s=9385d11d4eaa235ade51a86fcc096936 HTTP/1.1" 200 5741 "https://www.thebestgenealogy.com/tree/tree/BEST_2010-05-9.ged/individual/I2906/Luther-Kenneth-HUFFSTETLER" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_6) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/14.0.2 Safari/605.1.15"
71.83.70.172 - - [17/Dec/2024:12:54:55 -0500] "GET /tree/tree/BEST_2010-05-9.ged/media-thumbnail?xref=M890&fact_id=e16cb80f89f7e9e6ddd448f3573f8d10&w=200&h=200&fit=contain&mark=0&s=d0a48991d26c3d43019255f03a290134 HTTP/1.1" 200 8061 "https://www.thebestgenealogy.com/tree/tree/BEST_2010-05-9.ged/individual/I2906/Luther-Kenneth-HUFFSTETLER" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_6) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/14.0.2 Safari/605.1.15"
52.167.144.181 - - [17/Dec/2024:12:54:56 -0500] "GET /tree/tree/BEST_2010-05-9.ged/calendar/day?cal=%40%23DGREGORIAN%40&day=16&month=JUN&year=1921&filterev=BIRT-MARR-DEAT&filterof=all HTTP/1.1" 200 8847 "-" "Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm) Chrome/116.0.1938.76 Safari/537.36"
******* 71.83.70.172 - - [17/Dec/2024:12:54:55 -0500] "GET /tree/tree/BEST_2010-05-9.ged/media-thumbnail?xref=M885&fact_id=1c998e301640e6a9893014b52483971d&w=400&h=520&fit=contain&mark=0&s=6cdec48576a80d204a745dad2ec650ea HTTP/1.1" 200 32788 "https://www.thebestgenealogy.com/tree/tree/BEST_2010-05-9.ged/individual/I2906/Luther-Kenneth-HUFFSTETLER" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_6) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/14.0.2 Safari/605.1.15"
2a06:98c0:3600::103 - - [17/Dec/2024:12:55:01 -0500] "GET /tree/tree/BEST_2010-05-9.ged/family-book-2-5-0/X1650 HTTP/1.1" 406 34 "-" "Mozilla/5.0 (Linux; Android 6.0.1; Nexus 5X Build/MMB29P) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/131.0.6778.139 Mobile Safari/537.36 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
66.249.75.198 - - [17/Dec/2024:12:55:01 -0500] "GET /tree/tree/BEST_2010-05-9.ged/family-book-2-5-0/X1650 HTTP/1.1" 200 7408 "-" "Mozilla/5.0 (Linux; Android 6.0.1; Nexus 5X Build/MMB29P) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/131.0.6778.139 Mobile Safari/537.36 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
2a06:98c0:3600::103 - - [17/Dec/2024:12:55:09 -0500] "GET /tree/tree/BEST_2010-05-9.ged/family-list?surname=ROTHENBERGER&show=indi&alpha=R&show_marnm=yes HTTP/1.1" 406 34 "-" "Mozilla/5.0 (Linux; Android 6.0.1; Nexus 5X Build/MMB29P) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/131.0.6778.139 Mobile Safari/537.36 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
66.249.75.198 - - [17/Dec/2024:12:55:09 -0500] "GET /tree/tree/BEST_2010-05-9.ged/family-list?surname=ROTHENBERGER&show=indi&alpha=R&show_marnm=yes HTTP/1.1" 200 9985 "-" "Mozilla/5.0 (Linux; Android 6.0.1; Nexus 5X Build/MMB29P) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/131.0.6778.139 Mobile Safari/537.36 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
2a06:98c0:3600::103 - - [17/Dec/2024:12:55:10 -0500] "GET /tree/tree/BEST_2010-05-9.ged/individual-list?show=indi&show_marnm=no&surname=PATILLO&falpha= HTTP/1.1" 406 34 "-" "Mozilla/5.0 (Linux; Android 6.0.1; Nexus 5X Build/MMB29P) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/131.0.6778.139 Mobile Safari/537.36 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
66.249.75.198 - - [17/Dec/2024:12:55:10 -0500] "GET /tree/tree/BEST_2010-05-9.ged/individual-list?show=indi&show_marnm=no&surname=PATILLO&falpha= HTTP/1.1" 200 10109 "-" "Mozilla/5.0 (Linux; Android 6.0.1; Nexus 5X Build/MMB29P) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/131.0.6778.139 Mobile Safari/537.36 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
2a06:98c0:3600::103 - - [17/Dec/2024:12:55:11 -0500] "GET /tree/tree/BEST_2010-05-9.ged/individual-list?show_all=yes&surname=MYERS&falpha=D&show_marnm=no HTTP/1.1" 406 34 "-" "Mozilla/5.0 (Linux; Android 6.0.1; Nexus 5X Build/MMB29P) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/131.0.6778.139 Mobile Safari/537.36 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
66.249.75.198 - - [17/Dec/2024:12:55:11 -0500] "GET /tree/tree/BEST_2010-05-9.ged/individual-list?show_all=yes&surname=MYERS&falpha=D&show_marnm=no HTTP/1.1" 200 80974 "-" "Mozilla/5.0 (Linux; Android 6.0.1; Nexus 5X Build/MMB29P) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/131.0.6778.139 Mobile Safari/537.36 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
2a06:98c0:3600::103 - - [17/Dec/2024:12:55:17 -0500] "GET /tree/tree/BEST_2010-05-9.ged/individual/X4928/John-Claude-BORING HTTP/1.1" 406 34 "-" "Mozilla/5.0 (Linux; Android 6.0.1; Nexus 5X Build/MMB29P) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/131.0.6778.139 Mobile Safari/537.36 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
66.249.75.198 - - [17/Dec/2024:12:55:17 -0500] "GET /tree/tree/BEST_2010-05-9.ged/individual/X4928/John-Claude-BORING HTTP/1.1" 200 14254 "-" "Mozilla/5.0 (Linux; Android 6.0.1; Nexus 5X Build/MMB29P) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/131.0.6778.139 Mobile Safari/537.36 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
The screenshot... I too have no idea what it means. I was hoping someone else might know. It's just what they sent to show the usage.
I would consider my tree to be on the small size: 11,628 Individuals, 2,947 media objects, 30 users (only 1 is active other than me).
Attachment not found
Attachment not found
Last edit: 4 months 1 week ago by vdrumsroland.
Please Log in or Create an account to join the conversation.
- bertkoor
-
- Offline
- Platinum Member
-
- Greetings from Utrecht, Holland
4 months 1 week ago #9
by bertkoor
stamboom.BertKoor.nl runs on webtrees v2.2.1
Replied by bertkoor on topic High Resource Usage
A = Actual
L = Limit.
I saw in the table you posted some days ago that traffic was easy, and then got busy in the last hour. Not super busy, just 1/8th of the limit they put. I looked for that in column "IO".
I personally would not worry too much since your usage of their resources is within the limits they have set. Maybe it is more than the average WordPress/Joomla site they host.
I suspect you got visited by a bot, spider, whatever is scraping content and fetching page after page. Your access log should give more details. Ask them for help, where is the most traffic coming from? Worth blocking?
Or do this test: directly as you wake up, go at your webtrees site and make continuous edits for an hour long. Then look at the hosting usage metrics, compare with an hour before or same hour a day earlier.
L = Limit.
I saw in the table you posted some days ago that traffic was easy, and then got busy in the last hour. Not super busy, just 1/8th of the limit they put. I looked for that in column "IO".
I personally would not worry too much since your usage of their resources is within the limits they have set. Maybe it is more than the average WordPress/Joomla site they host.
I suspect you got visited by a bot, spider, whatever is scraping content and fetching page after page. Your access log should give more details. Ask them for help, where is the most traffic coming from? Worth blocking?
Or do this test: directly as you wake up, go at your webtrees site and make continuous edits for an hour long. Then look at the hosting usage metrics, compare with an hour before or same hour a day earlier.
stamboom.BertKoor.nl runs on webtrees v2.2.1
Please Log in or Create an account to join the conversation.
- fisharebest
-
- Away
- Administrator
-
4 months 1 week ago #10
by fisharebest
Greg Roach - greg@subaqua.co.uk - @fisharebest@phpc.social - fisharebest.webtrees.net
Replied by fisharebest on topic High Resource Usage
Web-servers generally produce two log files.
The access log (which you showed us), and an error log.
The error log could be useful. Do you have these logs?
They could tell us whether PHP didn't respond within a time limit, or responded with an error, etc.
Your access logs show lots of robot requests for the individual/family lists. These lists require a lot of resources, and are the most resource-intensive pages that robots can access.
If you change the access level for these from visitor to member (temporarily), does it make a difference?
I note that you have a lot of third-party modules installed.
If you disable them (temporarily), does it make a difference?
Another thing you could investigate is the database. You could set up another site (in a subfolder?) with the same data/modules/etc., but using SQLite. SQLite is pretty fast for reading, but slow for writes, especially concurrent writes. So if the site is much faster (for a single user), then MySQL might be the culprit.
But overall, the general "feel" of your site is that the server is simply underpowered, requests are being blocked/queued, etc.
The access log (which you showed us), and an error log.
The error log could be useful. Do you have these logs?
They could tell us whether PHP didn't respond within a time limit, or responded with an error, etc.
Your access logs show lots of robot requests for the individual/family lists. These lists require a lot of resources, and are the most resource-intensive pages that robots can access.
If you change the access level for these from visitor to member (temporarily), does it make a difference?
I note that you have a lot of third-party modules installed.
If you disable them (temporarily), does it make a difference?
Another thing you could investigate is the database. You could set up another site (in a subfolder?) with the same data/modules/etc., but using SQLite. SQLite is pretty fast for reading, but slow for writes, especially concurrent writes. So if the site is much faster (for a single user), then MySQL might be the culprit.
But overall, the general "feel" of your site is that the server is simply underpowered, requests are being blocked/queued, etc.
Greg Roach - greg@subaqua.co.uk - @fisharebest@phpc.social - fisharebest.webtrees.net
Please Log in or Create an account to join the conversation.
- Franz Frese
-
- Offline
- Platinum Member
-
4 months 1 week ago #11
by Franz Frese
Replied by Franz Frese on topic High Resource Usage
Please Log in or Create an account to join the conversation.
- bertkoor
-
- Offline
- Platinum Member
-
- Greetings from Utrecht, Holland
4 months 1 week ago - 4 months 1 week ago #12
by bertkoor
which is a bit awkward. The main page of a domain should always respond, no matter what.
Try this link, from his signature.
www.thebestgenealogy.com/tree
stamboom.BertKoor.nl runs on webtrees v2.2.1
Replied by bertkoor on topic High Resource Usage
That page gives 403, and not found, redirects to thebestgenealogy.com/403-page-forbidden/dist/index.htmlI get
"Too many redirects"
for
www.thebestgenealogy.com
which is a bit awkward. The main page of a domain should always respond, no matter what.
Try this link, from his signature.
www.thebestgenealogy.com/tree
stamboom.BertKoor.nl runs on webtrees v2.2.1
Last edit: 4 months 1 week ago by bertkoor.
Please Log in or Create an account to join the conversation.
- vdrumsroland
-
Topic Author
- Offline
- New Member
-
4 months 1 week ago #13
by vdrumsroland
Replied by vdrumsroland on topic High Resource Usage
There is nothing in the error log file on the server.
"requests for the individual/family lists"
Where is this setting? I assume in the Control Panel > Manage tree > Privacy.
I currently have: show family tree & deceased to visitors & members.
I only have two third-party modules running/installed, Fancy Trees & Vesta.
"Too many redirects"
This might be coming from a country block I had installed on my Wordpress, I just removed it. The Webtree is simply linked from the Wordpress site. However, Cloudflare is filtering countries too.
"requests for the individual/family lists"
Where is this setting? I assume in the Control Panel > Manage tree > Privacy.
I currently have: show family tree & deceased to visitors & members.
I only have two third-party modules running/installed, Fancy Trees & Vesta.
"Too many redirects"
This might be coming from a country block I had installed on my Wordpress, I just removed it. The Webtree is simply linked from the Wordpress site. However, Cloudflare is filtering countries too.
Please Log in or Create an account to join the conversation.
- dust2k
-
- Offline
- New Member
-
Less
More
- Posts: 19
4 months 1 week ago #14
by dust2k
jiazu.ausmis.com runs on Webtree 2.2.1
Replied by dust2k on topic High Resource Usage
Finding our server load issues often it's quite difficult for non system admin background people, there are steps but generally you can work out which process causing your CPU/RAM/Disk IO issue, plenty tools you can use, after you found out the process, sometime is your web server, your application server or PHP CGI or database, then you need get into the the process's log to locate the issues...during these steps of investigation, using google or chatgpt can help you lots if you didn't do it before.
jiazu.ausmis.com runs on Webtree 2.2.1
Please Log in or Create an account to join the conversation.
- fisharebest
-
- Away
- Administrator
-
4 months 1 week ago #15
by fisharebest
Greg Roach - greg@subaqua.co.uk - @fisharebest@phpc.social - fisharebest.webtrees.net
Replied by fisharebest on topic High Resource Usage
> Where is this setting? I assume in the Control Panel
These are modules. Set the access in the modules/lists section
These are modules. Set the access in the modules/lists section
Greg Roach - greg@subaqua.co.uk - @fisharebest@phpc.social - fisharebest.webtrees.net
Please Log in or Create an account to join the conversation.
- vdrumsroland
-
Topic Author
- Offline
- New Member
-
2 months 2 weeks ago - 2 months 2 weeks ago #16
by vdrumsroland
Replied by vdrumsroland on topic High Resource Usage
Hey, I’m back with a few more details. Not sure if they’ll help, but my hosting provider keeps pointing me back to Webtrees. Honestly, it’s getting pretty frustrating because I feel like I’m not getting much support beyond them just saying it’s a "resource limit" issue.
They’ve recommended a few things, which I’ve already done—leveraging browser cache, enabling gzip compression, adding the 7G firewall code to my htaccess file, and blocking bad bots with additional rules. They also provided some stats, showing the top IP hits (mine is highest at 2,148, followed by my hosting at 1,235), and the most accessed content, which are the calendar views (day: 19,155, month: 11,283, year: 3,916).
The top bot hits include Bingbot (38,801 hits in a week!) and Googlebot (1,791), with everything else much lower. They keep saying the issue is due to I/O resource limits being hit, which is currently set at just 1MB. The errors I’m running into are 502 & 503, which happen when those limits are exceeded. There’s no real pattern—sometimes just visiting the site triggers an error, and other times it happens while working within the tree.
Interestingly, my main site, thebestgenealogy.com, which runs on WordPress, doesn’t have these issues at all. The problems only affect Webtrees, which is housed in a subfolder (/tree).
Anyway, here are the stats they sent over in case they help figure out what’s going on! I masked my IP for security purposed, just in case. 😊
They’ve recommended a few things, which I’ve already done—leveraging browser cache, enabling gzip compression, adding the 7G firewall code to my htaccess file, and blocking bad bots with additional rules. They also provided some stats, showing the top IP hits (mine is highest at 2,148, followed by my hosting at 1,235), and the most accessed content, which are the calendar views (day: 19,155, month: 11,283, year: 3,916).
The top bot hits include Bingbot (38,801 hits in a week!) and Googlebot (1,791), with everything else much lower. They keep saying the issue is due to I/O resource limits being hit, which is currently set at just 1MB. The errors I’m running into are 502 & 503, which happen when those limits are exceeded. There’s no real pattern—sometimes just visiting the site triggers an error, and other times it happens while working within the tree.
Interestingly, my main site, thebestgenealogy.com, which runs on WordPress, doesn’t have these issues at all. The problems only affect Webtrees, which is housed in a subfolder (/tree).
Anyway, here are the stats they sent over in case they help figure out what’s going on! I masked my IP for security purposed, just in case. 😊
Code:
Below is an excerpt of our resource usage audit logs:
=== General Stats ================================
From 3 February 2025
To 9 February 2025
Period 7 Days
Total Hits 50766
Avg Hits/Day 7252
Avg Hits/Month 220582
Entry Proc Faults 199
Number Proc Faults 0
Virtual Mem Faults 0
Physical Mem Faults 92
CPU Faults 1
IO Faults 512
=== Top Domains ==================================
***** (91%) Addon Domain: thebestgenealogy.com (Hits: 46058 Uniques: 509) *****
--- Hits Per Day ---
3 February 11833 | 4 February 11759 | 5 February 12707
6 February 7162 | 7 February 1451 | 8 February 583
9 February 563
--- Top IPs ---
2148 *.*.*.*
1235 2606:bd00:1234:700:1::3e6
648 157.55.39.195
642 157.55.39.194
570 52.167.144.147
560 52.167.144.22
538 2a06:98c0:3600::103
531 157.55.39.52
517 207.46.13.78
516 52.167.144.19
--- Top Accessed Content/Files ---
19155 /tree/tree/BEST_2010-05-9.ged/calendar/day
11283 /tree/tree/BEST_2010-05-9.ged/calendar/month
3916 /tree/tree/BEST_2010-05-9.ged/calendar/year
996 /wp-cron.php
727 /tree/tree/BEST_2010-05-9.ged/individual-list
621 /tree/tree/BEST_2010-05-9.ged/media-thumbnail
554 /tree/tree/BEST_2010-05-9.ged/family-list
457 [i]/tree/module/[/i]_vesta_personal_facts_/Tab/BEST_2010-05-9.ged
404 [i]/tree/module/[/i]_vesta_personal_facts_/Vestals
382 /wp-admin/admin-ajax.php
--- Top Bot Hits ---
38801 [url]http://www.bing.com/bingbot.htm[/url]) Chrome/116.0.1938.76 Safari/537.36
1791 [url]http://www.google.com/bot.html[/url]
43 [url]https://duckduckgo.com/duckduckbot[/url])'
30 Googlebot-Image/1.0
23 [url]http://www.bing.com/bingbot.htm[/url]) Chrome/100.0.4896.127 Safari/537.36
14 [url]http://www.bing.com/bingbot.htm[/url]) Chrome/112.0.0.0 Safari/537.36
5 Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_1) AppleWebKit/601.2.4 (KHTML, like Gecko) Version/9.0.1 Safari/601.2.4 facebookexternalhit/1.1 Facebot Twitterbot/1.0
4 [url]https://developers.facebook.com/docs/sharing/webmasters/crawler[/url]
3 [url]http://www.pinterest.com/bot.html[/url]
1 [url]https://opensiteexplorer.org/dotbot[/url]; help@moz.com
As you can see, the domain causing the resource usage is thebestgenealogy.com, specifically with the "/tree" website.
Last edit: 2 months 2 weeks ago by vdrumsroland.
Please Log in or Create an account to join the conversation.
- bertkoor
-
- Offline
- Platinum Member
-
- Greetings from Utrecht, Holland
2 months 2 weeks ago #17
by bertkoor
stamboom.BertKoor.nl runs on webtrees v2.2.1
Replied by bertkoor on topic High Resource Usage
Many hits on the calendar? Bots have no business there.
Check your robots.txt.
www.thebestgenealogy.com/tree/robots.txt
It lists a calendar link, but this is not where these pages really are, so bots go wild anyway.
You also have an old webtrees 1.7 robots.txt here:
www.thebestgenealogy.com/tree/robots.txt
If you remove it, then webtrees will serve a generated one (from memory, not from disk)
Fetch with a browser, and merge that with the robots.txt in your domain root.
Check your robots.txt.
www.thebestgenealogy.com/tree/robots.txt
It lists a calendar link, but this is not where these pages really are, so bots go wild anyway.
You also have an old webtrees 1.7 robots.txt here:
www.thebestgenealogy.com/tree/robots.txt
If you remove it, then webtrees will serve a generated one (from memory, not from disk)
Fetch with a browser, and merge that with the robots.txt in your domain root.
stamboom.BertKoor.nl runs on webtrees v2.2.1
Please Log in or Create an account to join the conversation.
- vdrumsroland
-
Topic Author
- Offline
- New Member
-
2 months 2 weeks ago #18
by vdrumsroland
Replied by vdrumsroland on topic High Resource Usage
Thank you so much for your reply and help! I simply renamed it to OLD, refreshed the page, and it looks like the new one populated correctly. I then added it to the main site’s robots.txt file. Now, I just have to wait and see if this helps with the resource limit.
It’s frustrating because, after all these years of using Webtrees, this is the first time I’ve run into this issue. The blame has been placed on my previous hosting companies for not managing limits properly. But with this new host, it seems like they actually enforce strict limits. Whether that’s true or not, I can’t say—that’s just what I’ve been told.
In the past, I was always on an "unlimited" shared hosting package, but now it feels like I’ve been moved to a more restricted server. I just wish I could get a clear answer on what’s really going on! 🤷♂️
It’s frustrating because, after all these years of using Webtrees, this is the first time I’ve run into this issue. The blame has been placed on my previous hosting companies for not managing limits properly. But with this new host, it seems like they actually enforce strict limits. Whether that’s true or not, I can’t say—that’s just what I’ve been told.
In the past, I was always on an "unlimited" shared hosting package, but now it feels like I’ve been moved to a more restricted server. I just wish I could get a clear answer on what’s really going on! 🤷♂️
Please Log in or Create an account to join the conversation.
- bertkoor
-
- Offline
- Platinum Member
-
- Greetings from Utrecht, Holland
2 months 2 weeks ago #19
by bertkoor
stamboom.BertKoor.nl runs on webtrees v2.2.1
Replied by bertkoor on topic High Resource Usage
Typo, the url real bots visitis no other than
www.thebestgenealogy.com/robots.txt
(without "tree")
Also there is a challenge with the sitemap. I don't think you can have it merged from Joomla and webtrees.
Maybe you can make a subdomain tree.yourdomain.net, then it stands on its own.
Also there is a challenge with the sitemap. I don't think you can have it merged from Joomla and webtrees.
Maybe you can make a subdomain tree.yourdomain.net, then it stands on its own.
stamboom.BertKoor.nl runs on webtrees v2.2.1
Please Log in or Create an account to join the conversation.
- vdrumsroland
-
Topic Author
- Offline
- New Member
-
2 months 2 weeks ago #20
by vdrumsroland
Replied by vdrumsroland on topic High Resource Usage
Since I’ve never manually moved my tree to a different directory or site before, I wanted to check—what’s the best way to do this? The only instructions I’ve found are for moving webtrees from one server to another (webtrees.net/faq/move-server/), but not for just moving it within the same site.
Right now, my data folder path is written in full. Would I just need to change that to a relative path since it’s staying within the same site? I just want to make sure I don’t mess anything up! 😅
Right now, my data folder path is written in full. Would I just need to change that to a relative path since it’s staying within the same site? I just want to make sure I don’t mess anything up! 😅
Please Log in or Create an account to join the conversation.