This Help forum is for issues relates to the latest release (1.0). For issues related to beta or svn version please use their own Help forum.
When requesting help please provide as much information as possible. Explain what version of webtrees, PHP and MYSQL you are using. If possible provide a URL to your site so we can see the problem first-hand.
Tip: Think about putting these details in your signature, so it appears in the footer of ALL your messages
  • Page:
  • 1

TOPIC:

Performance questions 10 years 2 months ago #1

  • Todd Minear
  • Todd Minear's Avatar Topic Author
  • Visitor
  • Visitor
Hello,

First - thanks for providing such great software for free!

I have started a new site and I have a few questions please:

* So far, with ~1,300 individuals... the 'Interactive chart' loads a bit slow to my liking; and I'm wondering what the best way to speed this up is? I host my own site and I believe I have plenty of resources (PHP memory limit is at 256M, I don't know if anything else to check regarding performance). I tried looking at a Resource Monitor but am unable to determine if there is a bottleneck in RAM, CPU, Disk speed, or internet bandwidth.
I also realize that perhaps the 'Interactive Chart' is simply too demanding and will always be slow displaying a lot of individuals. It would still be good to know what resource(s) impact it the most.

* My vision for my site is to go public, is there a built-in/recommended way to provide new visitors with info about the site? (Info that might be a page or two, and seems more substantial than the 'Welcome' message or the 'News' section).
It's not a performance question... but I'm curious about recommendations. Create a static web page and somehow attach the
link in webtrees?


Regards,
Todd

Please Log in or Create an account to join the conversation.

Re: Performance questions 10 years 2 months ago #2

  • ToyGuy
  • ToyGuy's Avatar
  • Offline
  • Moderator
  • Moderator
  • Live like it's Christmas every day - Santa Stephen
  • Posts: 4925
Todd
I'm unable to give you specifics about your site as I can't see it (you've provided no link). Additionally, you have not yet set up a signature to provide us with details about your system - something we strongly suggest at the beginning of these forums.

1) So - to answer more in general, on a properly configured and adequate system, I see no lack of speed or responsiveness in the 'interactive tree' (I know of no interactive 'chart' feature, although it falls under the chart menu icon). On my 89,100+ GEDCOM (larger than yours) here's a link to my GGGF's tree:
John Powell's interactive tree

2) You may wish to visit our home page and our FAQ page to see some examples of enhanced ways to 'invite' cousins to join the site. No need to use the default text, or our text, but rather be creative and use either the NEWS block or an HTML block to change your welcome to suit your desired needs, perhaps even providing a link to a much more robust set of pages, if you believe it necessary. It is our experience that no one reads all this stuff anyway. On our e-com site, they scan for a phone number and call, rather than read the volume of information we provide online to answer all their questions. Same seems true of the gene site.

Welcome page for MyArnolds webtrees site
MyArnolds genealogy FAQ page

Hope all this helps.
Santa Stephen the Fabled Santa
Latest webtrees at MyArnolds.com
Hosted by webtreesonline.com , a division of GeneHosts LLC
MacOS 10.6.8, Apache 2.2+, PHP 5.4.16, MySQL 5.5.28

Please Log in or Create an account to join the conversation.

Re: Performance questions 10 years 2 months ago #3

  • Todd Minear
  • Todd Minear's Avatar Topic Author
  • Visitor
  • Visitor
Thanks Stephen for the response.

My site is currently here

I wonder if perhaps I've run into a bug? If I click on the link to your interactive tree, it fairly quickly
displays the tree. However, if I then click on a different chart and then click back to the Interactive Tree it
loads about half-way and the pauses for a bit or even fails to completely load the tree.

I get this behavior from both IE8 and Chrome ver.9


Regards,
Todd

Please Log in or Create an account to join the conversation.

Re: Performance questions 10 years 2 months ago #4

  • ToyGuy
  • ToyGuy's Avatar
  • Offline
  • Moderator
  • Moderator
  • Live like it's Christmas every day - Santa Stephen
  • Posts: 4925
Todd
Your site loads very slowly - any/every page. I would say it is running on either very slow processors or has significant bandwidth connection issues. I'm trying via Safari and FF and while you don't post the stats in the footer, it is slooooowwwww. This would have a dramatic effect on the interactive tree chart as it is one of the most processor intensive functions as it requires an enormous number of calculations and SQL access calls.

You probably changed people (to my default I183) when you selected a different chart. There is a problem in this line that creates the issue you describe with the interactive tree. Not a bug, but rather a GEDCOM error that I've not yet been able to identify. Personally, I never use the IATC, but I know some do.
Santa Stephen the Fabled Santa
Latest webtrees at MyArnolds.com
Hosted by webtreesonline.com , a division of GeneHosts LLC
MacOS 10.6.8, Apache 2.2+, PHP 5.4.16, MySQL 5.5.28

Please Log in or Create an account to join the conversation.

Re: Performance questions 10 years 2 months ago #5

  • Todd Minear
  • Todd Minear's Avatar Topic Author
  • Visitor
  • Visitor
I'm fairly certain it's not a connection bandwidth issue, as the performance seems about the same even while
working at the host itself.

It is an old processor, but still a Dual Core 2.3Ghz dual core... for ~1,300 individuals. (BTW - you mention "stats in the footer"; are you referring to number of individuals in GEDCOM? I really have looked through the FAQ's and forums, but no... I have not
scrutinized each post... is there a benchmark to collect stats?).

Would problems in the GEDCOM cause this? I don't mind upgrading my server but would like more evidence that it's the
bottleneck first. I did run gedcheck but it seems to fail (perhaps an access problem?)


GEDCHECK
*****************************
ERROR 2: fopen(-) [function.fopen]: failed to open stream: No such file or directory
0 Error occurred on in function fopen
1 called from line 771 of file gedcheck.php

Warning: fopen(-) [function.fopen]: failed to open stream: No such file or directory in /var/www/wordpress/webtrees/gedcheck.php on line 771

ERROR 2: fgets(): supplied argument is not a valid stream resource
0 Error occurred on in function fgets
1 called from line 772 of file gedcheck.php

Warning: fgets(): supplied argument is not a valid stream resource in /var/www/wordpress/webtrees/gedcheck.php on line 772

ERROR 2: fclose(): supplied argument is not a valid stream resource
0 Error occurred on in function fclose
1 called from line 790 of file gedcheck.php

Warning: fclose(): supplied argument is not a valid stream resource in /var/www/wordpress/webtrees/gedcheck.php on line 790


ERROR 8: Undefined variable: gedfile
0 Error occurred on line 798 of file gedcheck.php

Notice: Undefined variable: gedfile in /var/www/wordpress/webtrees/gedcheck.php on line 798

ERROR 8: Undefined variable: gedfile
0 Error occurred on line 802 of file gedcheck.php

Notice: Undefined variable: gedfile in /var/www/wordpress/webtrees/gedcheck.php on line 802

ERROR 2: Invalid argument supplied for foreach()
0 Error occurred on line 802 of file gedcheck.php

Warning: Invalid argument supplied for foreach() in /var/www/wordpress/webtrees/gedcheck.php on line 802
No errors found at this level.
*****************************

Please Log in or Create an account to join the conversation.

Re: Performance questions 10 years 2 months ago #6

  • fisharebest
  • fisharebest's Avatar
  • Offline
  • Administrator
  • Administrator
  • Posts: 14515
<<BTW - you mention "stats in the footer">>

This is a per-gedcom configuration option. Select the "show execution statistics". It will add a line in the footer showing the CPU and memory usage, number of DB queries, etc.

A bit like the "time to create page" footer on this forum.

<<GEDCHECK>>

This is an old (PGV) utility. It works entirely from files on disk.

webtrees is a database application - it does not save data to disk.

BUT, from the manage gedcoms page, there is an "export to disk" option. Use this, and then gedcheck will work from the file on disk (in your /data/ directory).
Greg Roach - This email address is being protected from spambots. You need JavaScript enabled to view it. - fisharebest.webtrees.net

Please Log in or Create an account to join the conversation.

Re: Performance questions 10 years 2 months ago #7

  • ToyGuy
  • ToyGuy's Avatar
  • Offline
  • Moderator
  • Moderator
  • Live like it's Christmas every day - Santa Stephen
  • Posts: 4925
As a database application, a great deal of the performance (outside of bandwidth issues) has to do with the configuration of your MySQL database. Is the collation correct? Are you using INNO-DB or MyISAM engine? Is the configuration optimized? There are many techniques and settings for the my.cnf configuration for your DB that may assist, but usually one of the several default versions (my-large or my-big) should be good. Also, checki out this blog for hints: www.mysqlperformanceblog.com/
or Google MySQL database optimization to see what you can do at your end. Greg is working wonders on the schema and indexing end to maximize responsiveness.

Finally, you say bandwidth is not an issue, but then it may be access. A traceroute to your server was VERY slow, and balked (choked) at: gig15-0.rlghncg-bsr1.nc.rr.com (24.25.0.85) 38.619 ms 38.507 ms 40.235 ms
Looks like you are running freeDNS on a dynamic IP. This is OK, but will never be as fast as a fixed IP DNS.
Santa Stephen the Fabled Santa
Latest webtrees at MyArnolds.com
Hosted by webtreesonline.com , a division of GeneHosts LLC
MacOS 10.6.8, Apache 2.2+, PHP 5.4.16, MySQL 5.5.28

Please Log in or Create an account to join the conversation.

Re: Performance questions 10 years 2 months ago #8

  • Todd Minear
  • Todd Minear's Avatar Topic Author
  • Visitor
  • Visitor
Ok - Thanks.

I've got the stats enabled now... will make it much easier to compare performance.

The GEDCHECK showed no Critical errors, and no Errors, and a good amount of Warnings (I'm assuming those aren't so important).

I havent done anything with MySQL... I had to check to find out that it's using INNO-DB. I loaded webtrees on a Linux
server meant for WordPress (Wordpress site has no traffic on it). I'll don't know much about optimizing DB so I'll read through
that link when I get a chance.

I can look at bandwidth again to be sure. I don't think (or see how) the Dynamic IP would be much of an issue... the client should cache the IP address upon first lookup; and I don't mind making users wait 40ms for the first lookup. (Is my logic wrong there?)


Thanks very much Stephen for helping me along with this.


Regards,
Todd

Please Log in or Create an account to join the conversation.

Do you need a web hosting solution for your webtrees site?
If you prefer a host that specialises in webtrees, the following page lists some suppliers able to provide one for you: 

Re: Performance questions 10 years 2 months ago #9

  • ToyGuy
  • ToyGuy's Avatar
  • Offline
  • Moderator
  • Moderator
  • Live like it's Christmas every day - Santa Stephen
  • Posts: 4925
No problem - just ask, but it appears you are running the web server in a Virtual Machine. Performance on from those processors will never come close to a direct operating system. Just don't expect the kind of results you see on Greg's, Nigel's or my sites where we are hosted directly via DNS on direct or limited sharing machines with substantial bandwidth and speedy processors.
Santa Stephen the Fabled Santa
Latest webtrees at MyArnolds.com
Hosted by webtreesonline.com , a division of GeneHosts LLC
MacOS 10.6.8, Apache 2.2+, PHP 5.4.16, MySQL 5.5.28

Please Log in or Create an account to join the conversation.

Re: Performance questions 10 years 2 months ago #10

  • Todd Minear
  • Todd Minear's Avatar Topic Author
  • Visitor
  • Visitor
Ok.. things are running much better now & thought I'd just post a follow-up.

Virtual Machine: Just wanted to comment that I actually do believe the VM will come close to a direct OS; in my case I use Virtualization for isolation rather than resource sharing; these days Virtualization is very good about use of resources (with the biggest issue being disk I/O... but in my own case, my databases are small and my caches/buffers are larger than entire databases).
I don't have too much control over bandwidth; CPU upgrade is already planned in a few Months. :-)

Optimization... I must say I'm surprised that Mysql comes so poorly optimized 'out of the box'; I had mistakenly made the assumption (I know what they say about assumptions...) that upon
installation my.cnf would be somewhat optimized based on current RAM, etc... In fact it's set to low levels.

So for those who read this thread, if you've done a new install to run webtrees on LAMP (Linux Apache Mysql PHP) then I recommend performing optimizations. The Execution time for the Interactive tree has gone from ~17 seconds to ~5 seconds for me.

Stephen: I do have another follow-up question if your up for it: Did the webtrees install script "pick" the InnoDB type or is this chosen based on my defaults for MySQL? (The obvious next question being: for webtrees, is there a preference to DB type?).


Thanks,
Todd

Please Log in or Create an account to join the conversation.

Re: Performance questions 10 years 2 months ago #11

  • fisharebest
  • fisharebest's Avatar
  • Offline
  • Administrator
  • Administrator
  • Posts: 14515
<<Did the webtrees install script "pick" the InnoDB type or is this chosen based on my defaults for MySQL?>>

The setup script explicitly picks InnoDB, but will fall-back to MyISAM if InnoDB is not available.

You can set the default engine in my.cnf, but we don't rely on the defaults.

<<is there a preference to DB type>>

InnoDB is much better. Row-level locks, referential integrity, performance, etc.

IIRC, from MySQL5.5, the default will change from MyISAM to InnoDB.
Greg Roach - This email address is being protected from spambots. You need JavaScript enabled to view it. - fisharebest.webtrees.net

Please Log in or Create an account to join the conversation.

Re: Performance questions 10 years 2 months ago #12

  • ToyGuy
  • ToyGuy's Avatar
  • Offline
  • Moderator
  • Moderator
  • Live like it's Christmas every day - Santa Stephen
  • Posts: 4925
@Greg
Yes, the default did change (for MySQL) with the release of 5.5.
v5.5.9 is out a few days ago and fixes a few niggles they had with earlier versions. That said, IMHO, it does not appear to have provided any speed benefits, but it is no laggard.

Todd, as Greg has said, no 'forcing the issue' by webtrees, but it/he and I prefer InnoDB for many reasons and it is now MySQL's default. Additionally, you'll be surprised by what you can do for even more tweaks, but much depends on the memory available to the DB. Mine runs almost entirely in memory due to the extra available and that speeds processing even further after defining the my.cnf file to support it. Remember, I've got 88,000 indis+, so a few functions still are slower than I expect or would prefer. For instance, I can generate a list via header general search much faster than I can do so via a INDI list command. Must attack the data differently, so there are improvements yet to make.
Santa Stephen the Fabled Santa
Latest webtrees at MyArnolds.com
Hosted by webtreesonline.com , a division of GeneHosts LLC
MacOS 10.6.8, Apache 2.2+, PHP 5.4.16, MySQL 5.5.28

Please Log in or Create an account to join the conversation.

Re: Performance questions 10 years 2 months ago #13

  • Todd Minear
  • Todd Minear's Avatar Topic Author
  • Visitor
  • Visitor
Aye, I've got excess RAM right now in relation to the size of my database.... I have innodb_buffer_pool_size set to 768MB and
my entire database is smaller than that. I expect my DB to grow soon though.

Any thoughts/comments regarding Indexing? (are there any worth adding?)


BR,
Todd

Please Log in or Create an account to join the conversation.

Re: Performance questions 10 years 2 months ago #14

  • fisharebest
  • fisharebest's Avatar
  • Offline
  • Administrator
  • Administrator
  • Posts: 14515
I wrote some notes on MySQL parameters for the WIKI, and these settings have served me well.

You didn't mention the query cache, but if you don't have one, you should.

As for indexes, we're looking at these. We're finding that queries that should work in theory, actually make some queries slower. Also, MySQL5.0 and 5.1 handle indexes differently, so we don't want to go round adding indexes at random. We wanted to do this for 1.1.0, but ran out of time, so will look again for 1.1.1

Having unnecessary indexes will slow down the query analyser, as there are more for it to consider on every query.
Greg Roach - This email address is being protected from spambots. You need JavaScript enabled to view it. - fisharebest.webtrees.net

Please Log in or Create an account to join the conversation.

Re: Performance questions 10 years 2 months ago #15

  • Todd Minear
  • Todd Minear's Avatar Topic Author
  • Visitor
  • Visitor
I did have the query cache, but I bumped it higher based on the Wiki.

I'm pretty happy with performance now - Thanks Fellas!


BR,
Todd


PS... very much looking forward to ver 1.1

Please Log in or Create an account to join the conversation.

  • Page:
  • 1
Powered by Kunena Forum