Web based family history software

Question cURL error when upgrading to new version 2.0.4

  • sleebooth
  • Topic Author
  • Offline
  • New Member
  • New Member
More
3 years 10 months ago #1 by sleebooth
I'm getting:

"Download github.com/fisharebest/webtrees/releases.../webtrees-2.0.4.zip…

cURL error 28: Operation timed out after 55000 milliseconds with 11190786 out of 30263235 bytes received (see curl.haxx.se/libcurl/c/libcurl-errors.html )"


when I try to upgrade to webtrees 2.04. (I have had the same error the last two upgrades but did manual upgrades.

When I asked my ISP if they could anything, they said,

The error regarding cURL is generated due to loop back connections to the same server being disabled on our shared hosting platform. The reason for this is that a singe badly written script could cause an infinite amount of connections back to the same server, which would be detrimental for the platform.

These connections cannot be enabled on request basis I'm afraid.

In this case, you might need to manually update the software or check with the developer if there is no workaround for an automated update that does not require a loop back connection.


Is there anything you can do, Greg, or am I doomed forever to do manual updates?

John's SLEE/BOOTH Family Tree
sleebooth.co.uk
webtrees v2.0.19
PHP Version 7.4.28

Please Log in or Create an account to join the conversation.

More
3 years 10 months ago #2 by fisharebest
Replied by fisharebest on topic cURL error when upgrading to new version 2.0.4
> that does not require a loop back connection.

I do not understand what they are saying.

The only thing I know called a "loop back connection" is IP address 127.0.0.1 - which always refers to the current server.

We are making an HTTP request to a server in an amazon data center. I don't see how the two are connected…

Google searches for "loop back connection site:haxx.se" or "loop back connection curl" do not provide any answers.

If you can find anything, let me know.

Greg Roach - greg@subaqua.co.uk - @fisharebest@phpc.social - fisharebest.webtrees.net

Please Log in or Create an account to join the conversation.

More
3 years 10 months ago - 3 years 10 months ago #3 by eparlak
Sometimes download from github is realy slow. Usually after 2 or 3 tries can be downloaded successfully.
If you always get this error you can try upload files to local, then download from there. For this you need to change some data in database.

Make backup before start

First you need to upload webtrees-2.0.4.zip to your website's public folder or some other websites you think it will be fast.
For database you need to change "wt_site_setting/LATEST_WT_VERSION" from;
Code:
2.0.4|2.0.0|https://github.com/fisharebest/webtrees/releases/download/2.0.4/webtrees-2.0.4.zip
to
Code:
2.0.4|2.0.0|http://localhost/public/webtrees-2.0.4.zip
Change localhost to your domain

I tried with fresh 2.0.3 and it worked.

Last edit: 3 years 10 months ago by eparlak.

Please Log in or Create an account to join the conversation.

  • sleebooth
  • Topic Author
  • Offline
  • New Member
  • New Member
More
3 years 10 months ago #4 by sleebooth
Replied by sleebooth on topic cURL error when upgrading to new version 2.0.4
You said you couldn't see this being anything to do with loopback on the server.

However, I was trying to run the upgrade using Chrome under Windows (and I have several tabs open - I'm not sure if this is connected to the issue). After checking the devtools output I noted the "Stalled issue". I found
developers.google.com/web/tools/chrome-d...e#timing-explanation

For some reason that made me rerun the upgrade script on Microsoft Edge with only the one tab open .It appears to have run successfully. Can you explain why this might be?

However, there are several anomalies (bugs) now appearing in the webtrees 2.04 output:
  1. The shadow figure icon for an individual is not visible in Edge:

    while it is in Chrome:

  • The search page has wrongly sized gender(?) icons:
  • The "Changes in the last 30 days" list refers incorrectly to families thus showing blanks:

    e.g. The first one links to sleebooth.co.uk/tree/sleebooth.ged/family/X172/-

    whereas it should link to sleebooth.co.uk/tree/sleebooth.ged/sourc...er-Honeychurch-Devon


  • Are these replicated on your sites or has the upgrade failed?

    John's SLEE/BOOTH Family Tree
    sleebooth.co.uk
    webtrees v2.0.19
    PHP Version 7.4.28

    Please Log in or Create an account to join the conversation.

    • bertkoor
    • Away
    • Platinum Member
    • Platinum Member
    • Greetings from Utrecht, Holland
    More
    3 years 10 months ago #5 by bertkoor
    Replied by bertkoor on topic cURL error when upgrading to new version 2.0.4
    Regarding the giant icons, that's already reported:
    github.com/fisharebest/webtrees/issues/3265

    stamboom.BertKoor.nl runs on webtrees v1.7.13

    Please Log in or Create an account to join the conversation.

    More
    3 years 10 months ago #6 by fisharebest
    Replied by fisharebest on topic cURL error when upgrading to new version 2.0.4
    > The "Changes in the last 30 days" list refers incorrectly to families thus showing blanks:

    Already reported and fixed: github.com/fisharebest/webtrees/issues/3264

    > The search page has wrongly sized gender(?) icons

    Already reported and fixed: github.com/fisharebest/webtrees/issues/3265

    > However, I was trying to run the upgrade using Chrome under Windows

    I guess that's possible, but it would seem unlikely (as we are waiting for a server process to finish).

    Difficult to investigate this, as I don't have access to any Windows machines.

    If you want to test this, you can edit the file app/webtrees.php and change "2.0.4" to a lower version (e.g. 2.0.3)

    github.com/fisharebest/webtrees/blob/2.0...pp/webtrees.php#L101

    This will trigger the upgrade to run again. However, it won't matter if it fails or suceeds - because you have already upgraded!
    I use this "trick" when testing/developing the upgrade script.

    Greg Roach - greg@subaqua.co.uk - @fisharebest@phpc.social - fisharebest.webtrees.net

    Please Log in or Create an account to join the conversation.

    • sleebooth
    • Topic Author
    • Offline
    • New Member
    • New Member
    More
    3 years 10 months ago #7 by sleebooth
    Replied by sleebooth on topic cURL error when upgrading to new version 2.0.4
    I have investigated this as you suggested.

    I thought it might have been an extension in Chrome that was causing the problem so switched them all off.

    The upgrade still would not run. The only way I could consistently succeed in running the upgrade to completion, even with all my extensions enabled, was to run it in the only open tab. I tried opening different urls in second tabs but all of them prevented the upgrade completing. Weird!

    Still at least I know what I need to do to successfully upgrade. Run it in the only open tab in the browser.

    John's SLEE/BOOTH Family Tree
    sleebooth.co.uk
    webtrees v2.0.19
    PHP Version 7.4.28

    Please Log in or Create an account to join the conversation.

    More
    3 years 10 months ago #8 by fisharebest
    Replied by fisharebest on topic cURL error when upgrading to new version 2.0.4
    > Weird!

    Agreed. I can't see why this might be connected, but if it works for you...

    Greg Roach - greg@subaqua.co.uk - @fisharebest@phpc.social - fisharebest.webtrees.net

    Please Log in or Create an account to join the conversation.

    • sleebooth
    • Topic Author
    • Offline
    • New Member
    • New Member
    More
    3 years 10 months ago #9 by sleebooth
    Replied by sleebooth on topic cURL error when upgrading to new version 2.0.4
    And thanks for the bug fixes.

    I'm trying to learn how to use GitHub with webtrees. I have cloned fisharebest/webtrees using GitHub Desktop.

    If you fix an issue, how do I download the changes onto my repository? Will GitHub Desktop tell me there is a change? I'm finding the terms Push/Pull/Fetch confusing. Do I just "Fetch origin"?

    John's SLEE/BOOTH Family Tree
    sleebooth.co.uk
    webtrees v2.0.19
    PHP Version 7.4.28

    Please Log in or Create an account to join the conversation.

    More
    3 years 10 months ago #10 by fisharebest
    Replied by fisharebest on topic cURL error when upgrading to new version 2.0.4
    I have never used github desktop. I always use command-line tools.

    Here is the official documentation for keeping your fork up-to-date:

    help.github.com/en/github/collaborating-...uests/syncing-a-fork

    The only thing I would suggest is to use "git rebase upstream/master" instead of "git merge upstream/master".

    "rebase" reapplies you local modifications "on top" of the latest code, rather than merging them into it.


    So, your local repository will be linked to two remote repositories at github.
    One is your copy (sleebooth/webtrees) - usually called "origin",
    The other is my copy (fisharebest/webtrees) - usually called "upstream".

    I make commits to my local copy, and push them to fisharebest/webtrees.

    You then need to pull (=fetch and merge) or rebase (=fetch and re-apply) my commits to your local copy.

    You can then push your local changes (and my new commits) to your repo on github. Actually, you don't have to do this, but (a) it makes a good backup, and (b), you can share it with other people - for example to show me your changes and ask questions about them.

    Greg Roach - greg@subaqua.co.uk - @fisharebest@phpc.social - fisharebest.webtrees.net

    Please Log in or Create an account to join the conversation.

    • sleebooth
    • Topic Author
    • Offline
    • New Member
    • New Member
    More
    3 years 10 months ago #11 by sleebooth
    Replied by sleebooth on topic cURL error when upgrading to new version 2.0.4
    thanks for that help.

    Just to complete this thread, my ISP (Heart Internet) came back with:

    "Apologies, I misread the error which is a timeout one and not a loop back connection error.

    The timeout could happen if the size of the update is too big of if the network connection is not fast enough. This could have happened because as you mentioned you had multiple tabs opened which caused the timeout for the script."

    John's SLEE/BOOTH Family Tree
    sleebooth.co.uk
    webtrees v2.0.19
    PHP Version 7.4.28

    Please Log in or Create an account to join the conversation.

    More
    3 years 10 months ago - 3 years 10 months ago #12 by Novice
    I am also getting the same error, downloading only about half of the data before hitting a timeout.
    Could someone please help and either post a more detailed/dumbed down/step by step guide to eparlak's solution above, or potentially better yet how to increase the curl timeout so it just takes a bit longer and downloads the whole thing?

    Is 55000 milliseconds an arbitrary thing coded into webtrees? Is it just a db/config file entry somewhere that can be changed. Seems like an interesting design choice.
    Last edit: 3 years 10 months ago by Novice. Reason: Asking about 5500 miliseconds

    Please Log in or Create an account to join the conversation.

    More
    3 years 10 months ago - 3 years 10 months ago #13 by eparlak
    I find another solution which increase curl timeout

    1. Open vendor/guzzlehttp/guzzle/src/Handler/CurlFactory.php
    2. Change
    Code:
    $conf[CURLOPT_TIMEOUT_MS] = $options['timeout'] * 1000;
    to
    Code:
    $conf[CURLOPT_TIMEOUT_MS] = $options['timeout'] * 0;
    3. Change
    Code:
    $conf[CURLOPT_CONNECTTIMEOUT_MS] = $options['connect_timeout'] * 1000;
    to
    Code:
    $conf[CURLOPT_CONNECTTIMEOUT_MS] = $options['connect_timeout'] * 0;

    It may not work btw. When i decrease the timeout i was getting error. But without changing any file i was able to upgrade. So you need to try and see what happens.
    Last edit: 3 years 10 months ago by eparlak.

    Please Log in or Create an account to join the conversation.

    More
    3 years 10 months ago #14 by Novice
    Thank you,

    That's looking like an improvement, but now I am getting
    504 Gateway Time-out
    Instead

    Please Log in or Create an account to join the conversation.

    More
    3 years 10 months ago #15 by fisharebest
    Replied by fisharebest on topic cURL error when upgrading to new version 2.0.4
    > That's looking like an improvement, but now I am getting 504 Gateway Time-out

    There are lots of processes, and each has its own timeout.

    By default, most webservers (e.g. apache, nginx, iis) have a 60 second timeout.
    They start a PHP process, and wait 60 seconds for a result.
    If there is no response from PHP in 60 seconds, it sends a "Gateway timeout" error.

    For this reason, webtrees sets a 55 second timeout in the cURL parameters.
    FYI, here is the line that does it.
    github.com/fisharebest/webtrees/blob/2.0...gradeService.php#L49

    If we can't fetch the file in 55 seconds, there is no point continuing, as the webserver will discard whatever we do.

    So if you want to increase timeouts (or your PHP execution time limit) beyound 60 seconds, you will need to update your webserver's "gateway timeout" as well.

    Unless you run your own server or VPS, you probably won't be able to do this.

    Greg Roach - greg@subaqua.co.uk - @fisharebest@phpc.social - fisharebest.webtrees.net

    Please Log in or Create an account to join the conversation.

    More
    3 years 10 months ago - 3 years 10 months ago #16 by Novice
    So given that this is a fairly big file, why do it this way?
    Why not use a regular HTTP download, or split the file into several and download them one at a time?

    So the best way to proceed, is download it manually, load it onto a server, and change the pointers in the database to grab it locally?

    Maybe in a future update there could be an inclusion of a "browse" box that lets you upload a .zip file from local storage if the automated one fails, it'll point you to where to get it, then ask for an upload?

    Alternatively, I am using nginx, where could I change the server timeout?
    Last edit: 3 years 10 months ago by Novice.

    Please Log in or Create an account to join the conversation.

    More
    3 years 10 months ago #17 by Novice
    Following the instructions
    stackoverflow.com/questions/24453388/ngi...-504-gateway-timeout
    I added
    fastcgi_read_timeout 600;
    proxy_connect_timeout 600;
    proxy_send_timeout 600;
    proxy_read_timeout 600;
    send_timeout 600;

    To
    location ~ \.php$ {
    Block of the site.
    It now takes longer before getting a new error:

    "The server’s time limit has been reached."

    Please Log in or Create an account to join the conversation.

    More
    3 years 10 months ago #18 by Novice
    Ok, for anyone else having this problem, the solution is to follow the instructions here:


    1. Open vendor/guzzlehttp/guzzle/src/Handler/CurlFactory.php
    2. Change

    $conf[CURLOPT_TIMEOUT_MS] = $options * 1000;

    to

    $conf[CURLOPT_TIMEOUT_MS] = $options * 0;

    3. Change

    $conf[CURLOPT_CONNECTTIMEOUT_MS] = $options * 1000;

    to

    $conf[CURLOPT_CONNECTTIMEOUT_MS] = $options * 0;

    Followed by:
    www.scalescale.com/tips/nginx/504-gateway-time-out-using-nginx/

    Please Log in or Create an account to join the conversation.

    More
    3 years 10 months ago #19 by fisharebest
    Replied by fisharebest on topic cURL error when upgrading to new version 2.0.4
    > So given that this is a fairly big file, why do it this way?

    30MB is a relatively small file. It is being fetched from a content-delivery service (hosted at Amazon) directly to your server.

    The original error message reported that it downloaded 11MB in 55 seconds.
    This is a direct link between two data-centres that are (relatively) close geographically.
    200KB/second? You should expect 100x that.

    The question should be "why so slow", rather than "why not wait longer".

    > It now takes longer before getting a new error:
    > "The server’s time limit has been reached."

    This looks like a webtrees error message. It is usually triggered a couple of seconds before reaching your PHP execution time limit. You increase this in your PHP.INI

    Greg Roach - greg@subaqua.co.uk - @fisharebest@phpc.social - fisharebest.webtrees.net

    Please Log in or Create an account to join the conversation.

    More
    3 years 10 months ago #20 by Novice
    So I made the changes above, and everything worked.
    Went to revert after it all worked, and realized that I was the dumb who forgot to note the original values :/
    Does
    php.ini
    max_execution_time = 60

    www.conf
    ;request_terminate_timeout = 30
    *note the ; preceding the line

    Seem like they are reasonable defaults for me to use? I'm pretty sure that's what they originally were.

    Please Log in or Create an account to join the conversation.

    Powered by Kunena Forum
    }