This Help forum is for issues relates to the latest release (1.7.9). For issues related to beta or github version please use their own Help forum.
Before asking for help please read "How to request help" by clicking on that tab above here."

TOPIC: Can home computer be a synchronized mirror?

Can home computer be a synchronized mirror? 9 months 4 weeks ago #1

  • WGroleau
  • WGroleau's Avatar
  • Offline
  • Gold
  • Posts: 1498
At the moment, no one is editing my site but me, so if I work off-line on my laptop, its trivial to upload changes when I regain internet access.

However, it would be nice if it were possible to have MySQL keep the databases in sync with replication when on line and to play catch up when reconnected after some time offline. (As opposed to manually re-importing the GEDCOM.)

Can this be done? And if so, would it break down if/when another account is added who's able to edit?
--
Wes Groleau
UniGen.us/
PHP 7.1.13; MySQL 5.6.38; Apache
The administrator has disabled public write access.

Can home computer be a synchronized mirror? 9 months 3 weeks ago #2

  • fisharebest
  • fisharebest's Avatar
  • Online
  • Administrator
  • Posts: 10569
webtrees updates many tables, even when "read-only". For example, the session table, the hit_counter table, etc.

For this reason, database synchronisation probably won't work. MySQL "master-slave" replication assumes all writes go to the master database.
Greg Roach - This email address is being protected from spambots. You need JavaScript enabled to view it. - fisharebest.webtrees.net
The administrator has disabled public write access.

Can home computer be a synchronized mirror? 9 months 3 weeks ago #3

  • janth
  • janth's Avatar
  • Offline
  • New
  • Posts: 29
You also have the problem with all the media files, the references of which are maintained in the database but the actual media are stored in the local filesystem. But you could potentially do periodic backups of the database and media files and restore them on the other host. If you are interested I can offer scripts to do that from my Debian Linux/Apache2.4/PPH7.0/MySQL platform. I use them for a nightly backup to my dropbox drive and have used them quite easily to restore after updating the OS and other software.

Jan
The administrator has disabled public write access.

Can home computer be a synchronized mirror? 9 months 3 weeks ago #4

  • WGroleau
  • WGroleau's Avatar
  • Offline
  • Gold
  • Posts: 1498
Media isn’t a problem; it’s easy enough to keep directory structures in sync. And periodic backups of both machines are already happening. But as I said, I’d like to keep working when no internet connection is available, and not cuss myself out for forgetting to run a script before or after each login.
--
Wes Groleau
UniGen.us/
PHP 7.1.13; MySQL 5.6.38; Apache
Last Edit: 9 months 3 weeks ago by WGroleau.
The administrator has disabled public write access.

Can home computer be a synchronized mirror? 9 months 3 weeks ago #5

  • mp
  • mp's Avatar
  • Offline
  • Junior
  • Posts: 204
My suggestion is to work with a single instance of webtrees online and with a gedcom viewer on your tablet/notebook/phone offline. For changes I would write a note by hand and insert it later to the datebase online. It seems to be easier than to syncronize a mysql database automatically.
Martin - ffp.bauschaffen.de
The administrator has disabled public write access.

Can home computer be a synchronized mirror? 9 months 3 weeks ago #6

  • WGroleau
  • WGroleau's Avatar
  • Offline
  • Gold
  • Posts: 1498
The best GEDCOM viewer I know of is webtrees. Second best is one I wrote for myself ages ago and abandoned it when I discovered webtrees's predecessor, PGV. Maybe PGV is second best, and mine was third. Third for me—others would not care for editing actual GEDCOM enhanced with comments and hyperlinks.
--
Wes Groleau
UniGen.us/
PHP 7.1.13; MySQL 5.6.38; Apache
The administrator has disabled public write access.

Can home computer be a synchronized mirror? 9 months 3 weeks ago #7

  • janth
  • janth's Avatar
  • Offline
  • New
  • Posts: 29
This reference may be what you are looking for.

dev.mysql.com/doc/refman/5.7/en/replication.html
The administrator has disabled public write access.

Can home computer be a synchronized mirror? 9 months 3 weeks ago #8

  • WGroleau
  • WGroleau's Avatar
  • Offline
  • Gold
  • Posts: 1498
janth wrote:
This reference may be what you are looking for.

dev.mysql.com/doc/refman/5.7/en/replication.html
As Greg explained, there are many parts one would NOT want to synchronize, therefore doing it by MySQL would not be wise. Thus, I put in a feature request for an alternative.
--
Wes Groleau
UniGen.us/
PHP 7.1.13; MySQL 5.6.38; Apache
The administrator has disabled public write access.
Do you need a web hosting solution for your webtrees site?
If you prefer a host that specialises in webtrees, the following page lists some suppliers able to provide one for you: 

Can home computer be a synchronized mirror? 9 months 3 weeks ago #9

  • mp
  • mp's Avatar
  • Offline
  • Junior
  • Posts: 204
On my tablet/phone I use GedView (iOS) as ged file viewer. On a notebook you can use webtrees with LAMP/XAMPP/MAMP as viewer (and for testing new versions of webtrees or f.i. PHP).

But for changes I use webtrees online only.
Martin - ffp.bauschaffen.de
The administrator has disabled public write access.

Can home computer be a synchronized mirror? 9 months 3 weeks ago #10

  • WGroleau
  • WGroleau's Avatar
  • Offline
  • Gold
  • Posts: 1498
Well, since I am the only one making changes, I can do it anywhere and import a GEDCOM file to update my other site. But if and when other people start making changes, that won't work. And I don't want to depend on a lot of notes hanging around until I get back on-line (because they WON'T hang around; they will get lost).
--
Wes Groleau
UniGen.us/
PHP 7.1.13; MySQL 5.6.38; Apache
The administrator has disabled public write access.

Can home computer be a synchronized mirror? 2 weeks 5 days ago #11

  • chess
  • chess's Avatar
  • Offline
  • New
  • Posts: 4
Hi.

Just new to webtrees and I just set up master slave replication by mysql, too. I realized that the table wt_log is a problem: replication stops, because of duplicate entries. I just ignored this table and until now it seems to work. Will see. Anyone an idea how "dangerous" it is to ignore the log table? Are there any other tables I could/should ignore which could pose replication problems? It is not for backup just to get some other people in the family the possibility to view the data. They are not allowed to do chances. And I own the royalty to change data on my internal master server. Media is okay to update manually.

Thanks
chess
The administrator has disabled public write access.

Can home computer be a synchronized mirror? 2 weeks 5 days ago #12

  • fisharebest
  • fisharebest's Avatar
  • Online
  • Administrator
  • Posts: 10569
> I realized that the table wt_log is a problem: replication stops, because of duplicate entries

So you have set up master-slave replication *AND* you are also updating the slave?

With master-slave, it is normal for the slave to be read-only. Either for load-balancing or as a standby/backup.

I guess you could change this table to use the blackhole storage engine.
Greg Roach - This email address is being protected from spambots. You need JavaScript enabled to view it. - fisharebest.webtrees.net
The administrator has disabled public write access.

Can home computer be a synchronized mirror? 2 weeks 5 days ago #13

  • chess
  • chess's Avatar
  • Offline
  • New
  • Posts: 4
No, not both. But when I logged in there was something written in the log and replication stopped with an error.

I just ignored some more tables to let users change there user settings without letting the master overwrite these.

Goal is to only update the data. Do you know which tables exactly need to be copied/replicated?

I ignored for now the following tables (I just looked into the mysql database and whta makes sense - or maybe not - to ignore):
log, site setting, hit counter, block, session, user gedcom setting, user setting, next id

Thanks for the quick reply.

Chess
The administrator has disabled public write access.

Can home computer be a synchronized mirror? 2 weeks 5 days ago #14

  • fisharebest
  • fisharebest's Avatar
  • Online
  • Administrator
  • Posts: 10569
> No, not both. But when I logged in

By logging in to the slave, you *ARE* updating the database. Creating sessions, logs, settings, etc.

Not sure what you are trying to achieve. Your master is presumably on the same network, so access it directly.
Or, migrate the data from your master to your slave by downloading a GEDCOM file from your master and importing it on the slave.
Greg Roach - This email address is being protected from spambots. You need JavaScript enabled to view it. - fisharebest.webtrees.net
The administrator has disabled public write access.

Can home computer be a synchronized mirror? 2 weeks 5 days ago #15

  • chess
  • chess's Avatar
  • Offline
  • New
  • Posts: 4
It's is just for my personal feeling of security to have a working version I backup at home etc. and is stored on a NAS. But I don't want the NAS to be reached from the internet. So I setup a raspberry as a lamp server connected to the internet. And so I want the data to be replicated to the slave. Because I am a lazy person I want to use replication (which is quite fast besides) instead of export/import data as mysql can do this for me.

As you stated webtrees is writing in different tables but I guess the actual data is not changed during viewing the website. But the website will not working with a RO database - I guess. So I just want to replicate the tables which hold data for the ancestors (and maybe the user logins) but not for the individual user settings or tables which are crucial to the function of webtrees.

It is of course some kind of "dirty" but as long as it works I am fine with it.

But maybe you can give me a hint which tables hold data that is changed during the use of the website or configuration of users and which tables are *only* for the data. Do you think the ignores I stated above are enough or too many?

Thanks
Chess
The administrator has disabled public write access.

Can home computer be a synchronized mirror? 2 weeks 5 days ago #16

  • fisharebest
  • fisharebest's Avatar
  • Online
  • Administrator
  • Posts: 10569
>It is of course some kind of "dirty" but as long as it works I am fine with it.

> It's is just for my personal feeling of security

IMHO, these two objectives are opposites, and you cannot acheive both.

How about you set up a master-slave and do not touch the slave.

Then, have a daily job which copies (mysqldump) the slave into a "local working" copy.

This way, you'll have a full, synchronised database - plus a working copy that is at most a day old.

Also, don't forget to rsync your media files from your master server to your slave one.
Greg Roach - This email address is being protected from spambots. You need JavaScript enabled to view it. - fisharebest.webtrees.net
The administrator has disabled public write access.

Can home computer be a synchronized mirror? 2 weeks 5 days ago #17

  • chess
  • chess's Avatar
  • Offline
  • New
  • Posts: 4
This does not help. When I do a dump and import every day it is the same problem as replicating. Users/Family members are to use the Slave webtrees version. But when they change f.i. the default theme I overwrite these personal settings with the dump I import from the internal server as they may not write the settings from the webserver to my internal server.

And yes exactly I am trying to achieve both: give family members the possibility of viewing the latest data AND keep their personal preferences without doing a dump import.

This is not contradictory imho but for me this seems to be a fast and save way for my data. If someone hacks the Slave... doesn't matter I'm able to import a dump.

Thank you for the help. I will have an eye on the lamp and see if it's working with the parameters I am using now.

Chess
The administrator has disabled public write access.
Powered by Kunena Forum