This Help forum is for issues relates to the latest release (1.7.4). For issues related to beta or github version please use their own Help forum.
Before asking for help please read "How to request help" by clicking on that tab above here."
  • Page:
  • 1

TOPIC:

excessive number of duplicate families 4 years 11 months ago #1

  • wiskup
  • wiskup's Avatar Topic Author
  • Visitor
  • Visitor
Hello Forum members,

Our group is in the process of cleaning up our database and have come upon a strange situation. Thousands of "duplicate" families show up when we check for duplications. We call them phantom families because the card has only (... ...) listed for each individual, no children and no parents see attached image

Our question is, has anyone ever come across this situation and if so how can these phantom families be removed? This situation has inflated the actual number of families(30,00+) to over 110,000+ families.

Thank you for you help,

D. Wiskup
Attachments:

Please Log in or Create an account to join the conversation.

excessive number of duplicate families 4 years 11 months ago #2

  • thomas52
  • thomas52's Avatar
  • Away
  • Premium Member
  • Premium Member
  • Western North Carolina
  • Posts: 912
Have you tried > Control Panel > Manage family trees > Check for errors?
webtrees is designed to check & correct itself in many instances. You may need to save your gedcom, and then import it again from scratch, allowing webtrees to re-build the data. This remedies many problems.
"Failure is an amazing teacher." (L'échec est un professeur extraordinaire.)

Please Log in or Create an account to join the conversation.

excessive number of duplicate families 4 years 11 months ago #3

  • wiskup
  • wiskup's Avatar Topic Author
  • Visitor
  • Visitor
Thank you I will let the head editor know of the possible solution. I will let you know if it works. D. Wiskup

Please Log in or Create an account to join the conversation.

excessive number of duplicate families 4 years 11 months ago #4

  • fisharebest
  • fisharebest's Avatar
  • Away
  • Administrator
  • Administrator
  • Posts: 14491
Individuals and families have links in both directions.
For each link from an individual to a family, there must also be a link from the family to the individual.

It is not clear if

1) you have inconsistent links. e.g. the individual->family link exists, but the family->individual link does not exist.

2) these family records are orphans (no links to with any individual).

The solution to these two problems is different.

It would be interesting to know how these records were created. Did webtrees create them, or were they created by another application?
Greg Roach - This email address is being protected from spambots. You need JavaScript enabled to view it. - fisharebest.webtrees.net

Please Log in or Create an account to join the conversation.

excessive number of duplicate families 4 years 11 months ago #5

  • Marlin
  • Marlin's Avatar
  • Visitor
  • Visitor
The data was initially imported via GEDCOM of about 90,000 names and has subsequently grown to about 184,000 names. I did not realize that WT has a rebuilding nature as was suggested by Thomas 52. I had considered exporting our data via GEDCOM and using the clean up features of say RootsMagic to purge it. I will now try the simple export then re import to see what it does.

To respond to fisharebest --- the expected family to individual and back links appear to work properly and this phantoms appear with their GEDCOM comprised of only @Fxxx@ FAM and an ID number.

All of us HGRC Great New Mexuco Database editors are newbees to WT and have been learning by doing and still have a lot to learn.
Marlin

Please Log in or Create an account to join the conversation.

excessive number of duplicate families 4 years 11 months ago #6

  • bertkoor
  • bertkoor's Avatar
  • Offline
  • Platinum Member
  • Platinum Member
  • Greetings from Utrecht, Holland
  • Posts: 2268

fisharebest wrote: It would be interesting to know how these records were created. Did webtrees create them, or were they created by another application?


Judging by the rather high number shown in the attachment (F101466) it's quite possible this is created by webtrees :-/
Maybe it's some leftovers of a rejected change, I've never seen this before.

I also see it contains a GUID. By default webtrees does not generate that, but there's an option in the control panel under Preferences of the tree to switch that on.
If that now is set off and never has been set on, then that is a clue these records originate from another application.

Also under Geneanology Data - Changes Log you can search who dunnit (if you haven't exported and imported yet because that will purge the change log)
You have to play a bit with that form. Initially I could not retrieve the change log of a family of which I knew for sure it existed, but after first searching for an individual and filling in specific dates it did work.
stamboom.BertKoor.nl runs on webtrees v1.7.13

Please Log in or Create an account to join the conversation.

Last edit: by bertkoor.

excessive number of duplicate families 4 years 11 months ago #7

  • fisharebest
  • fisharebest's Avatar
  • Away
  • Administrator
  • Administrator
  • Posts: 14491
> I also see it contains a GUID. By default webtrees does not generate that, but
> there's an option in the control panel under Preferences of the tree to switch that on.
> If that now is set off and never has been set on, then that is a clue these
> records originate from another application.

Also, the record does not contain a "last change" timestamp.

webtrees creates these automatically - although it is possible to hide these using privacy.
Another reason to think that these records do not come from webtrees.

Note - we are not trying to blame anyone/thing here. We are just concerned that there may be a bug in webtrees, in which case we'll need to find/fix it.


I've written an SQL query which will find these families which have no members and no links to them.

Be certain to accept/reject all pending changes before running it.
SELECT f_id, f_file FROM wt_families
WHERE f_gedcom NOT LIKE '%\n1 CHIL%'
AND   f_gedcom NOT LIKE '%\n1 HUSB%'
AND   f_gedcom NOT LIKE '%\n1 WIFE%'
AND NOT EXISTS (
    SELECT * FROM wt_individuals
    WHERE i_file=f_file
    AND i_gedcom LIKE CONCAT('%@', f_id, '@%')
);

If you're happy that this is finding the right records, you can delete them in one operation by changing the first line to
DELETE FROM wt_families


After making any changes to the GEDCOM data in the database, it is usually necessary to export/import the data.

It's probably not necessary for this particular update (as we're not changing anydates/places/links/names/etc.) - but it can do no harm.


PS - make a backup before you update the DB like this!
Greg Roach - This email address is being protected from spambots. You need JavaScript enabled to view it. - fisharebest.webtrees.net

Please Log in or Create an account to join the conversation.

excessive number of duplicate families 4 years 11 months ago #8

  • Marlin
  • Marlin's Avatar
  • Visitor
  • Visitor
You are addressing the right issue for sure. It appears that your SQL query would indeed be very useful. I do not know how to run it. My WT is running on a server at Southwest Cyberport, ( www.swcp.com/internet-access/ ) a provider of our organizion's web services. Can I run it from the WT control panel? Pardon my ignorance. Then there is the remaining question of where the phantoms came from.
Marlin

Please Log in or Create an account to join the conversation.

Do you need a web hosting solution for your webtrees site?
If you prefer a host that specialises in webtrees, the following page lists some suppliers able to provide one for you: 

excessive number of duplicate families 4 years 11 months ago #9

  • fisharebest
  • fisharebest's Avatar
  • Away
  • Administrator
  • Administrator
  • Posts: 14491
> I do not know how to run it.

99% of web hosts provide access to the database using a tool called "phpMyAdmin".

(This link suggests you're one of the 99% - members.swcp.com/wiki/How_to_back_up_your_MySQL_database)

Find phpMyAdmin, and if you can't find the "run SQL" page, tell us which version of phpMyAdmin you're using and we can point you towards instructions for that version.

> Then there is the remaining question of where the phantoms came from.

The control panel has a "Changes" log on the family trees page. It will show you the history of a particular record. Look up the history of one of these families. It may give some clues.
Greg Roach - This email address is being protected from spambots. You need JavaScript enabled to view it. - fisharebest.webtrees.net

Please Log in or Create an account to join the conversation.

excessive number of duplicate families 4 years 11 months ago #10

  • Marlin
  • Marlin's Avatar
  • Visitor
  • Visitor
Many thanks, I will see what I can do.

Please Log in or Create an account to join the conversation.

excessive number of duplicate families 4 years 11 months ago #11

  • Jackie
  • Jackie's Avatar
  • Offline
  • Platinum Member
  • Platinum Member
  • Posts: 4755
Hi,

I've seen this before.
It happened when one of my users created a family record, and then realised she added the wrong husband to the wrong wife.
Instead of removing the link between spouses and re-linking them to the right individual, she simply deleted the names of the wrong wife/husband

I ended up with one family with two individuals with no name.

Please Log in or Create an account to join the conversation.

Last edit: by Jackie.
  • Page:
  • 1
Powered by Kunena Forum