Question Feature Request: Improved support for UID / _UID

hermann wrote:

Franz Frese wrote:

If they are in 3 databases, they will for sure have 3 different uid's. So uid is useful in theory but useless in praxis.

 

Maybe at the beginning if they are created independently. When you identified that they are the same person you can copy.all the 3 UID to all the 3 INDI records.You are wrong, it is not useless.

Franz Frese wrote:

I guess nobody ever had a case identifying data from different databases as being the same individuum (by uid).

Your guess is completely wrong. Click and see: [url] gedbas.de/uid/58fcbf61b35a21e79bf54437e60f3d65 [/url]

Ha Ha,
der Einsender Philip Kuttler hat 2 Datenbanken hochgeladen und stellt fest, dass 2 Personen die selbe UID haben. Wahrscheinlich sind die beiden Datenbanken aus einer entstanden, die er lokal bei sich pflegt.

Die meisten der bei Gedbas hochgeladenen Datenbanken beinhalten noch keine UIDs.

---

Ha ha,
The submitter, Philip Kuttler, uploaded two databases and noted that two people have the same UID. The two databases likely originated from one he maintains locally.

Most of the databases uploaded to Gedbas don't yet contain UIDs.

norwegian_sardines wrote:

norwegian_sardines post=110506 userid=22

Only REFN can be used by the user, the other two are used by the system. A new UID is added each time the individual is added to all systems.

I don't understand what you are saying. A new UID could be automatically generated when a new data record is generated, for example a INDI record. If an individual is imported to a new tree then no new UID is generated.

REFN is a user defined identifier. Normally that identifier is only of value for the authors of a tree. UID is an identifier supported by genealogical programs to support the collaboration of users. EXID is an identifier into systems like FamilySearch or FindAGrave or VIAF or Roglo or Rodovid or others to identify persons or other entities. A user can use an EXID for example  to link a person in the tree to another copy of that person in other external databases.

webtrees supports REFN and_UID already. The support for REFN is OK. The support for UID is incomplete and therefore I formulated this feature request. webtrees should support EXID as soon as possible but that is another story.

Franz Frese wrote:

Ha Ha,

 

Entschuldige, Franz, aber ich empfinde Deinen Humor als hier völlig unangebracht. Es spielt keine Rolle, ob hier ein User die selbe Person mehrfach  hochgeladen hat, oder ob das verschiedene Personen waren. Es geht darum, dass man über die UID ein und die selbe Person in verschiedenen Bäumen identifizieren kann. Du hast gesagt, dass Du für UID keine Verwendung hast, deshalb möchte ich auf Deine Beiträge zu diesem Thema gerne verzichten. Zu anderen Themen schätze ich sie durchaus. 

Herman said:

REFN is a user defined identifier. Normally that identifier is only of value for the authors of a tree.

It does not have to be only for one author. I (if I only work a desktop app) could be collaborating with several other authors using desk top apps sending GEDCOMs back and forth or just talking via text or email about specific individuals we both have in our databases. We could mutually decide that each individual in our different databases share a mutual numbering system and enter that number in the REFN tag! Each time I emailed my collaborator I would just give this number and they would know who I was referencing.

The above is exactly what I do with two different authors who have very different databases but some overlap! If I found a member in my family (or a family line) that was also in your database, we could both use the same REFN to discuss the information we had about each person person(s) we have in common, much better than trying to track UID numbers!

hermann wrote:

If an individual is imported to a new tree then no new UID is generated.

I don't know why assigning a uuid should be limited to individuals entered by hand.

When in the control panel I have indicated "each individual should be assigned a generated uuid", then do an import, I would expect that:

- existing uuids are of course retained
- all individuals not already having a uuid get one assigned

How else? A datafix?

bertkoor wrote:

I don't know why assigning a uuid should be limited to individuals entered by hand.

When in the control panel I have indicated "each individual should be assigned a generated uuid", then do an import, I would expect that:

- existing uuids are of course retained
- all individuals not already having a uuid get one assigned

How else? A datafix?

You are right, Bert!
But first: the option in the control panel is labeled "Add unique identifiers". It is not restricted to INDI records. The comment is: "Unique identifiers allow the same record to be found in different family trees and in different systems. They will be added whenever records are created or updated. If you do not want unique identifiers to be displayed, you can hide them using the privacy rules.".

So the UID is added when records are created or updated but not when imported. Instead of changing that behaviour, I would prefer to have a data fix: "add unique identifiers to all records not already having one" and then you can select INDI, FAM or other records as usual. 
    

hermann wrote:

I’d like to suggest improved support for the UID (or _UID) tag in webtrees

Suggested improvements:

• Add the ability to search as user for UID in the advanced search panel.
• Use as admin matching UID/_UID values when suggesting merges.
• Add a new data fix to convert _UID to UID and vice versa as a function in the Extended Import/Export module (converting 5.5.1. to 7 and vice versa).
• Use the UID as a parameter in a new research link in the Fancy Research module..

In the latest release v4.2.6 of Extended Import/Export, I included the  _UID to UID conversion for the GEDCOM 7 export, i.e. from webtrees 5.5.1 to 7.

For the vice versa conversion from 7 to 5.5.1, I am not sure if we have use cases. If someone imports a GEDCOM 7 file into webtrees, the GEDCOM 7 UID tags will be shown correctly.

Hello all.

Please don't forget in the improvements to make, a way to insert references between records in notes (and titles, if possible)

I suggest options at:  How to find the reference of an individual

I really would like there was a way to write something like

#_UID:number#     
and make it just put a "link" with a default  text or maybe something like

​​​​​​#_UID:number:LINK#  or 
#_UID:number:LINK:text for the link#

That's why I proposed earlier this:#UID:number:ID# or #UID:number:NAME#But thinking about this again, because the UID is so long, I wouldn't like to show the ID itself. (Who knows, maybe someone does like it)

For example, someone would write this in a note:

#_UID:C277D3554D4F407EB6922984F771235C7976EC:LINK:This guy was also there#

and it would show a link like this:

This guy was also there

Or, 

#_UID:C277D3554D4F407EB6922984F771235C7976EC:NAME#

Or,

#_UID:C277D3554D4F407EB6922984F771235C7976EC:FULLNAME#

Or,

#_UID:C277D3554D4F407EB6922984F771235C7976EC:BIRTH:DATE#

You get the idea

Ely
 

elysch wrote:

Hello all.

Please don't forget in the improvements to make, a way to insert references between records in notes (and titles, if possible)

I suggest options at:  How to find the reference of an individual

I really would like there was a way to write something like

#_UID:number#     
and make it just put a "link" with a default  text or maybe something like

​​​​​​#_UID:number:LINK#  or 
#_UID:number:LINK:text for the link#

That's why I proposed earlier this:#UID:number:ID# or #UID:number:NAME#But thinking about this again, because the UID is so long, I wouldn't like to show the ID itself. (Who knows, maybe someone does like it)

webtrees has already implemented functionality to use references in text and notes. You can use any XREF in Notes/Text and webtrees will show the related record name with a link to the record. You need to insert the XREF as "@XREF@" into the text of the code.

GEDCOM example:
0 @X1@ INDI
1 PROP House
2 NOTE Bought the house from @I11163@

This will be shown as: 


For GEDCOM export, I implemented a GEDCOM filter within the Extended Import/Export  custom module, which replaces the @XREF@ references by the record names within a GEDCOM export. This feature was discussed here: Github issue 

webtrees has already implemented functionality to use references in text and notes. 

And it works fine as far as you don't merge different GEDCOMS or apply a renumbering of XREFs

That's why it would be better to use UID instead. Don't you think?



 

@elysch, webtrees is supporting that already if you have Markdown activated:
It should be possible to write a custom module that uses the UID number as a URL parameter to your own webtrees trees like
my.webtrees.de/uid/58fcbf61b35a21e79bf54437e60f3d65 . That link should show a search result in all your trees using the UID number.

It should be possible to write a custom module that uses the UID number as a URL parameter to your own webtrees trees like
my.webtrees.de/uid/58fcbf61b35a21e79bf54437e60f3d65  . That link should show a search result in all your trees using the UID number.

The missing thing is that as of today non the general search or the advanced search can find a record using the UID. I think that should be dealt with before a module like this makes sense... or maybe I'm exagerating
 

It seems webtrees does find a UID, but it does NOT find a _UID.

@elysch , webtrees is supporting that already if you have Markdown activated:
Code:
2 NOTE Bought the house from [Catharina Uuettwiller]( gedbas.de/uid/58fcbf61b35a21e79bf54437e60f3d65 )

Another problem with the markup approach is you will need to find and replace everywhere if you change the domain of your site, or if you share your GEDCOM. Needs maintenance.

The "#UID:number:LINK:text#" approach, is much cleaner.

There is code to avoid searching _UID fields because it was causing false positives.

It is in SearchService::rawGedcomFilter()

But I guess we should not do this if the search term is actually a UUID!

> But I guess we should not do this if the search term is actually a UUID!

Fix will be included in the next release.

fisharebest wrote:

... Fix will be included in the next release.
 

Thanks!

I tested the fix after adding the _UID of one idividual to a second individual so that the second individual got 2 _UIDs. I expected to find both individuals, but the fix only found the first individual.

It's working for me. I copy/paste the UID to a new individual, and the search returns the two individuals.

fisharebest wrote:

It's working for me. I copy/paste the UID to a new individual, and the search returns the two individuals.

 

You are correct. The search is working as expected. BUT the issue is a different one.

• When I copy and paste an 18 bytes (36 characters)  _UID (generated by Personal Ancestral File or Ancestral Quest) using the GUI via let's say www.example.net/tree/ExampleTree/add-fact/X1/_UID the form element cuts off the last 2 characters.
• When I copy and paste a 19 bytes (38 characters) _UID generated by webtrees the form field cuts off the last 4 characters.
• When I copy and paste a _UID of arbitrary length via the raw GEDCOM editor, webtrees does save that correctly and the search finds the duplicate _UIDs correctly. I guess this is what you did.

So, the issue is related to the form field and the _UID length and the confusion might have been caused by the GEDCOM authors / developers from FamilySearch themselves.

GEDCOM 7.0.16 reads on page 98 "New globally unique identifiers should be created and formatted as described in RFC 4122." which refers to 128 bit = 16 byte = 32 hex characters = 36 hex characters in 8-4-4-4-12 format including the 4 hyphens.
On page 99 it then reads "Note — Some systems used a 16-byte UUID with a custom 2-byte checksum for a total of 18 bytes:
checksum byte 1 = (sum of (byte ) for i 1 through 16) mod 256
checksum byte 2 = (sum of ((16 − i) × (byte )) for i 1 through 16) mod 256
Use of checksums for UIDs is discouraged except in cases where error-prone input is expected and an appropriate action to take in case of an error is known."
This is what FamilySearch obviously certified with Personal Ancestral File and Ancestral Quest and who knows what other applicationd did or do.

I didn't intend to open a can of worms, but I suggest to do two things now:

1. Eithers extend the GUI's form field to 38 characters so that a 38 byte _UID created by webtrees can be copied and pasted into it successfully. Or shorten the GUI's form field to 32 characters so that it follows RFC 4122. I admit this copying and pasting is a rare use case.
2. Revise the code for the _UID generation so that it generates only 128 bit = 16 byte = 32 hex characters according to RFC 4122 instead of creating 38 characters as it does now.

Let me know whether I should create a Github issue.

QUESTION:

Is webtrees now supporting v7.0.16 GEDCOM to this detail?  Should we start talking about all of the other items that are specific to this new GEDCOM release that are not fully implemented in webtrees?