Question
GEDCOM Import Error - Duplicate entry
- fisharebest
-
- Offline
- Administrator
-
Less
More
3 weeks 12 hours ago #116897
by fisharebest
Greg Roach - greg@subaqua.co.uk - @fisharebest@phpc.social - fisharebest.webtrees.net
Replied by fisharebest on topic GEDCOM Import Error - Duplicate entry
> I've read the GEDCOM specs again, and it implies that from the context (within a GEDCOM record) it should always be known what entity type a pointer points to.
I think you are right here. Which is why webtrees works for these sorts of duplicates.
But webtrees does have a few cases where we use XREFs independently. Change records, the search box in the header, etc.
The GEDCOM 5.5.1 spec does say anything about whether XREFs are case sensistive or not.
But if I remember correctly, GEDCOM 7 says they ARE case sensitive. So a good reason for us to move to case-sensitivity.
I think you are right here. Which is why webtrees works for these sorts of duplicates.
But webtrees does have a few cases where we use XREFs independently. Change records, the search box in the header, etc.
The GEDCOM 5.5.1 spec does say anything about whether XREFs are case sensistive or not.
But if I remember correctly, GEDCOM 7 says they ARE case sensitive. So a good reason for us to move to case-sensitivity.
Greg Roach - greg@subaqua.co.uk - @fisharebest@phpc.social - fisharebest.webtrees.net
Please Log in or Create an account to join the conversation.
- Anke
-
- Offline
- New Member
-
Less
More
- Posts: 11
6 days 12 hours ago #116974
by Anke
Replied by Anke on topic GEDCOM Import Error - Duplicate entry
I get this error message:
Deleting this record in the database (MariaBD) doen't help; this error message just lists the next record.
It all started by reimporting the GEDCOM file I had exported before. Do I have to empty the database before importing a GEDCOM file? Anyway, after the import I worked on the error messages on webtrees' error page, fixing some errors using the UI, some directly in the gedcom raw data. Everything seemed to be fine so far until I opened the error page again. I got the above SQL message, then errors stating currupt paths to pictures that had been there when I set them up, and now webtrees doesn't even show the frontend (My Page).
Anything I can do to save this without having to install webtrees from scratch and to work for days again, fixing the gedcom errors? (I don't understand too much ob what you were talking about above...)
Code:
SQLSTATE[23000]: Integrity constraint violation: 1062 Duplicate entry '4-M12' for key 'PRIMARY' (Connection: default, Host: localhost, Port: 3306, Database: d047c08f, SQL: insert into `wt_media` (`m_id`, `m_file`, `m_gedcom`) values (M12, 4, 0 @M12@ OBJE 1 FILE oberhof/Benjamin_Oberhof_1986.jpg 2 FORM JPG 3 TYPE PHOTO 2 TITL Benjamin Oberhof, 1986 1 CHAN 2 DATE 31 JUL 2026 3 TIME 10:14:51 2 _WT_USER Anke))
Deleting this record in the database (MariaBD) doen't help; this error message just lists the next record.
It all started by reimporting the GEDCOM file I had exported before. Do I have to empty the database before importing a GEDCOM file? Anyway, after the import I worked on the error messages on webtrees' error page, fixing some errors using the UI, some directly in the gedcom raw data. Everything seemed to be fine so far until I opened the error page again. I got the above SQL message, then errors stating currupt paths to pictures that had been there when I set them up, and now webtrees doesn't even show the frontend (My Page).
Anything I can do to save this without having to install webtrees from scratch and to work for days again, fixing the gedcom errors? (I don't understand too much ob what you were talking about above...)
Please Log in or Create an account to join the conversation.
- bertkoor
-
- Offline
- Platinum Member
-
- Greetings from Utrecht, Holland
6 days 10 hours ago - 6 days 10 hours ago #116975
by bertkoor
stamboom.BertKoor.nl runs on webtrees v2.2.6
Replied by bertkoor on topic GEDCOM Import Error - Duplicate entry
No need to panic. When you import in webtrees, then everything already in that specific tree is purged first.
My advice is to open the GEDCOM file with a capable text editor (eg Notepad++) and seach for "0 @M12@" without quotes. This should occur only once. If you find multiple, then you can fix it for example by changing one to M12_1 and the other to M12_2.
Other @M12@ references should change accordingly.
My advice is to open the GEDCOM file with a capable text editor (eg Notepad++) and seach for "0 @M12@" without quotes. This should occur only once. If you find multiple, then you can fix it for example by changing one to M12_1 and the other to M12_2.
Other @M12@ references should change accordingly.
stamboom.BertKoor.nl runs on webtrees v2.2.6
Last edit: 6 days 10 hours ago by bertkoor.
Please Log in or Create an account to join the conversation.
- Anke
-
- Offline
- New Member
-
Less
More
- Posts: 11
6 days 9 hours ago - 6 days 6 hours ago #116976
by Anke
Replied by Anke on topic GEDCOM Import Error - Duplicate entry
Thanks! That sounds promising. Only, almost all Object-Id's seem to be there twice, one time more at the top of the file and again in the lower part... Any idea how this can happen?
Edit:
Now, looking at the gedcom file I see that there aren't only duplicate links but also different syntax.
Example:
line 244
line 2378
Obviously something got mixed up while I was editing records the last few days, so that added or past© items were given existing/duplicate import links (@) and in the wrong order, i.e. "@X8@ OBJE" instead of "OBJE @X8@"...
I tend to start all over, importing my old TNG file again. I'd just like to understand what happened.
Edit:
Now, looking at the gedcom file I see that there aren't only duplicate links but also different syntax.
Example:
line 244
Code:
2 OBJE @X8@ 2 RESN PRIVACY, LOCKED 1 EVEN Flucht in den Westen
line 2378
Code:
0 @X8@ OBJE 1 FILE keding/keding_willi_aerzte_in_mecklb.jpg 2 FORM JPG
3 TYPE PHOTO
Obviously something got mixed up while I was editing records the last few days, so that added or past© items were given existing/duplicate import links (@) and in the wrong order, i.e. "@X8@ OBJE" instead of "OBJE @X8@"...
I tend to start all over, importing my old TNG file again. I'd just like to understand what happened.
Last edit: 6 days 6 hours ago by Anke.
Please Log in or Create an account to join the conversation.
- bertkoor
-
- Offline
- Platinum Member
-
- Greetings from Utrecht, Holland
6 days 6 hours ago #116977
by bertkoor
stamboom.BertKoor.nl runs on webtrees v2.2.6
Replied by bertkoor on topic GEDCOM Import Error - Duplicate entry
No idea. What program has created this file?
Do these records look similar?
Do these records look similar?
stamboom.BertKoor.nl runs on webtrees v2.2.6
Please Log in or Create an account to join the conversation.
- Sir Peter
-
- Offline
- Premium Member
-
Less
More
- Posts: 536
6 days 5 hours ago #116978
by Sir Peter
Peter
Replied by Sir Peter on topic GEDCOM Import Error - Duplicate entry
Anke, I believe you imported your GEDCOM file twice and checked the „Keep media objects“ checkbox. This would keep the media objects, not the respective files on OS level. That leads to duplicate media objects in the imported GEDCOM. Importing it once more that way will create a third version of all media objects and so on.
The only quick way to fix this is to import the original GEDCOM file and NOT check the „Keep media objects“. I‘d create a second tree for this.
The only quick way to fix this is to import the original GEDCOM file and NOT check the „Keep media objects“. I‘d create a second tree for this.
Peter
Please Log in or Create an account to join the conversation.
- Anke
-
- Offline
- New Member
-
Less
More
- Posts: 11
6 days 1 hour ago #116979
by Anke
Replied by Anke on topic GEDCOM Import Error - Duplicate entry
Hi Peter,
your explanation sounds logical in a way. So I imported the old TNG file into a new family tree, following your advice. As I couldn't find a quick way to hide my first family tree from everybody but myself, I deleted it.
Strange things happen:
A look at the file system showed me that the three modules I had installed for my first tree were gone, however, one of them could still be found in the database.
The cache folder was gone, too. I created it - without any subfolders. The cache folder is still empty, although I've been working in the media management.
After the import most of my pictures were visible in the media management, some showed something like a 404. What really drives me crazy is when I check the file system via FTP and I see that a .jpg exists, so just to rule out any mistyping on my side I copy the filename and manually paste it into the filename field - but webtrees still tells me that .jpg doesn't exist. Or when I delete a picture, I'm informed that either it doesn't exist or I don't have permission to delete it. Or when I take comfort in uploading multiple files via FTP, then webtrees recognizes some of them, ignoring others. Some things are a mystery to me.
There are a lot of things I like about webtrees, I wonder if I'll get along with things like the ones mentioned. Version 2.2.6 isn't a secret delta or nightly, is it.
your explanation sounds logical in a way. So I imported the old TNG file into a new family tree, following your advice. As I couldn't find a quick way to hide my first family tree from everybody but myself, I deleted it.
Strange things happen:
A look at the file system showed me that the three modules I had installed for my first tree were gone, however, one of them could still be found in the database.
The cache folder was gone, too. I created it - without any subfolders. The cache folder is still empty, although I've been working in the media management.
After the import most of my pictures were visible in the media management, some showed something like a 404. What really drives me crazy is when I check the file system via FTP and I see that a .jpg exists, so just to rule out any mistyping on my side I copy the filename and manually paste it into the filename field - but webtrees still tells me that .jpg doesn't exist. Or when I delete a picture, I'm informed that either it doesn't exist or I don't have permission to delete it. Or when I take comfort in uploading multiple files via FTP, then webtrees recognizes some of them, ignoring others. Some things are a mystery to me.
There are a lot of things I like about webtrees, I wonder if I'll get along with things like the ones mentioned. Version 2.2.6 isn't a secret delta or nightly, is it.
Please Log in or Create an account to join the conversation.
- fisharebest
-
- Offline
- Administrator
-
6 days 53 minutes ago #116980
by fisharebest
Greg Roach - greg@subaqua.co.uk - @fisharebest@phpc.social - fisharebest.webtrees.net
Replied by fisharebest on topic GEDCOM Import Error - Duplicate entry
> After the import most of my pictures were visible in the media management, some showed something like a 404. What really drives me crazy is when I check the file system via FTP and I see that a .jpg exists, so just to rule out any mistyping on my side I copy the filename and manually paste it into the filename field - but webtrees still tells me that .jpg doesn't exist
The 403/404/500 image placeholders actually have additional information in the HTTP headers.
When I added this feature, I was concerned about leaking information - e.g. filenames, which we don't normally show to visitors.
But since most people don't add media titles, the title is the filename, and since most people don't know how to look at HTTP headers, I made a change for 2.3.0
The "placeholders" for image errors now have an error message in the image itself.
The 403/404/500 image placeholders actually have additional information in the HTTP headers.
When I added this feature, I was concerned about leaking information - e.g. filenames, which we don't normally show to visitors.
But since most people don't add media titles, the title is the filename, and since most people don't know how to look at HTTP headers, I made a change for 2.3.0
The "placeholders" for image errors now have an error message in the image itself.
Greg Roach - greg@subaqua.co.uk - @fisharebest@phpc.social - fisharebest.webtrees.net
Please Log in or Create an account to join the conversation.
- Sir Peter
-
- Offline
- Premium Member
-
Less
More
- Posts: 536
5 days 13 hours ago #116983
by Sir Peter
You should make yourself familiar with GEDCOM, especially with the differences between files and media objects, sources and citations and when and how to reuse them instead of creating them anew. And most importantly do NOT mess with the database or the filesystem or you will make things worse.
There‘s one OS level activity I recommend. Create subdirectories under ./data/media for different trees and different categories of files (sources) like this:
./data/media/myFirstTree/Portraits
./data/media/myFirstTree/ChurchBooks
./data/media/mySecondTree/Portraits
./data/media/mySecondTree/ChurchBooks
Peter
Replied by Sir Peter on topic GEDCOM Import Error - Duplicate entry
This is perfectly correct!Thanks! That sounds promising. Only, almost all Object-Id's seem to be there twice, one time more at the top of the file and again in the lower part... Any idea how this can happen?
Edit:
Now, looking at the gedcom file I see that there aren't only duplicate links but also different syntax.
Example:
line 244Code:2 OBJE @X8@ 2 RESN PRIVACY, LOCKED 1 EVEN Flucht in den Westen
line 2378Code:0 @X8@ OBJE 1 FILE keding/keding_willi_aerzte_in_mecklb.jpg 2 FORM JPG 3 TYPE PHOTO
Obviously something got mixed up while I was editing records the last few days, so that added or past© items were given existing/duplicate import links (@) and in the wrong order, i.e. "@X8@ OBJE" instead of "OBJE @X8@"...
I tend to start all over, importing my old TNG file again. I'd just like to understand what happened.
Code:
2 OBJE @X8@ is a reference/pointer to this object.
0 @X8@ OBJE is the object itself.
You should make yourself familiar with GEDCOM, especially with the differences between files and media objects, sources and citations and when and how to reuse them instead of creating them anew. And most importantly do NOT mess with the database or the filesystem or you will make things worse.
There‘s one OS level activity I recommend. Create subdirectories under ./data/media for different trees and different categories of files (sources) like this:
./data/media/myFirstTree/Portraits
./data/media/myFirstTree/ChurchBooks
./data/media/mySecondTree/Portraits
./data/media/mySecondTree/ChurchBooks
Peter
Please Log in or Create an account to join the conversation.
- Anke
-
- Offline
- New Member
-
Less
More
- Posts: 11
4 days 9 hours ago #117005
by Anke
Replied by Anke on topic GEDCOM Import Error - Duplicate entry
I read a little about GEDCOM syntax but didn't find this info about an object reference and the object itself. I'm learning.
Yes, I am using subdirectories per family, I'm just not sure my folks will follow this system. There is probably no way to assign specific directories for uploads to users?
And, sorry, I will "mess" with program files and the DB if that means saving a lot of time, where webtrees doesn't provide a possibility to edit multiple records in one step - as long as I know what I'm doing and after saving backups. There are 434 GEDCOM errors listed in the family tree, about half of which are UId'S.
Yes, I am using subdirectories per family, I'm just not sure my folks will follow this system. There is probably no way to assign specific directories for uploads to users?
And, sorry, I will "mess" with program files and the DB if that means saving a lot of time, where webtrees doesn't provide a possibility to edit multiple records in one step - as long as I know what I'm doing and after saving backups. There are 434 GEDCOM errors listed in the family tree, about half of which are UId'S.
Please Log in or Create an account to join the conversation.