Web based family history software

This Help forum is for issues relates to the latest release (1.7.2). For issues related to beta or github version please use their own Help forum.
Before asking for help please read "How to request help" by clicking on that tab above here."

Question Accessing media through my face tagging addon

  • lambosprit
  • Topic Author
  • Offline
  • New Member
  • New Member
More
8 years 1 month ago #1 by lambosprit
Hi all,
Being new to webtrees I'm finding out what a wonderful piece of software it is. I've found it easy to get started and have loaded up my GEDCOM from an old version of family tree maker that my father used to create our original family tree.

Now I'm trying to add all the photo's we've collected over the years of long dead relatives. I started this but soon realised that some form of face tagging would help future recognition a lot so having done some work in php and mysql I set to work and now I have a set of files that allow me to do two things.
1) Select a photo and put it through openCV to find all the faces in it. This writes out the face location to a database table.
2) Show a photo with all attached users listed and highlight selected user on photo.

Now what I would like to do is be able to access the media stored in webtrees and link each tagged individual. I have worked out that the media and individual tagging is done in _media table but I need to create a new table for the actual tagging location in the photo. As part of this I would like to create a GEDCOM record to store the location but not the media itself.

So my problems:
1) To get the individual info I have to read the GEDCOM in the _individuals table but I can't figure out how to parse it using webtrees functions to get the users first and last name (I'm expecting it can be returned as some kind of array if I use the correct function.)
2) I can't access the media file because of the media firewall. Again I'm sure there is some function for this but can't find it.
3) I can't find any reference that tells me the standard for storing information like face tagging co-ordinates in a GEDCOM. I assume that it is therefore some user defined key but was wondering if this had been discussed with pro's and cons.

Thanks for any help and advice and I'll keep looking just in case I can work it out my self.

Please Log in or Create an account to join the conversation.

More
8 years 1 month ago #2 by fisharebest
Replied by fisharebest on topic Accessing media through my face tagging addon
1) get the users first and last name

Assuming you have an Individual object, then use Individual::getAllNames() to get a parsed array of names/parts

2) I can't access the media file because of the media firewall.

I guess webtrees privacy rules are applying, because you are trying to view living indivduals while not logged in.

I'm not sure what you are doing here. If you are writing code on the server, you can just access the file directly. You won't need to use the media-firewall.

I guess you need to log in to webtrees, and then your login session will give you access to the media.

3) I can't find any reference that tells me the standard for storing information like face tagging co-ordinates in a GEDCOM

There is no standard way of doing this.

Have you seen github.com/fisharebest/webtrees/issues/744

Greg Roach - greg@subaqua.co.uk - @fisharebest@phpc.social - fisharebest.webtrees.net

Please Log in or Create an account to join the conversation.

  • norwegian_sardines
  • Offline
  • Platinum Member
  • Platinum Member
More
8 years 1 month ago - 8 years 1 month ago #3 by norwegian_sardines
Replied by norwegian_sardines on topic Accessing media through my face tagging addon
@lambosprit asked:

3) I can't find any reference that tells me the standard for storing information like face tagging co-ordinates in a GEDCOM. I assume that it is therefore some user defined key but was wondering if this had been discussed with pro's and cons.

GEDCOM does not have a standard for saving face tagging cooridinates. You would need to save these coordinates outside of the GEDCOM standard. If you are clever you may be able to use the NOTE structure in such a way to create a human readable document that can also be parsed by your application/add-on to denote face locations in the associated phone/media object. Alternativelly you would need to add an additional record type to the database (Google Maps has its own record type as well) to maintain the location coordinates. Creating an additional record type is probably a better choice since the NOTE use idea will probably never be a human readable value. (webtrees uses human readable markup in other places but this does not contain XY coordinates, just heading markup)

Your face recognition idea has been asked for before and I think it is a great idea, we have just been waiting for a smart person to come along to create an add-on. Keep at it!!

Ken
Last edit: 8 years 1 month ago by norwegian_sardines.

Please Log in or Create an account to join the conversation.

  • lambosprit
  • Topic Author
  • Offline
  • New Member
  • New Member
More
8 years 1 month ago #4 by lambosprit
Replied by lambosprit on topic Accessing media through my face tagging addon
Thanks for the replys guy. I'm made progress on this and am starting to understand how webtrees works in the code.
I have a couple more questions now.

First, I can put a photo into the correct data folder as specified by webtrees and I create a record in the media table using the $WT_TREE->createRecord() function. I then use the mediafirewall to retrieve that photo as otherwise my program was denied access. The problem I have is that mediafirewall only returns the photo if it is linked to a familytree. Otherwise I get a red text saying photo is not in a family tree. Is there some data I'm missing? I haven't added any individual yet as no tags have been linked so I don't know yet what the user wants to create but I do know what family tree it needs to be added to (and I think I created the record in the media table with that data).

Second. I found it very difficult to start with to know how to invoke the functionality in the webtrees. I don't mean enabling it in the modules screen but actually putting links somewhere that the user can click on to go to the facetagging page. At the moment I've added a module tab and am seeing it on the individual. In effect I copied the media tab code and modified it. My question is, if i want to just add a photo and then tag it where would be the best place to enable the link (tab, side bar, which screen). If i want to view the tags same question really, where the best place to do it. I ask so I fit in with the webtrees standards rather than blazing my own trail.

On a point of protocol should I be asking all questions related to this module in one topic or should I raise new ones?

thanks again

thanks

Please Log in or Create an account to join the conversation.

More
8 years 1 month ago #5 by fisharebest
Replied by fisharebest on topic Accessing media through my face tagging addon
> The problem I have is that mediafirewall only returns the photo if it is linked to a familytree. Otherwise I get a red text saying photo is not in a family tree.

That's the intended functionality. Privacy/access is handled through the GEDCOM media object, so if there is no media object, then we cannot calculate privacy. Also, we protect the filenames, so the URL only contains the XREF of the media object and not the filename. Therefore there is no way to reference a file, other than through a media object.

There is nothing to stop you creating a media record. Perhaps use Tree::createRecord(). This function will even create a new/unique XREF for you if needed.

> but actually putting links somewhere that the user can click on to go to the facetagging page.

Perhaps your module could add a new menu to the media-object page? Look at how other modules create menus, and only add it when the current page is the media page?

Or create a tab? We already have two tabs (album and media) to give a choice of ways to view media.

You could create a new one, based on either of these.

The module system is pretty limited in this regard. It was coded in a bit of a hurry, in order to create version 1.0.0. We needed to guess what functions might be needed in the future. There aren't enough "hooks" into the system. The main problem is that much of the GUI is handled by the theme, and although it was originally intended that themes should be modules, it didn't get done. It will happen next time there is a major update to the module interface.

> On a point of protocol should I be asking all questions related to this module in one topic or should I raise new ones?

Since these are all questions about the same thing (i.e. development questions), it probably makes sense to keep them under one topic.

Greg Roach - greg@subaqua.co.uk - @fisharebest@phpc.social - fisharebest.webtrees.net

Please Log in or Create an account to join the conversation.

  • lambosprit
  • Topic Author
  • Offline
  • New Member
  • New Member
More
8 years 1 month ago - 8 years 1 month ago #6 by lambosprit
Replied by lambosprit on topic Accessing media through my face tagging addon
>That's the intended functionality. Privacy/access is handled through the GEDCOM media object, so if there is no media object, then we cannot calculate privacy. Also, we protect the filenames, so the URL only contains the XREF of the media object and not the filename. Therefore there is no way to reference a file, other than through a media object.

>There is nothing to stop you creating a media record. Perhaps use Tree::createRecord(). This function will even create a new/unique XREF for you if needed.

So using $WT_TREE->createRecord() is the same thing I assume. I do this before calling the mediafirewall so from what you're saying I think the photo should be returned. My m_media m_gedcom record is
0 @M15@ OBJE
1 FILE ../../../data/media/photo1.jpg
1 CHAN
2 DATE 19 FEB 2016
3 TIME 18:20:46
2 _WT_USER Admin1

and my m_file is set correctly.
I assume it doesn't have to be linked to an individual?

EDIT: I found the issue. I was adding the full path to the file name so it couldn't find it rather than not allowing it. Once corrected it all works.


>Or create a tab? We already have two tabs (album and media) to give a choice of ways to view media.

That's the way I've gone. I copied the media module and extended its functionality. Is there any way to hook into the media creation page in control panel in a module way or the media list in a module way?
Last edit: 8 years 1 month ago by lambosprit.

Please Log in or Create an account to join the conversation.

More
8 years 1 month ago #7 by fisharebest
Replied by fisharebest on topic Accessing media through my face tagging addon
1 FILE ../../../data/media/photo1.jpg

I'm not sure that paths beginning ../ will work as expected. webtrees makes no attempt to normalize these.

You should set the root of the media folder in your family tree settings, and the media object should contain a path/file within that.

> I assume it doesn't have to be linked to an individual

Correct. Media can be linked to other types of object - or to nothing at all.

Greg Roach - greg@subaqua.co.uk - @fisharebest@phpc.social - fisharebest.webtrees.net

Please Log in or Create an account to join the conversation.

  • lambosprit
  • Topic Author
  • Offline
  • New Member
  • New Member
More
8 years 1 month ago #8 by lambosprit
Replied by lambosprit on topic Accessing media through my face tagging addon
So I now have a version that I'm happy to call beta.

Whats the protocol for getting a test group together as I don't want to generally release this as it could screw up peoples trees if it has problems?

Please Log in or Create an account to join the conversation.

  • thomas52
  • Offline
  • Premium Member
  • Premium Member
  • Western North Carolina
More
8 years 1 month ago #9 by thomas52
Replied by thomas52 on topic Accessing media through my face tagging addon
Ask for volunteer beta testers to email you direct.

"Failure is an amazing teacher." (L'échec est un professeur extraordinaire.)

Please Log in or Create an account to join the conversation.

More
8 years 1 month ago #10 by fisharebest
Replied by fisharebest on topic Accessing media through my face tagging addon
You could post a link to the code here?

Once you're happy with it, there's an area on this site for third-party add-ons.

Greg Roach - greg@subaqua.co.uk - @fisharebest@phpc.social - fisharebest.webtrees.net

Please Log in or Create an account to join the conversation.

  • lambosprit
  • Topic Author
  • Offline
  • New Member
  • New Member
More
8 years 1 month ago #11 by lambosprit
Replied by lambosprit on topic Accessing media through my face tagging addon
Spoke too soon.

First problem. I have to keep reseting my password for logging into this forum everyday. Is there something you can check to see why it keeps being locked out?

Second and more importantly. I have tried every function I can find in webtrees but have not been able to find a solution. My problem is the following.

I have photos that already exist in webtrees and have been linked to an individual. I can now use my facetagging to open these photo's, find the faces and link an individual to each face. If I have tagged to an individual other than the originally linked one I create a linked fact. My problem is that if there is already a linked fact for that user to this media file I have no way of knowing so I create a second link.

I use the following code which fails as I have no access to the individual xref but hopefully it gives an idea of what I'm trying to do.
Code:
$record = GedcomRecord::getInstance($tags[0]["mediaId"],$WT_TREE); $linkedInd=$record->linkedIndividuals("OBJE");//we now have an array of individuals linked to this media foreach ($tags as $tag){ if (substr($tag["individual"],0,7) !="Unknown") { // only create a link if there is an individuals id $toggle = True;//should we create a link. Assume yes until confirmed otherwise if($linkedInd)){//check array exists before looping foreach($linkedInd as $indiv){//loop through list of individuals linked to this media record to see if it matches a tagged individual if($indiv->xref()==$tag["individual"]){######this line fails due to no access to protected xref####### $toggle=false;//link exists so don't create one } } } if($toggle){//no link for this individual so create one $indRecord = GedcomRecord::getInstance($tag["individual"],$WT_TREE); $indRecord->createFact('1 OBJE @' . $tag["mediaId"] . '@', true); Log::addEditLog('Media ID ' . $tag["mediaId"] . " successfully added to " . $tag["individual"]); } } }

Please Log in or Create an account to join the conversation.

More
8 years 1 month ago #12 by fisharebest
Replied by fisharebest on topic Accessing media through my face tagging addon
> if($indiv->xref()==$tag["individual"]){######this line fails due to no access to protected xref#######

$indiv->getXref()

Defined in class GedcomRecord (parent of class Individual)

Greg Roach - greg@subaqua.co.uk - @fisharebest@phpc.social - fisharebest.webtrees.net

Please Log in or Create an account to join the conversation.

More
8 years 1 month ago #13 by fisharebest
Replied by fisharebest on topic Accessing media through my face tagging addon
Tip

It is more efficient (and a little clearer) to use Individual::getInstance and Media::getInstance instead of GedcomRecord::getInstance.

The former can go straight to the right table in the database. The latter needs to check each of them in turn.

Greg Roach - greg@subaqua.co.uk - @fisharebest@phpc.social - fisharebest.webtrees.net

Please Log in or Create an account to join the conversation.

  • lambosprit
  • Topic Author
  • Offline
  • New Member
  • New Member
More
8 years 1 month ago #14 by lambosprit
Replied by lambosprit on topic Accessing media through my face tagging addon
Can't believe I didn't see that. It was late though. Works a treat. Would you be willing to test drive it Greg? I'll zip up the required files etc and send them to your sig email with some basic usage instructions.

Please Log in or Create an account to join the conversation.

More
8 years 2 weeks ago #15 by Shemwell
Replied by Shemwell on topic Accessing media through my face tagging addon
Hows the face tagging addon moving along?. I eagerly anticipate the addition.

Barry

Please Log in or Create an account to join the conversation.

  • lambosprit
  • Topic Author
  • Offline
  • New Member
  • New Member
More
8 years 1 week ago #16 by lambosprit
Replied by lambosprit on topic Accessing media through my face tagging addon
The basic functionality is all done and working great on my system. Just adding some stuff to allow manual creation of tags if the face detection fails.

Was hoping Greg would help me test it but not heard from him yet.

Anyone with a windows webserver who fancies a go?

Please Log in or Create an account to join the conversation.

More
8 years 1 week ago #17 by Shemwell
Replied by Shemwell on topic Accessing media through my face tagging addon
I'm in...

Barry

Please Log in or Create an account to join the conversation.

More
8 years 1 week ago #18 by joeysun
Replied by joeysun on topic Accessing media through my face tagging addon
I am very interested in this project, Unfortunately my server is in a Linux ecosystem.

Doug
webtrees v2.1.19 at Our Family Tree (Jiapu 家譜/家谱)
PHP Version v8.1.x, LiteSpeed V8.1, MariaDB 15.1 | protected/'hindered' by ModSecurity

Please Log in or Create an account to join the conversation.

More
8 years 1 week ago #19 by Tel
I'd be happy to test it out. I couldn't see an obvious way to email you my contact details but if you want to get in touch via my contact link on mynorfolkancestors.net I'll pass on details. Alternately, if you use GIT, SVN or similar perhaps post a link to your Project files here.

Terry
running webtrees 2.1.16 at mynorfolkancestors.net
on PHP 8.2, MySQL 5.7.

Please Log in or Create an account to join the conversation.

  • lambosprit
  • Topic Author
  • Offline
  • New Member
  • New Member
More
7 years 11 months ago #20 by lambosprit
Replied by lambosprit on topic Accessing media through my face tagging addon
Sorry for taking so long to post this but life gets in the way.

**** Warning **** No promises that this will work and won't screw up you're set up so please bear this in mind if you try it and back up everything first

This is a beta version so no GIT yet but once tested by a few people I'll work towards making it a webtrees module proper.

So to try it here are some simple instructions

Download the zip file from face_tagging.zip

unzip into the webtrees folder.
run the database_tagging.sql script against the webtrees database to add the new wt_media_tagging. If you've changed the database prefix modify the script's table name before running.

Download and install opencv 3.0.0

open webtrees and goto control panel->modules->module administration
enable face tagging

Goto an individual with a photo and open the face tagging tab. Should see a face icon on left next to photo. Click and it will open the face tagging. Wait while it finds faces. How long this takes depends on your server spec and the photo so give it a few minutes. The photo will then return with a list of found faces on left. You can click on left which will take you to the face or you can move mouse over face to see tag. Click on a face with a tag and the find for individual will be displayed. Once tags complete click save button at bottom of picture. Close by clicking on x top right of window.

Once a photo is tagged the face icon will disappear. To see the tagging click on the photo name under the photo.


Good luck and please give feedback on how it went so I can fix improve things

Please Log in or Create an account to join the conversation.

Powered by Kunena Forum
}