Question v2.0.17 - Things that no longer work after update

I am also on v2.0.17.
Parameter Text set to none - no difference.
Parameter Text set to markdown - no diffenrece.
With or without "almost markdown" CSS - no difference.

Text is still not properly formatted.

After looking in my data now (webtrees v2.0.17), the problem with missing line breaks seems to occur only with source citation text. In notes (shared notes or inline notes), the line break for CONT is implemented correctly and Markdown formatting is also displayed.

Example from demo site for v2.0.17:



Example from demo site for v2.1-dev:



According to Gedcom, there are CONC and CONT, where CONC is the continuation of the previous line and CONT is a new line. This has nothing to do with Markdown or other formatting.

jprause wrote:

I am also on v2.0.17.
Parameter Text set to none - no difference.
Parameter Text set to markdown - no diffenrece.
With or without "almost markdown" CSS - no difference.

Text is still not properly formatted.

There were problems in 2.0.17.

These are fixed jn 2.0-dev and 2.1-dev - i.e. the next released.

fisharebest wrote:

> But line breaks in texts (CONT) still do not work.

The markdown specification says that

* lines of text should be joined together to create one paragraph
* to create a new paragraph, use two line-breaks
* to force a line-break, add two spaces to the end of the previous line.

In webtrees werden Daten erfasst und im Gedcom-Format gespeichert. Zusätzlich gibt es die Möglichkeit, Textinformationen mit Markdown ergänzend zu formatieren. Warum wird mit v2.1 und der Verwendung von Code in v2.0.17 dieses Konzept so verändert, dass bei Verwendung von Markdown der komplette Text Markdown-formatiert werden muss? Dies führt dazu, dass alle bisher erfassten Texte nachformatiert werden müssten. Entschuldigung, warum?

In webtrees data is captured and stored in Gedcom format. Additionally there is the possibility to format text information with Markdown supplementary. Why does v2.1 and the use of code in v2.0.17 change this concept so that when Markdown is used, all text must be completely Markdown formatted? This leads to the fact that all previously captured text would have to be reformatted. Sorry, why?

mp wrote:

In webtrees data is captured and stored in Gedcom format. Additionally there is the possibility to format text information with Markdown supplementary. Why does v2.1 and the use of code in v2.0.17 change this concept so that when Markdown is used, all text must be completely Markdown formatted? This leads to the fact that all previously captured text would have to be reformatted. Sorry, why?

This is my opinion and question, too.

GEDCOM Specification says:

Grammar Rules
! Long values can be broken into shorter GEDCOM lines by using a subordinate CONC or CONT
tag. The CONC tag assumes that the accompanying subordinate value is concatenated to the
previous line value without saving the carriage return prior to the line terminator. If a
concatenated line is broken at a space, then the space must be carried over to the next line. The
CONT assumes that the subordinate line value is concatenated to the previous line, after inserting
a carriage return.

This should be so regardless of Markdown is used or not.

But, if you want "almost markdown", then a simple CSS change should give you what you want:

Code:

<style> .markdown > p { white-space: pre-wrap; } </style>

This CSS addition will probably not take effect until v2.0.18-dev or v2.1. Since I am not familiar with composer and npm, what is the possibility to test this already? Or is a v2.0.18 planned for the near future?

As I said previously, markdown says that consecutive lines of text are joined into one paragraph.

To add a paragraph-break, use a blank line.
To add a line-break, add two spaces to the end of the previous line.

We use the phpleague/commonmark ( commonmark.thephpleague.com/ ) package to format markdown text.

This is a very well supported and very popular package.

It has an undocumented feature. It keeps the line-breaks in the original text. If you have

Then it will generate

This means that we can use the CSS above to make the line-breaks appear.

It also means that if the library changes its behaviour, then this will stop.
i.e. it is not guaranteed to work in the future, and if it stops working, the package maintainers will not consider this to be a bug.

Thank you very much for the detailed answer.

However, it does not explain why an additional formatting option is changed to full text formatting without any apparent need? How are long-time users supposed to find all the text to be adjusted in source citations, inline notes and shared notes in order to reformat them accordingly?

Previously, a new CONT tag was translated with a <br> and an empty CONT tag with a <p>. What is the problem with keeping it like this, since a free text formatted with Markdown is something different than information created in a Gedcom structure? I believe that many previous users are rarely Markdown professionals and have only used parts of the available syntax to structure the data.

> Previously, a new CONT tag was translated with a <br> and an empty CONT tag with a <p>.

Previously, the CSS above was included in the core code.

This CSS should only have been included for plain-text notes - not for markdown notes.

This was a bug.

To restore the existing "bug", you can restore the "incorrect" CSS.

> What is the problem with keeping it like this

It prevents people from using 100% correct markdown.

As an alternative to the CSS, it should be straightforward to use the search/replace data-fix to add two-spaces to the end of the previous line - which will have the same effect, and will also be correct markdown.

GEDCOM 7.0 allows notes to be either plain-text or HTML. I am working to include markdown in a later version.

When this happens, it will be more important to follow the standards strictly.

mp wrote:

Previously, a new CONT tag was translated with a <br> and an empty CONT tag with a <p>.

GEDCOM 5.5.1 says:
Long values can be broken into shorter GEDCOM lines by using a subordinate CONC or CONT
tag. The CONC tag assumes that the accompanying subordinate value is concatenated to the
previous line value without saving the carriage return prior to the line terminator. If a
concatenated line is broken at a space, then the space must be carried over to the next line. The
CONT assumes that the subordinate line value is concatenated to the previous line, after inserting
a carriage return.

So CONC means removing the carriage return while CONT means keeping the carriage return.

In my webtrees installation I did NOT even enable Markdown, but CONT tags are now concatenated without keeping the carriage return. I consider this a GEDCOM violation. And from my point of view it has nothing to do with Markdown.

> In my webtrees installation I did NOT even enable Markdown, but CONT tags are now concatenated without keeping the carriage return. I consider this a GEDCOM violation. And from my point of view it has nothing to do with Markdown.

That was a bug - and has been fixed for 2.0.18 and 2.1.0.

Good news, thanks!

> As an alternative to the CSS, it should be straightforward to use the search/replace data-fix to add two-spaces to the end of the previous line - which will have the same effect, and will also be correct markdown.

Can you share a suitable RegEx expression for this? Thank you.

I agree that you should always stay close to the standard so that future updates are safe and easy.

But in this case we have a communication disaster. The modification from 2.0.16 to 2.0.17 came out of the blue. Most users had no idea what to expect. Suddenly all text fields in sources look like dirt. The reformatting in my family tree affects hundreds of text fields, across all webtrees family trees it is possibly hundreds of thousands.

Such a serious change should not be done so incidentally. An appropriate announcement for the implementation with sources and examples and a transition period for the changeover, in which both worlds work, should be granted to the users. One can see well from the discussion what displeasure this change has caused and will still cause.

When will 2.0.18 be available or would it be better and safe to downgrade?

Updating to 2.0.19, I can see now that the CONT Bug was fixed Great
Thank you

However, I might propose to adapt a little bit the "Markdown" parameter.

In fact, to be compatible with the former versions, where it was possible to use markdown in the Note (and not in the Text (Source)) it could be helpful to separate the 2 parameters.
1 parameter to decide Markdown for "Note"
1 parameter to decide Markdown for "Text"

Does it make sense ?

fisharebest wrote:

> What is the problem with keeping it like this

It prevents people from using 100% correct markdown.

If the standards come into conflict, shouldn't GEDCOM take priority? My albeit rather basic understanding is that limitations on line lengths were removed so markdown should in theory all go within one TEXT line anyway if so desired; overriding CONT seems to cause more problems than it fixes. The two trailing spaces to force a line-break seem to be a very much less than ideal solution as they're invisible and often get "cleaned" by external text-editors and sometimes even copy-paste. Enforcing this particular interpretation of markdown/GEDCOM interoperability seems to be causing a lot of problems without bringing any advantages IMHO.

I've seen other examples of mixed standards (e.g. HTML with BBcode and various Wiki-type markups) that coexist without having the same issues. I suppose if I was forced to make a choice between using markdown with broken linebreaks or not using at all it I'd go without for various reasons, but IMHO it shouldn't have to be either/or.

vometia said:

My albeit rather basic understanding is that limitations on line lengths were removed so markdown should in theory all go within one TEXT line anyway if so desired

Not when reading the current specification of GEDCOM v5.5.1 which still thinks data lines should only be a maximum of 255. Here is the specific documentation from the current v5.5.1 document:

! The total length of a GEDCOM line, including level number, cross-reference number, tag, value, delimiters, and terminator, must not exceed 255 (wide) characters.

If the standards come into conflict, shouldn't GEDCOM take priority?

In theory you could be correct but then a lot of people would be unhappy not just the markdown users! Many genealogy programs create their own dialect of GEDCOM and don't follow GEDCOM rules, if webtrees only followed GEDCOM then a lot of people would have issues! I am in a minority now of people who embraced webtrees as the only genealogy program they use. As more people import GEDCOMs from software that does not follow GEDCOM rules and don't embrace webtrees as their only genealogy program I am more likely to stop using webtrees, fork to my own copy, or stop helping others with GEDCOM issues.

The two trailing spaces to force a line-break seem to be a very much less than ideal solution as they're invisible and often get "cleaned" by external text-editors and sometimes even copy-paste. Enforcing this particular interpretation of markdown/GEDCOM interoperability seems to be causing a lot of problems without bringing any advantages IMHO.

I use markdown exclusively and it brings me great advantages over plain-text. Headings, bullets, html links are all valuable and not available in plain-text NOTEs!

You don't have to use markdown, I suggest to people that regularly import GEDCOM from other programs to not use markdown. It would get overlaid with every import! If on the other hand a webtrees user embraces webtrees, markdown is very valuable.

but IMHO it shouldn't have to be either/or.

markdown was initially designed as a replacement for plain-text.

What about to the proposal to separate the 2 parameters.
1 parameter to decide Markdown for "Note"
1 parameter to decide Markdown for "Text" (Sources)

norwegian_sardines wrote:

vometia said:

My albeit rather basic understanding is that limitations on line lengths were removed so markdown should in theory all go within one TEXT line anyway if so desired

Not when reading the current specification of GEDCOM v5.5.1 which still thinks data lines should only be a maximum of 255. Here is the specific documentation from the current v5.5.1 document:

! The total length of a GEDCOM line, including level number, cross-reference number, tag, value, delimiters, and terminator, must not exceed 255 (wide) characters.

If the standards come into conflict, shouldn't GEDCOM take priority?

In theory you could be correct but then a lot of people would be unhappy not just the markdown users! Many genealogy programs create their own dialect of GEDCOM and don't follow GEDCOM rules, if webtrees only followed GEDCOM then a lot of people would have issues! I am in a minority now of people who embraced webtrees as the only genealogy program they use. As more people import GEDCOMs from software that does not follow GEDCOM rules and don't embrace webtrees as their only genealogy program I am more likely to stop using webtrees, fork to my own copy, or stop helping others with GEDCOM issues.

The two trailing spaces to force a line-break seem to be a very much less than ideal solution as they're invisible and often get "cleaned" by external text-editors and sometimes even copy-paste. Enforcing this particular interpretation of markdown/GEDCOM interoperability seems to be causing a lot of problems without bringing any advantages IMHO.

I use markdown exclusively and it brings me great advantages over plain-text. Headings, bullets, html links are all valuable and not available in plain-text NOTEs!

I agree it's an improvement over plaintext, though it also lacks quite a lot of stuff I'd like to use (fonts, justifying, italic, strikeout, non-fixed-width tables etc) though my experiences of hand-writing reams of HTML mean I'm not enthusiastically embracing that as the alternative either. Which I'm just adding as a "be careful what you wish for" and I know HTML is coming. Yikes.

But in this particular case, the problem is that the means of signifying a newline is really unhelpful IMHO. I wouldn't totally object if I could see it (or its absence) and copy it, but both are awkward.

If that sounds negative, it's actually not (er, rather perversely) but because I'm actually really keen on webtrees: it's a nice robust system that's actually more standards-compliant than most and it doesn't require a Windows-based PC. I'm just being a bit "but why can't it be perfect? For everyone?" I suppose ideally there would be an option settable by the user to express their preference though I accept the risk that it can become a maintenance headache.

I'm probably also guilty of being a bit purist about it considering the Vesta add-on now seems to cater for what I want; so I'm not really sure why I'm so determinedly editing my GED bits to be complaint with the method du jour...

norwegian_sardines wrote:

You don't have to use markdown, I suggest to people that regularly import GEDCOM from other programs to not use markdown. It would get overlaid with every import! If on the other hand a webtrees user embraces webtrees, markdown is very valuable.

but IMHO it shouldn't have to be either/or.

markdown was initially designed as a replacement for plain-text.

I suppose I've viewed it as an optional extra that supplements plaintext, both in terms of the effects it has and being used on an ad hoc basis; I have a lot of legacy data (that's gradually being cleansed of some of that legacy but I'm not convinced it'll ever be 100% free of it) and have been using markdown when it seems appropriate and plaintext when it's not needed...

Thank you for the quick update to 2.0.19.

Fortunately the markdown formatting finally works again including line breaks and tables.

I 'only' had to invest several hours for
- update to 2.0.19
- GEDCOM export
- using a powerful editor on the *.ged file showing [spaces], [cr] and [nl]
- couting all text fields of sources (almost 600)
- modifying all relevant text fields (almost 500) with 2 additional spaces @ end of lines (estimated more than 3000)
- GEDCOM import
- partial data check @ relevant format representatives

phew

Question: What does the strict markdown spec say for tables? I recognized that I only need 2 spaces for normal line breaks. Tables don't need additional spaces. Will that change in future?