Bug 70234 - FILESAVE: DOC/DOCX - Track change of fields not saved correctly
Summary: FILESAVE: DOC/DOCX - Track change of fields not saved correctly
Status: VERIFIED FIXED
Alias: None
Product: LibreOffice
Classification: Unclassified
Component: filters and storage (show other bugs)
Version:
(earliest affected)
4.1.2.3 release
Hardware: Other All
: medium normal
Assignee: Not Assigned
URL:
Whiteboard: target:6.4.0 target:6.3.0.1
Keywords: bibisected, dataLoss, regression
Depends on:
Blocks: Track-Changes DOCX-Fields DOCX-Track-Changes
  Show dependency treegraph
 
Reported: 2013-10-07 15:15 UTC by László Németh
Modified: 2019-08-07 15:59 UTC (History)
9 users (show)

See Also:
Crash report or crash signature:


Attachments
Removed page count field (7.40 KB, application/vnd.oasis.opendocument.text)
2013-10-07 15:15 UTC, László Németh
Details
Restored page count field by DOCX export (3.85 KB, application/vnd.openxmlformats-officedocument.wordprocessingml.document)
2013-10-07 15:16 UTC, László Németh
Details
Removed input field with change tracking (7.53 KB, application/vnd.oasis.opendocument.text)
2013-10-07 15:17 UTC, László Németh
Details
Restored input field by DOCX export (with deleted content) (3.88 KB, application/vnd.openxmlformats-officedocument.wordprocessingml.document)
2013-10-07 15:17 UTC, László Németh
Details
bibisect results for 4.1 regression (12.90 KB, text/plain)
2017-05-10 16:52 UTC, Terrence Enger
Details
bibisect results for 4.2 dataloss (8.27 KB, text/plain)
2017-05-10 20:59 UTC, Terrence Enger
Details
even without track-changes active, the field doesn't get exported to .docx (9.83 KB, application/vnd.oasis.opendocument.text)
2017-10-13 07:47 UTC, Justin L
Details

Note You need to log in before you can comment on or make changes to this bug.
Description László Németh 2013-10-07 15:15:50 UTC
Created attachment 87241 [details]
Removed page count field

See attached test files (ODT: before, DOCX: after).
Comment 1 László Németh 2013-10-07 15:16:35 UTC
Created attachment 87242 [details]
Restored page count field by DOCX export
Comment 2 László Németh 2013-10-07 15:17:13 UTC
Created attachment 87243 [details]
Removed input field with change tracking
Comment 3 László Németh 2013-10-07 15:17:55 UTC
Created attachment 87244 [details]
Restored input field by DOCX export (with deleted content)
Comment 4 steve 2015-04-16 07:17:20 UTC
This bug is lacking reproduce steps. My guess:

1. open first test file (I see "1" but that is deleted using "Track changes")
2. store as docx file
3. open docx file with LO

Currently: "1" is back there but without the Track changes info that it has been deleted

Expected: docx should have identical info as the odt file has.

Sidenote: opening the docx file with Word shows the track changes delete info. So this is def. a LO bug and as it looks not on the export side of things, but rather in correctly interpreting the exported information in docx files.

OSX 10.10.3 + LO Version: 4.5.0.0.alpha0+
Build ID: 09a1e4f36128f64029d45a38d9b059bf11ea0821
TinderBox: MacOSX-x86_64@49-TDF, Branch:master, Time: 2015-04-15_23:40:09
Locale: de_

Confirmed (since this has never been confirmed before, but was on NEW already for whatever reason).
Comment 5 tommy27 2016-04-16 07:24:06 UTC Comment hidden (obsolete)
Comment 6 Yousuf Philips (jay) (retired) 2017-05-09 05:58:34 UTC
So the problem here is that the <w:del> tag is only being placed around the first run element, rather than the entire group of run elements related to the field.

How it used to be in 4.0

<w:del ... w:id="0">
  <w:r>
    <w:rPr></w:rPr>
    <w:fldChar w:fldCharType="begin"></w:fldChar>
  </w:r>
  <w:r>
    <w:instrText>PAGE</w:instrText>
  </w:r>
  <w:r>
    <w:fldChar w:fldCharType="separate" />
  </w:r>
  <w:r>
    <w:delText>1</w:delText>
  </w:r>
  <w:r>
    <w:fldChar w:fldCharType="end" />
  </w:r>
</w:del>

How it is in 4.1 and above

<w:del ... w:id="0">
  <w:r>
    <w:rPr></w:rPr>
    <w:fldChar w:fldCharType="begin"></w:fldChar>
  </w:r>
</w:del> <- the problem happens here
<w:r>
  <w:instrText>PAGE</w:instrText>
</w:r>
<w:r>
  <w:fldChar w:fldCharType="separate" />
</w:r>
<w:r>
  <w:t>1</w:t>
</w:r>
<w:r>
  <w:fldChar w:fldCharType="end" />
</w:r>
Comment 7 Yousuf Philips (jay) (retired) 2017-05-09 13:54:04 UTC
Comment 6 covered attachment 87241 [details] and the page number field. Things are worse for attachment 87243 [details] and the input field.

Similar to the page number field, in 4.1 the <w:del> tag is only being placed around the first run element for the input field, but in 4.2 and above the entire field gets removed from export.

4.1

<w:pPr>
  <...>
<w:del ... w:id="0">
  <w:r>
    <w:rPr></w:rPr>
    <w:fldChar w:fldCharType="begin"></w:fldChar>
  </w:r>
</w:del> <-- the 4.1 bug mentioned in comment 6
<w:r>
  <w:instrText>FILLIN "example"</w:instrText>
</w:r>
<w:r>
  <w:fldChar w:fldCharType="separate" />
</w:r>
<w:r>
  <w:t>Example</w:t>
  <w:br />
</w:r>
<w:r>
  <w:fldChar w:fldCharType="end" />
</w:r>
<w:r>
  <w:rPr></w:rPr>
  <w:fldChar w:fldCharType="begin"></w:fldChar>
</w:r>
<w:r>
  <w:instrText>FILLIN "example"</w:instrText>
</w:r>
<w:r>
  <w:fldChar w:fldCharType="separate" />
</w:r>
<w:r></w:r>
<w:r>
  <w:fldChar w:fldCharType="end" />
</w:r>
</w:p>

4.2 and above

<w:p>
  <...>
  <w:del w:id="0" ...>
    <w:r>
      <w:rPr></w:rPr>
    </w:r>
  </w:del>
  <w:r>
    <w:rPr></w:rPr>
  </w:r>
</w:p>

@Xisco, @Raal, @Terrence: Would appreciate if any of you could bibisect both the 4.1 regression and 4.2 dataloss.
Comment 8 Yousuf Philips (jay) (retired) 2017-05-09 17:05:00 UTC
So it seems that even the ooxml code exported in 4.0 for the input field file is reported to be invalid when opened in MS Word and its likely because the field is being exported both in the <w:del> tag as well as outside of it.

<w:p>
  <...>
  <w:del ... w:id="0">
    <w:r>
      <w:rPr></w:rPr>
      <w:fldChar w:fldCharType="begin"></w:fldChar>
    </w:r>
    <w:r>
      <w:instrText>FILLIN &quot;example&quot;</w:instrText>
    </w:r>
    <w:r>
      <w:fldChar w:fldCharType="separate" />
    </w:r>
    <w:r>
      <w:delText>Example</w:delText>
      <w:br />
    </w:r>
    <w:r>
      <w:fldChar w:fldCharType="end" />
    </w:r>
  </w:del>
<!-- the w:r elements below shouldnt be there ->
  <w:r>
    <w:rPr></w:rPr>
    <w:fldChar w:fldCharType="begin"></w:fldChar>
  </w:r>
  <w:r>
    <w:instrText>FILLIN &quot;example&quot;</w:instrText>
  </w:r>
  <w:r>
    <w:fldChar w:fldCharType="separate" />
  </w:r>
  <w:r></w:r>
  <w:r>
    <w:fldChar w:fldCharType="end" />
  </w:r>
</w:p>
Comment 9 Terrence Enger 2017-05-09 20:20:40 UTC
@Yosuf,

I do not understand enough about the bug.  STR, detailed enough to
allow for my very small experience with track-changes, might help.

Does comment 9 mean that the 4.1 regression is really older than 4.1?
My oldest bibisect repository is 41max.

Terry.
Comment 10 Yousuf Philips (jay) (retired) 2017-05-10 04:42:01 UTC
(In reply to Terrence Enger from comment #9)
> I do not understand enough about the bug.  STR, detailed enough to
> allow for my very small experience with track-changes, might help.

1) Open attachment 87241 [details]
2) Save it as a .docx
3) Close document
4) rename the .docx to .zip
5) Unzip file
6) Open /word/document.xml
7) Search for </w:del> and see if its followed by </w:p>
8) If it is then regression hasnt occurred yet

> Does comment 9 mean that the 4.1 regression is really older than 4.1?
> My oldest bibisect repository is 41max.

The 4.1 regression happened after 4.0 was branched so 41max should find it.

For the dataloss that happened in 4.2, here are the steps
1) Open attachment 87243 [details]
2) Save it as a .docx
3) Close document
4) rename the .docx to .zip
5) Unzip file
6) Open /word/document.xml
7) Search for FILLIN
8) If it is found then regression hasnt occurred yet
Comment 11 Terrence Enger 2017-05-10 16:52:12 UTC
Created attachment 133218 [details]
bibisect results for 4.1 regression

Thank you, Yosuf, for those STR.  A result that can be checked without
user interaction works wonderfully with `git bisect run`.

Working in the bibisect-42max repository on debian-stretch, I see that
the 4.1 regression happened somewhere in the 33 source hashes ...

          commit    s-h       date
          --------  --------  ----------
    good  940934b5  db1118a6  2013-11-06
    bad   d8ba9023  a09f7fdd  2013-11-07

(I expect to be able return to bibisecting in about five hours, unless
somebody else gets here first.)

Terry.
Comment 12 Terrence Enger 2017-05-10 20:59:49 UTC
Created attachment 133224 [details]
bibisect results for 4.2 dataloss

Working in the 42max bibisect repository on debian-stretch, I see ...

          commit    s-h       date
          --------  --------  ----------
    good  9c9229dd  06ed06d2  2013-11-19
    bad   d0d8af75  c2b55219  2013-11-19

I am removing keyword bibisectRequest and adding bibisected (not
"bisected" because of range determined for 4.1 regression).
Comment 13 Yousuf Philips (jay) (retired) 2017-05-11 09:53:31 UTC Comment hidden (no-value)
Comment 14 Xisco Faulí 2017-09-14 10:44:01 UTC
Hi Terrence,
Which is the correct range of commits, comment 11 or comment 12 ?
Comment 15 Terrence Enger 2017-09-15 14:16:16 UTC
Xisco,

The behaviour of LibreOffice got worse in two stages.  Yosuf in c#7
describes the stages with the output markup at each stage; in c#10 he
gives STR at each stage.  So, I think c#11 and c#12 are both correct,
just for different problems.

Does this help?  Is there something more that I can do?
Comment 16 Yousuf Philips (jay) (retired) 2017-09-17 16:09:25 UTC Comment hidden (obsolete)
Comment 17 Justin L 2017-10-13 07:08:11 UTC
(In reply to Terrence Enger from comment #11)
> bibisect results for 4.1 regression
> Working in the bibisect-42max repository on debian-stretch, I see that
> the 4.1 regression happened somewhere in the 33 source hashes ...
>           commit    s-h       date
>           --------  --------  ----------
>     good  940934b5  db1118a6  2013-11-06
>     bad   d8ba9023  a09f7fdd  2013-11-07
The most likely commit seems to be cp#1000015 DOCX export: fix not-well-formed XML on redline end + hyperlink
https://cgit.freedesktop.org/libreoffice/core/commit/?id=7813bd3113beb2b2860f9eb213c5c67ef54f5bf0

However, this section of code has completely changed, so it won't be trivial to resolve.

The commit identified in comment 12 is the very large commit "Resolves: #i33737# enable in-place editing of Input Fields"
https://cgit.freedesktop.org/libreoffice/core/commit/?id=c2b5521921b806ff7b04cdacebde3834d2aafd4b
Comment 18 Justin L 2017-10-13 07:47:29 UTC
Created attachment 136944 [details]
even without track-changes active, the field doesn't get exported to .docx
Comment 19 QA Administrators 2018-10-14 02:57:24 UTC Comment hidden (obsolete)
Comment 20 Terrence Enger 2018-10-30 23:37:35 UTC
Working with debian-buster and daily Linux dbgutil bibisect repository
version 2018-10-30, commit 2e184378:

    Version: 6.2.0.0.alpha1+
    Build ID: 2e184378da6058af14acf208d6469fadf44d04ed
    CPU threads: 4; OS: Linux 4.18; UI render: default; VCL: gtk3; 
    Locale: en-CA (en_CA.utf8); Calc: threaded

I still see both bugs.
Comment 21 László Németh 2019-06-04 06:47:12 UTC
Part of the problem was the bad (missing) import, fixed in https://gerrit.libreoffice.org/#/c/73397/
Comment 22 Commit Notification 2019-06-04 09:15:05 UTC
László Németh committed a patch related to this issue.
It has been pushed to "master":

https://git.libreoffice.org/core/+/d160bca0dcb80f1b517b79221d5c3a43bbad8639%5E%21

tdf#70234 DOCX: import tracked deletion of fields

It will be available in 6.4.0.

The patch should be included in the daily builds available at
https://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
https://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.
Comment 23 Commit Notification 2019-06-04 12:10:20 UTC
László Németh committed a patch related to this issue.
It has been pushed to "master":

https://git.libreoffice.org/core/+/caf2f5fff39caf06204f71d0c2276b415ef047c3%5E%21

tdf#70234 DOCX: export tracked deletion of fields

It will be available in 6.4.0.

The patch should be included in the daily builds available at
https://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
https://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.
Comment 24 Commit Notification 2019-06-04 16:45:02 UTC
László Németh committed a patch related to this issue.
It has been pushed to "master":

https://git.libreoffice.org/core/+/222f6a1c43b7ae7bf8e2f1fbc1dbe6457e611ab7%5E%21

tdf#70234 DOCX export: fix only expanded fields

It will be available in 6.4.0.

The patch should be included in the daily builds available at
https://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
https://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.
Comment 25 Commit Notification 2019-06-05 09:29:20 UTC
László Németh committed a patch related to this issue.
It has been pushed to "libreoffice-6-3":

https://git.libreoffice.org/core/+/fe317b8d0845657703803d51e16f01e529740ef0%5E%21

tdf#70234 DOCX: import tracked deletion of fields

It will be available in 6.3.0.1.

The patch should be included in the daily builds available at
https://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
https://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.
Comment 26 Commit Notification 2019-06-05 12:55:32 UTC
László Németh committed a patch related to this issue.
It has been pushed to "libreoffice-6-3":

https://git.libreoffice.org/core/+/1d9513ee4036c5af93a263aeb173dbc7d74e843e%5E%21

tdf#70234 DOCX: export tracked deletion of fields

It will be available in 6.3.0.1.

The patch should be included in the daily builds available at
https://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
https://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.
Comment 27 Xisco Faulí 2019-06-05 15:09:49 UTC
Verified in

Version: 6.4.0.0.alpha0+
Build ID: 0d6ec494f83fb26524bf3a5fc7af27c225293e87
CPU threads: 4; OS: Linux 4.15; UI render: default; VCL: gtk3; 
Locale: ca-ES (ca_ES.UTF-8); UI-Language: en-US
Calc: threaded

@László Németh, thanks for fixing this issue!
Comment 28 Commit Notification 2019-06-05 18:07:02 UTC
László Németh committed a patch related to this issue.
It has been pushed to "libreoffice-6-3":

https://git.libreoffice.org/core/+/7dc4f5d686f7834702ff133b4e75f29ffa6b5ec3%5E%21

tdf#70234 DOCX export: fix only expanded fields

It will be available in 6.3.0.1.

The patch should be included in the daily builds available at
https://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
https://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.
Comment 29 Commit Notification 2019-06-14 09:18:18 UTC
Xisco Fauli committed a patch related to this issue.
It has been pushed to "libreoffice-6-3":

https://git.libreoffice.org/core/+/f5a5afc7f6525a07c124e154e2c727a6985f4044%5E%21

tdf#70234: This change didn't get cherry-picked

It will be available in 6.3.0.1.

The patch should be included in the daily builds available at
https://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
https://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.