1

LibreOffice uses XML-based formats which are zipped, making the resulting files relatively small, but useless for diff purposes in SVN. However, I recently learned that there are the flat XML equivalents (e.g. an .ods spreadsheet becomes a .fods flat XML spreadsheet) which are essentially text, and may be diff'ed in SVN.

Now, the thing about text vs binary in SVN is usually that if you have a 20KB file and it's compressed, then a tiny change will cost you another 20KB to commit if it's binary; while it might cost only a few bytes if it's text and only the diff is stored.

In my case I have a typical spreadsheet, which takes up 164KB as .fods (flat XML) and 18.3KB as .ods (compressed XML). When I add a few cells and save, doing a diff shows that over 50% of the file changed. And given that the flat XML version is 164KB, that would mean that it's actually more efficient to store the binary version.

So, am I missing something, or is this flat XML thing really inefficient?

Gigi
  • 28,163
  • 29
  • 106
  • 188
  • 1
    @bahrep this is not a duplicate - it's very different from the referenced question. See comments on the top-voted answer here for context. – Gigi Nov 22 '18 at 00:09

1 Answers1

1

This is essentially a duplicate of: Will Subversion efficiently store OpenXML Office documents?

The answer there is still true. There is work towards resolving this. Which you can read about a bit with Stefan's response to a similar question on the dev@subversion.apache.org list.

Format 7 in that thread is talking about FSFS format 7, an upcoming part of 1.9.0. Unfortunately, since then I believe the work that Stefan did towards this has slipped out of format 7 (but I could be wrong on that) and into the FSX back end, which is an experimental storage mechanism that will also be present in 1.9.0 but won't be recommended for production usage yet (but I could be wrong on that).

To your question about the flat XML, yes that'll help a lot. If you read the entire thread (rather than the single response I provided) I'm pretty sure it gets mentioned as a possible workaround for the time being.

It sounds like you're using using svn diff to get an idea of how much storage space the flat XML will give you. Unfortunately that's not going to really help you much. First of all Subversion uses a binary delta format that's far different from the unified diff format.

Some of your assumptions even about the zipped case are not really true. Just because you changed part of a zipped XML bundle doesn't mean the whole file will have changed, see Stefan's email that I linked to.

Also we don't store deltas just to the preceding revision of a file. Rather we use the skip delta algorithm to determine which revision to store a delta from and even sometimes store a full text. Purpose of that is to limit the amount of work done to calculate the full text of any given revision. Things are a little more complicated than that with 1.8 where there are some options for fsfs.conf that alters the skip delta algorithm.

If you want to get an accurate idea of if the flat files are efficient, you should do some experimentation and see how the repository size on disk grows.

Community
  • 1
  • 1
Ben Reser
  • 5,695
  • 1
  • 21
  • 29
  • My question is very different from the one you pointed out (which I read, by the way). I'm not asking anything about storage of the zipped format. I'm asking why it seems that storing flat XML versions of documents seems to be much more expensive than storing the compressed binary version, when a small change should cause a small delta, as opposed to changes throughout the document. – Gigi Jan 09 '14 at 06:53
  • Added some more details based on your comment. – Ben Reser Jan 09 '14 at 07:06
  • Thanks, that was helpful. My assumptions were based on this, by the way: https://wiki.documentfoundation.org/Libreoffice_and_subversion . I suppose that if statistics comparing compressed vs flat XML efficiency don't exist somewhere, I would do well to share the results in a blog post. – Gigi Jan 09 '14 at 08:03
  • I'd drawn the same conclusions (as you'll see if you read the whole thread I linked to) it seems like a pretty safe conclusion. But Stefan made some good technical arguments for while we can do better than that even with zipped XML bundles. I'd be interested in reading such a blog post. :) – Ben Reser Jan 09 '14 at 16:53