Bug 493734

Do you have a non-sensitive version of the file that's making gnucash choke? I don't want your private financial data, obviously, but without an example file, it's going to be hard to test/fix.

The sample files I have all work OK.

Do you have old .xac files in your save dir, if so, do they open OK? (and can you find a reasonable difference where it stops opening them)

Ah, I didn't know that the *.xac were in the same format as the *.gnucash files, and therefore could be opened directly.  I usually delete them, but fortunately they were still around this time.

Okay, here's what I have:

-rw-r--r-- 1 neil users 18227 Apr  3 21:22 2009.gnucash (FAILS)
-rw-r--r-- 1 neil users 18914 Apr  3 21:22 2009.gnucash.20090331195953.xac (OPENS!)
-rw-r--r-- 1 neil users  1450 Apr  3 21:22 2009.gnucash.20090331200013.log
-rw-r--r-- 1 neil users  2622 Apr  3 21:22 2009.gnucash.20090331200534.log
-rw-r--r-- 1 neil users 17530 Apr  3 21:22 2009.gnucash.20090331200534.xac (FAILS)
-rw-r--r-- 1 neil users 17779 Apr  3 21:22 2009.gnucash.20090331201708.xac (FAILS)
-rw-r--r-- 1 neil users   170 Apr  3 21:22 2009.gnucash.20090331201709.log

It's odd that the first file that fails (20090331200534) is SMALLER than the previous file that worked (20090331195953); maybe it's truncated??  I would have been adding entries, not deleting any, I think.

How do I figure out which *.log files go with which *.xac files?  i.e. does 20090331200534.log contain changes that are made before 20090331200534.xac was saved, or after?

How do I interpret the *.log files?  e.g.:
===== START
C       8083d8ecdaa2b4dcc115c366ceba9f81        a6593fe44785a2052ac30e5708802384
        2009-03-31 20:02:24.000000 -0400        2009-03-31 20:02:24.000000 -0400
        2009-03-24 00:00:00.000000 -0400        db03ba7f6b2e1aa886a74182b0c8fda2
        Meals Out               Dinner                          n       1128/100
        1128/100        1969-12-31 19:00:00.000000 -0500
C       8083d8ecdaa2b4dcc115c366ceba9f81        d27cab79bfee9c48285c0208fd2b8d92
        2009-03-31 20:02:24.000000 -0400        2009-03-31 20:02:24.000000 -0400
        2009-03-24 00:00:00.000000 -0400        ecb97484f729aece1d09b333bf0b7df6
        Cash in Wallet          Dinner                          n       -1128/10
0       -1128/100       1969-12-31 19:00:00.000000 -0500
===== END
Does the "1128/100" mean $11.28?  Was this on Mar. 24 or Mar 31? (not sure why there are two dates, and I don't remember all the transactions)

While I don't want to attach the files to this ticket, I could send them to you if needed (as long as they don't end up posted somewhere, obviously).

(In reply to comment #3)
> Ah, I didn't know that the *.xac were in the same format as the *.gnucash
> files, and therefore could be opened directly.  I usually delete them, but
> fortunately they were still around this time.
> 
> Okay, here's what I have:
> 
> -rw-r--r-- 1 neil users 18227 Apr  3 21:22 2009.gnucash (FAILS)
> -rw-r--r-- 1 neil users 18914 Apr  3 21:22 2009.gnucash.20090331195953.xac
> (OPENS!)
> -rw-r--r-- 1 neil users  1450 Apr  3 21:22 2009.gnucash.20090331200013.log
> -rw-r--r-- 1 neil users  2622 Apr  3 21:22 2009.gnucash.20090331200534.log
> -rw-r--r-- 1 neil users 17530 Apr  3 21:22 2009.gnucash.20090331200534.xac
> (FAILS)
> -rw-r--r-- 1 neil users 17779 Apr  3 21:22 2009.gnucash.20090331201708.xac
> (FAILS)
> -rw-r--r-- 1 neil users   170 Apr  3 21:22 2009.gnucash.20090331201709.log
> 
> It's odd that the first file that fails (20090331200534) is SMALLER than the
> previous file that worked (20090331195953); maybe it's truncated??  I would
> have been adding entries, not deleting any, I think.

Yeah, that's a little weird - I'm seeing the file length to be monotonically increasing, here.

> How do I figure out which *.log files go with which *.xac files?  i.e. does
> 20090331200534.log contain changes that are made before 20090331200534.xac was
> saved, or after?

'ls -lart' to see which was written last, I suppose.
> How do I interpret the *.log files?  e.g.:
> ===== START
> C       8083d8ecdaa2b4dcc115c366ceba9f81       
> a6593fe44785a2052ac30e5708802384
>         2009-03-31 20:02:24.000000 -0400        2009-03-31 20:02:24.000000
> -0400
>         2009-03-24 00:00:00.000000 -0400       
> db03ba7f6b2e1aa886a74182b0c8fda2
>         Meals Out               Dinner                          n      
> 1128/100
>         1128/100        1969-12-31 19:00:00.000000 -0500
> C       8083d8ecdaa2b4dcc115c366ceba9f81       
> d27cab79bfee9c48285c0208fd2b8d92
>         2009-03-31 20:02:24.000000 -0400        2009-03-31 20:02:24.000000
> -0400
>         2009-03-24 00:00:00.000000 -0400       
> ecb97484f729aece1d09b333bf0b7df6
>         Cash in Wallet          Dinner                          n      
> -1128/10
> 0       -1128/100       1969-12-31 19:00:00.000000 -0500
> ===== END
> Does the "1128/100" mean $11.28?

Yes.

> Was this on Mar. 24 or Mar 31? (not sure why
> there are two dates, and I don't remember all the transactions)

The three dates are (as I understand it) transaction date, reconcile date, and 'date entered into GnuCash'.

> While I don't want to attach the files to this ticket, I could send them to you
> if needed (as long as they don't end up posted somewhere, obviously).  

What you can try is opening the last working version, and replaying the later logs with the 'Import -> Replay" function, saving it then as a new file name, and seeing if it still corrupts the file. If it does, you may be able to narrow it down some.

Although I can successfully open the "2009.gnucash.20090331195953.xac" file, replaying the 2 changes from 2009.gnucash.20090331200013.log and saving the file results in a new file which isn't readable.  The same thing happens when I type in the changes by hand!  Even opening 2009.gnucash.20090331195953.xac, saving it as a new name, and opening that new file fails!

How should I proceed?  I could send you the 2009.gnucash.20090331195953.xac file (which opens, but gets corrupted whenever you save it).

At this point, I'm thinking of running two instances of gnucash, and just retyping all the 2009 transactions!  Any way to avoid several hours of absolute boredom doing this? :-)

Is there any way of saving EVERYTHING (or at least all transactions; I could set up the account structure and scheduled transactions by hand) as text, and replaying the whole thing into a brand new file?

Cc'ing one of the upstream GnuCash maintainers, who may have a preference as to how to handle this info.

Is anything output into /tmp/gnucash.trace?

File contents are included below.

=== started gnucash; successfully opened another file ===
* 15:40:40  WARN <gnc.app-util> /home/neil/.gnucash/config-1.8.auto:7:15: While evaluating arguments to gnc:lookup-option in expression (gnc:lookup-option gnc:*options-entries* "__gui" ...):
/home/neil/.gnucash/config-1.8.auto:7:15: Unbound variable: gnc:*options-entries*
In /home/neil/.gnucash/config-1.8.auto:
   7: 0* (let* ((option #)) ((lambda # #) option))
   7: 1* [gnc:lookup-option ...
=== loaded 2009.gnucash.20090331195953.xac here ===
* 15:41:58  WARN <gnc.backend.file> Invalid timestamp in data file.  Look for a 'trn:date-posted' entry with a date of 1969-12-31 or 1970-01-01.
* 15:41:58  WARN <gnc.backend.file> Invalid timestamp in data file.  Look for a 'trn:date-posted' entry with a date of 1969-12-31 or 1970-01-01.
* 15:41:58  WARN <gnc.backend.file> Invalid timestamp in data file.  Look for a 'trn:date-posted' entry with a date of 1969-12-31 or 1970-01-01.
=== saved as a temporary file; opened that file, got parse error dialog ===
* 15:42:32  WARN <gnc.backend.file.sx> no template account with name [b87ad7bdd7e42b408c9c61b5aaaccc60]
* 15:42:32  WARN <gnc.backend.file.sx> no template account with name [cea5efac6c071655b182dfd255af1198]
* 15:42:32  WARN <gnc.backend.file.sx> no template account with name [76c4c5e4263b4068a9ccafe81887083b]
* 15:42:32  WARN <gnc.backend.file.sx> no template account with name [1295ad57eceba823ebee79bdd2813cd7]
* 15:42:32  WARN <gnc.backend.file.sx> no template account with name [0d325cf2fa4ab4cd7a783625208d134b]
* 15:42:32  WARN <gnc.backend.file.sx> no template account with name [271c7a3cb13b54fc114b770f9085d501]

There were 3 dates in 2009.gnucash.20090331195953.xac with a value of 1969-12-31, so I changed them all (in a backup copy) to 2009-01-01.  Loading then saving then re-opening the saved file still produces the error.  So, I thin it's related to the "no template account" problem, not the "invalid timestamp" problem, although I don't know why the timestamps are invalid.

This bug appears to have been reported against 'rawhide' during the Fedora 11 development cycle.
Changing version to '11'.

More information and reason for this action is here:
http://fedoraproject.org/wiki/BugZappers/HouseKeeping

It could be related to scheduled transactions.
When I just open my file, (which adds one scheduled transaction)
and then save it, I can't open it again.

This is an old file, and I only recently started using 2.2.9 (if that helps)

thanks,

Cameron

There has been a noticed problem on Win32 with it outputting a date of "-1" which can't be parsed properly.  I'm not familiar enough with the SX system to comment on the lack of template accounts, but it does seem pretty strange.

*** Bug 509704 has been marked as a duplicate of this bug. ***

Was there ever a resolution?  I just finished installing fc11 and updated to gnucash 2.2.9, which caused the same problem.  Could cause big trouble with domestic bliss, years and years of records.

/tmp/gnucash.trace has lots of lines of the form:

* 01:06:11  INFO <gnc.commodity> [gnc_commodity_table_insert] insert 0x2acdf30 XAG into nsp=0x2cc10c0 CURRENCY

and every so often a collection like

* 01:06:11  INFO <qof.engine> [qof_event_generate_internal] id=3 hi=0x2a3c660 han=0x8dfa50 data=(nil)
* 01:06:11  INFO <qof.engine> [qof_event_generate_internal] id=2 hi=0x28a7f30 han=0x15bbf40 data=(nil)
* 01:06:11  INFO <qof.engine> [qof_event_generate_internal] id=1 hi=0x27c1970 han=0x6581ae0 data=(nil)
* 01:06:11  INFO <gnc.account> [xaccAccountCommitEdit] freeing splits for account 0x2abd000 (Template Root)
* 01:06:11  INFO <gnc.account> [xaccAccountRecomputeBalance] acct=Template Root starting baln=0/1

I also tested on ver. 2.0.5 and had the same problem.  As noted above, I can open/edit/save a recent xac file but can't open the newly saved file.

The file worked about 1 week ago on an old ubuntu box and an older version of gnucash.

Nope, no resolution that I'm aware of.  I ended up re-keying all of my 2009 data into a new file.  Files for the previous years won't resave a usable copy, but I don't need to edit them anyways.

(In reply to comment #10)
> It could be related to scheduled transactions.

Indeed.  I've noticed that some historic files from Jan 2003 and earlier, which didn't have any scheduled transactions, still open fine in 2.2.9 on Fedora 11.

FWIW, 2.2.7 on Fedora 9 opens the problematic files I had (from Jan 2004 -- Jan 2006).  I've resaved them but have kept the original files around for testing any proposed bug fix mentioned here.

(In reply to comment #15)

I also ran into problems related to scheduled transactions when I switched from gnucash 2.0.5 to 2.2.9. When I open my "old" file with gnucash 2.2.9, there is no error. However:
- if I try to edit a scheduled transaction (in the scheduled transaction editor), gnucash crashes
- if I just save my file without editing anything (including without accepting the pending scheduled transactions) and restart gnucash, I get an "error parsing file". /tmp/gnucash.trace contains several warnings "no template account with name [...]"

When I look at the XML data in the newly saved file, I see that the whole <gnc:template-transactions> block has disappeared. The problem is that this block is supposed to contain template account blocks that are referred to from the schedxaction blocks. I also noticed that the version of the schedxaction blocks changed from version 1.0.0 to 2.0.0.

I found a sort of acceptable workaround:
- open the "old" file
- open the scheduled transaction editor and delete all the scheduled transactions
- save the file and restart gnucash: the file opens without error
- re-create all the scheduled transactions.
- save the file and restart gnucash: the file opens without error

When I look at the XML data in the new file, I see that the template-transactions block is back.

So this bug looks like a backward compatibility regression.

It's pretty poor for a financial package not to be able to reliably read, say, the last six tax years' worth of information...

I have to say its more than annoying. Since some versions the files I kept working since 2002 do not open anymore. I am waiting since one year now for a fix that allows my files to be taken to the new version - still not loosing hope.
The very reason I choose gnucash was the trouble commercial software produces during upgrades.
It is NO solution for me in re-typing all of the transactions or forgetting all the transactions and starting a fresh file.

Unfortunately, I have not had time to investigate this and I don't have a file that trips it up. If you do, I suggest attaching it to the upstream bug at https://bugzilla.gnome.org/show_bug.cgi?id=573702.

This message is a reminder that Fedora 11 is nearing its end of life.
Approximately 30 (thirty) days from now Fedora will stop maintaining
and issuing updates for Fedora 11.  It is Fedora's policy to close all
bug reports from releases that are no longer maintained.  At that time
this bug will be closed as WONTFIX if it remains open with a Fedora 
'version' of '11'.

Package Maintainer: If you wish for this bug to remain open because you
plan to fix it in a currently maintained version, simply change the 'version' 
to a later Fedora version prior to Fedora 11's end of life.

Bug Reporter: Thank you for reporting this issue and we are sorry that 
we may not be able to fix it before Fedora 11 is end of life.  If you 
would still like to see this bug fixed and are able to reproduce it 
against a later version of Fedora please change the 'version' of this 
bug to the applicable version.  If you are unable to change the version, 
please add a comment here and someone will do it for you.

Although we aim to fix as many bugs as possible during every release's 
lifetime, sometimes those efforts are overtaken by events.  Often a 
more recent Fedora release includes newer upstream software that fixes 
bugs or makes them obsolete.

The process we are following is described here: 
http://fedoraproject.org/wiki/BugZappers/HouseKeeping

This will be fixed in the next 2.3.x milestone (c.f. the referenced upstream bug.)

You can work around this by just deleting all the scheduled transactions from your accounts file. At least it worked for me. Then you can manually re-enter just those, much less work than re-entering everything!

FWIW I also deleted the scheduled transactions namespace thingy at the top of the file, probably not necessary but thought I'd mention it for completeness.

(In reply to comment #22)
> You can work around this by just deleting all the scheduled transactions from
> your accounts file. At least it worked for me. Then you can manually re-enter
> just those, much less work than re-entering everything!
> 
> FWIW I also deleted the scheduled transactions namespace thingy at the top of
> the file, probably not necessary but thought I'd mention it for completeness.

Maybe I should qualify; I mean, rename your accounts file to 'accounts.gz', then 'gunzip accounts.gz' then 'vim accounts' which is an XML file and delete all the scheduled transaction tags and their guts.

This was fixed in 2.4.0 final.

gnucash-2.4.0-1.fc14 has been submitted as an update for Fedora 14.
https://admin.fedoraproject.org/updates/gnucash-2.4.0-1.fc14

gnucash-2.4.0-1.fc13 has been submitted as an update for Fedora 13.
https://admin.fedoraproject.org/updates/gnucash-2.4.0-1.fc13

gnucash-2.4.0-1.fc14 has been pushed to the Fedora 14 testing repository.  If problems still persist, please make note of it in this bug report.
 If you want to test the update, you can install it with 
 su -c 'yum --enablerepo=updates-testing update gnucash'.  You can provide feedback for this update here: https://admin.fedoraproject.org/updates/gnucash-2.4.0-1.fc14

gnucash-2.4.0-1.fc13 has been pushed to the Fedora 13 stable repository.  If problems still persist, please make note of it in this bug report.

gnucash-2.4.0-1.fc14 has been pushed to the Fedora 14 stable repository.  If problems still persist, please make note of it in this bug report.