Created attachment 372971 [details] A simple QIF test file created by Quicken 2014 When I import a QIF file of transactions, it appears to duplicate transactions that are transfers from one account to another in the original Quicken database. I was able to create a simple Quicken file (2014 vintage) that illustrates the problem. I would attach the QDF file but it looks like I can attach only one file so I attached only the QIF file. I created a new Quicken database with $100 in Checking, then transferred it to Savings, then created a QIF file (attached). I ran gnucash 3.2 and imported the file, left everything default, and the result in gnucash is *two* $100 transfers. The bug is not going to affect me anymore, but the next person who imports will probably stumble over it. Probably a lot of people now need the import function because the new owner of Quicken is now charging a per-year annual fee to use it.
You can attach as many files as you want, one at a time. But don't bother attaching a QDF file, nobody here would be able to read it. The QIF you attached has two transactions. They're probably intended to be the same one but since QIF has no transaction ID field there's no way for GnuCash to know that.
I'm not intimately familiar with QIF files, but it looks like there are 3 transactions: 1) Put $100 in Checking 2) Checking: Transfer $100 to Savings 3) Savings: Receive $100 from Checking The result in gnucash is TWO entries in the Savings account transferring $100 from Checking into Savings twice for a total of $200 transferred. Are you saying there's no way to know that 2 and 3 are the same transaction and so that's why the transaction is duplicated? That's a pretty big issue when importing QIF and would be good to fix that somehow. I would think you could figure out that it's the same transaction because it's: 1) same date 2) same amount 3) both show a transfer between the same two accounts That means it's just a transfer and should be entered only once.
Created attachment 373217 [details] Perl script to remediate dup txns in QIF file Perl script strips duplicate transfer transactions from QIF file
The attached Perl script will preserve the side of a transfer in a split, but will otherwise leave out the second side of the transfer based on the order it appears in the file. It matches on account, transfer account, date and amount. It prints info on STDERR for transactions that are skipped. It worked on my relatively complex QIF file, but I am not an expert on the format and could have missed something. Use at your own risk until the bug is fixed.
Created attachment 373218 [details] Perl script to remediate dup txns in QIF file tweaked to make account name comparisons case insensitive
I can confirm there's some code to try identify inter-QIF or intra-QIF account transfers, and avoid importing them. But I'm not sure if/how/when it should trigger at all. It's all arcane.
I've just pushed a change to fix the dupe finding algorithm in QIF imports. @Paul can you please test the build (after 3.9.5) at https://code.gnucash.org/builds/win32/maint/ tomorrow or later when it's ready?
I've long since past the time I needed it, I just fixed the problems manually. I think I'll pass on testing it and hopefully somebody else will need the facility and can test it. But thanks for fixing the bug!
I was having the same issue with duplicate transfer transactions on import of QIF files. I happened upon this bug and fix while looking into the issue. I've not used bugzilla before but I would like to help test the fix. I have found the issue to be fixed, based on limited testing so far. I imported a large QIF file that had been exported from Quicken 2007. It had approx 2500 entries (counted as '^' at of records) with multiple dozens of accounts. With a unfixed GnuCash version (3.9 Linux flatpak) it would contain many (at least 100) duplicate transactions between accounts that were not matched. With a new test version (gnucash-maint-C3.9-19-g2bdc93669-D3.9-11-g43b5edd) (Linux Flatpak) the duplicates were not present. And the account balances at the end of the period, for bank accounts at least, matched the original Quicken 2007 balances, indicating the import removed the duplicates and otherwise imported OK. I do plan on continuing testing, further verifying the remainder of accounts, but may not be able to complete that for a week. Please let me know if there is any specific data, logs, or results I should gather, or specific tests to run. Should I also check the Windows version?
Thanks Patrick. Commit 96980193 was the fix. If all your tests confirm this bug to be fix, please feel free to mark it RESOLVED FIXED.
Results of testing of correction of Bug 796857 - Import of QIF duplicates transfer transactions Summary: Bug is fixed. Data imported correctly. No regression problems noticed during test. Verified on Linux and Windows. Recommend Bug 796857 be set to RESOLVED FIXED. I am new here and either have no permissions, or I just can't figure out how to mark as RESOLVED FIXED. Details: Correction was made in Commit 96980193 incorporated into GnuCash builds after 3.9.5. Test Date: 4/17/2020 Test Data (QIF files): - 1 personal large real life data export from Quicken 2007 (Accounts: Roughly >10 Bank/Cash, >20 Income, several Credit Card, >200 Expense, a few loans, >10 Assets Accounts, thousands of transactions overall) Unfortunately I only have Quicken 2007 sample data, I can't upload my test file as it is my personal financial data. I could provide screen shot comparisons of post import account summaries. GnuCash Versions used: Pre Bug baseline: 2.6.19 (Linux) rev c1b5e6c8d+ built on 2018-04-09 Pre Bugfix: 3.9-2 (Linux) gnucash-maint-C3.9-2-g0d0fddcfc-D3.9-11-g43b5edd.flatpak Post Bugfix: 3.9-19 (Linux) gnucash-maint-C3.9-19-g2bdc93669-D3.9-11-g43b5edd.flatpak Post Bugfix: 3.9-32 (Windows) gnucash-3.9-2020-04-10-git-3.9-32-g642885a18+.setup.exe Results: Pre Bugfix: 3.9-2: Balances widely off from 2.6.19 baseline. Many duplicate entries. Post Bugfix: 3.9-19 (Linux) and 3.9-32 (Windows): All balances matched 2.6.19 baseline. No duplicates noted. No regression issues noted. Thanks to Christopher for making the correction.
Thanks for testing!
