Created attachment 373818 [details] Sample of GnuCash Gnucash version: 4.1 (and 3.8, so this is not a very new problem) Steps to reproduce: 1) Start QFX import for an account I have never imported before. 2) Go through the "Select security/currency" dialogs to match the securities in the QFX file to GnuCash securities. 3) Select the GnuCash account that corresponds to the overall brokerage account. 4) Select the GnuCash account that corresponds to the first "Stock account for security XXXX" in the QFX. At this point GnuCash hangs for quite a while, using a 100% of one core. Sampling GnuCash at this point shows that all the time is under `LibofxContext::transactionCallback`. I'll attach the sampling output, in case it's useful. I had thought that Gnucash hung permanently, but it just came back while I was typing this up. I measured, and it takes about 9 minutes for Gnucash to become responsive again. This is on a fairly new (late 2019) high-end Macbook Pro. I assume there's something about the account structure or the specific QFX file here that leads to this; presumably not all QFX imports are this slow... I'm not attaching any info on those points because I'd prefer not to share various financial information if I can avoid it, but if having specific information about those would be helpful I'm happy to work out what information needs to be provided and how to sanitize it.
I tried to keep going and put in the other accounts that were relevant. I got more multi-minute hangs, then after I finished matching up the last account I got a hang that's lasted an hour so far. At that point I just killed GnuCash...
How many transactions are already in the import destination accounts and how many transactions in the imports? It looks from the spin report that it's bogging down in the duplicate transaction queries. We just merged https://github.com/Gnucash/gnucash/pull/721 that addresses exactly that problem.
There were 0 transactions in the destination accounts that are being imported into: these are new accounts at a new (to me) financial institution that only provides transaction history in QFX, which is why I was trying the importer. There are 28 transactions in the QFX file being imported: 7 transactions each for 4 different accounts. Would the number of transactions in the book outside the destination accounts matter? There are quite a lot of those. I've noticed the duplicate queries being pretty slow, though not nearly this slow, when doing QIF imports too, though those are usually into accounts with a bunch of existing transactions for me... If there's a place I can download a binary that includes https://github.com/Gnucash/gnucash/pull/721 I'd be happy to test how it behaves with my steps above. I could try building from source too, I guess, if really needed....
Hmm, 28 transactions is pretty small, and if it's really a new account the query should be instantaneous: The PR 721 changes shouldn't matter at all. Sorry, we do nightly builds only for Linux Flatpak and Windows.
A closer look at your spin dump shows that the actual time is spent in check_import_map_data rather than processing the imported transactions. That's bug 797833. *** This bug has been marked as a duplicate of bug 797833 ***
On second thought, no, bug 797833 is different. It's related to bug 797945, but not entirely a duplicate of that either.
It turned out to be an easy fix after profiling showed that the delay was mostly due to editing and committing every account with KVP slots, even those that aren't Bayes Import Maps. That gets expensive in a large account tree. Simply checking the import map before opening the account for editing changed the time required for the test account on bug 797945 from 96 seconds to effectively instantaneous--as in I can't find any trace of the operation in the profiler. The fix will be in GnuCash 4.2 which we'll release at the end of the month.
