GnuCash
Contact   Instructions
Bug 797213 - import map editor shows no maps, takes ~22 seconds to display nothing, yet QIF importer remembers maps somehow
Summary: import map editor shows no maps, takes ~22 seconds to display nothing, yet QI...
Status: RESOLVED FIXED
Alias: None
Product: GnuCash
Classification: Unclassified
Component: Import - Other (show other bugs)
Version: 3.5
Hardware: PC Windows
: Normal normal
Target Milestone: ---
Assignee: import
QA Contact: import
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2019-04-24 11:00 EDT by Dale Phurrough
Modified: 2021-09-24 17:47 EDT (History)
5 users (show)

See Also:


Attachments

Description Dale Phurrough 2019-04-24 11:00:52 EDT
No maps are listed in the import map editor, yet QIF importer remembers maps.

===Setup
Windows 10
GnuCash 3.5

===Repo
1. Import various accounts and categories from Quicken via 5+ QIF files.
2. Create new accounts or change accounts during the QIF category->GC accounts step
3. Notice in later QIF files, the mappings are being remembered
4. After the lap QIF is imported, choose menu tools/import map editor

===Result
The wait icon for ~21 seconds
The import map editor appears with nothing listed.
I click on non-bayesian and Online ID. It is quick to respond. Nothing listed.
I click back to Bayesian.
Wait icon for ~22 seconds
Nothing appears.

===Expected
A list of the maps chosen/used during the QIF import
A quick responding map editor; significantly less than 22 seconds to respond.

===Notes
I don't have a file to provide as the one I discovered this behavior is my personal information and contains too much confidential data.
Until small files can be created which cause this behavior, is there any trace/logs that I could provide to assist?

This has been seen before by other users:
http://gnucash.1415818.n4.nabble.com/GNC-Import-Map-Editor-in-3-3-td4704566.html
Comment 1 Bob 2019-04-26 11:11:51 EDT
When I wrote the import map editor I did not know that the QIF maps were stored in a separate file 'qif-accounts-map' and so they are not shown.

The QIF importer is written in 'scheme' and so that file is based on it, at some point it will be re-written in c/c++ and maybe then it could be included.

Regarding the ~20 second delay, I think this is due to a conversion routine that scans all accounts for existing Bayesian KVP data with the intention to convert it to a different format. If it finds any, a flag is set so the routine would not run again but as you have no existing entries it never gets set and hence every time you open the editor it runs.
Comment 2 Dale Phurrough 2019-04-26 13:25:02 EDT
Got it. The map editor not showing QIF maps is by design. It would be an new feature enhancement to add that. I recommend that be a separate issue.

I suggest this issue then be focused on the multi-second delay. That it could be the import map KVP conversion flag (not) being set in the empty-map case.
Comment 3 Christopher Lam 2020-09-11 04:26:13 EDT
The bug here is the map-conversion is lengthy and must show heartbeat via gnc_window_set_progress. Anyone knows where the map conversion takes place?

The QIF map editor is a different issue altogether.
Comment 4 Bob 2020-09-11 08:46:44 EDT
Chris,

Look in Accounts.cpp and there is a function 'check_import_map_data'
Comment 5 Christopher Lam 2020-09-11 11:22:02 EDT
Thanks Bob. Because the import map conversion has no access to UI, it must receive a QofProgressFunc to set the progress bar. This function may also write a text mode progress message. I'll add a suggested solution onto PR #784.
Comment 6 Christopher Lam 2020-09-15 09:30:26 EDT
Thanks Bob -- this check_import_map_data function seems callable from various sources eg import-backend, dialog-imap-editor - the former has no access to UI to set progressbar window. Any ideas?
Comment 7 Christopher Lam 2020-09-15 23:08:30 EDT
^ Addendum: in short, check_import_map_data could be modified to accept a QofPercentageFunc and be called directly from UI *before* any import-map related activity. But I'm not sure the comprehensive list of UI actions which will eventually call check_import_map_data. Tools>Import Map Editor will call it, but also import-export?
Comment 8 Christopher Lam 2020-09-17 21:54:33 EDT
@dale please try nightly after 18-september https://code.gnucash.org/builds/win32/maint/?C=M;O=D likely fixed by jralls for bug 797895
Comment 9 John Ralls 2020-09-17 22:38:30 EDT
(In reply to Christopher Lam from comment #8)
> @dale please try nightly after 18-september
> https://code.gnucash.org/builds/win32/maint/?C=M;O=D likely fixed by jralls
> for bug 797895

Note that that's about the imap-editor being slow to start up, nothing about the QIF importer.
Comment 10 Dale Phurrough 2020-09-23 08:39:36 EDT
Hi, Christopher. I'm booked this week. I scheduled time to look at your change Tuesday 29th. :-)
Comment 11 Dale Phurrough 2020-09-29 17:10:42 EDT
I tested the repro steps with nightly gnucash-4.2-2020-09-29-git-4.2-4-g066632713+.setup.exe

1. Choosing menu tools/import map editor, the UI appears instantly
2. This UI defaulted to 'Bayesian' and I see two accounts. Under both those
   accounts I see many string->account maps. As I look at the individual
   maps, the Match String for the majority of them appears incorrect.
   For example, "Tuesday" is the match string for 4 maps, "Thursday" for 2,
   "DEBIT" for 4, etc. Are you confident the data being displayed in that
   column is meaningful?
3. Click 'non-bayesian' the UI instantly responds. I see nothing listed.
4. Click 'online' the UI instantly responds. I see 12 entries.
   Five of the entries are paired duplicates. Two are unique.
   All of them have values/maps that appear correct.
5. Click 'bayesian' again the UI appears instantly.

As an example of 4, my 'online' list has two entries for an IRA account I have
at Fidelity Investments. The two entries are:

Account Name         Based On     Match String             Mapped to Account Name
Fidelity:IRA:Cash    Online Id    fidelity.com 123456789   Fidelity:IRA:Cash
Fidelity:IRA:Cash    Online HBCI  fidelity.com,123456789   Fidelity:IRA:Cash

Notice the only differences are:
1. the Based On "Id" or "HBCI"
2. The Match string replaces a space with a comma


Overall, my experience of the speed in displaying the 'bayesian' maps is resolved. It appears almost instantly and I see some entries (even if those
entries are curious).

Note You need to log in before you can comment on or make changes to this bug.