When running report: === Report error An error occurred while running the report. === GL is in CAD, stocks being held are USD. Account structure is: Assets (CAD) > Investments (CAD) >> Trading Account (CAD) >>> USD Holdings (USD) >>>> MSFT (stock account with commodity type MSFT) >>>> VTI (stock account with commodity type VTI) When running the balance sheet or chart of accounts, the error above is displayed. If running a reduced balance sheet account with only individual accounts selected, it generally works... unless this happens: === [2019.05.10 20:25:12] <CDB-Man> hmm, in the account tree view, it looks like I can use the "Present (CAD)" column to see what has no $ values assigned [2019.05.10 20:34:01] <CDB-Man> warlord: microsoft currently doesnt have an issue, but VTI does, see screenshot: https://i.imgur.com/1FHAURk.png [2019.05.10 20:34:37] <CDB-Man> if I retype the value in priceDB, it fixes the $ value from null/blank to the current value [2019.05.10 20:35:56] <CDB-Man> despite correcting that one though, there still seems to be more hidden somewhere... [2019.05.10 21:00:32] <CDB-Man> so far, it seems only USD stocks have issues (my ledger is in CAD) [2019.05.10 21:07:00] <CDB-Man> hmm, it looks like the balance seheet report fails if "show exchange rates" is checked [2019.05.10 21:07:06] <CDB-Man> so SOMETHING has a broken FX rate === Looking at tracefile, it looks like there is an invalid argument error with a backtick causing the issue, see tracefile below: === * 21:14:42 WARN <gnc.scm> In ice-9/eval-string.scm: 44: 19 [read-and-eval #<input: string 115a4ee0> #:lang ...] 37: 18 [lp (gnc:report-run 9)] In report.scm: 784: 17 [gnc:report-run 9] In c-interface.scm: 70: 16 [gnc:backtrace-if-exception #<procedure 1178e498 at report.scm:785:5 ()>] 22: 15 [gnc:call-with-error-handling #<procedure 1178e498 at report.scm:785:5 ()> ...] In ice-9/boot-9.scm: 160: 14 [catch #t #<procedure 1178e480 at c-interface.scm:23:8 ()> ...] In c-interface.scm: 27: 13 [#<procedure 1178e480 at c-interface.scm:23:8 ()>] In report.scm: 788: 12 [#<procedure 1178e498 at report.scm:785:5 ()>] 762: 11 [gnc:report-render-html # #t] In balance-sheet.scm: 640: 10 [balance-sheet-renderer # #f #f] In html-acct-table.scm: 536: 9 [gnc:make-html-acct-table/env/accts (# # # # ...) (# # # # ...)] 928: 8 [gnc:html-acct-table-add-accounts! # #] 773: 7 [traverse-accounts! (# # # # ...) 0 0 ...] In srfi/srfi-1.scm: 613: 6 [for-each #<procedure 115c6f60 at html-acct-table.scm:774:9 (acct)> #] In html-acct-table.scm: 775: 5 [#<procedure 115c6f60 at html-acct-table.scm:774:9 (acct)> #] In commodity-utilities.scm: 987: 4 [gnc:sum-collector-commodity #<procedure 11767520 at report-utilities.scm:355:4 (action commodity amount)> ...] In srfi/srfi-1.scm: 573: 3 [map #<procedure eeae070 at report-utilities.scm:335:7 (pair)> (# # # ...)] In commodity-utilities.scm: 997: 2 [#<procedure f0d1990 at commodity-utilities.scm:989:10 (curr val)> # 1] 848: 1 [gnc:exchange-by-pricedb-nearest # # 1577854799] In unknown file: ?: 0 [gnc-pricedb-convert-balance-nearest-price-t64 # 1 # ...] ERROR: Throw to key `numerical-overflow' with args `("/" "Numerical overflow" #f #f)'.
The error is displayed when "show exchange rates" is selected and there's a USD stock included in the selected accounts.
Created attachment 373276 [details] Tracefile Below occurs when running a report involving USD stocks, such as balance sheet, and the checkbox to show exchange rates is enabled: === ERROR: Throw to key `numerical-overflow' with args `("/" "Numerical overflow" #f #f)'. * 21:43:27 WARN <gnc.scm> In ice-9/boot-9.scm: 160: 19 [catch #t #<procedure 1125c5e8 at c-interface.scm:23:8 ()> ...] In c-interface.scm: 29: 18 [#<procedure 1125c5e8 at c-interface.scm:23:8 ()>] In unknown file: ?: 17 [eval-string "(gnc:report-run 93)" #<undefined>] ?: 16 [call-with-input-string "(gnc:report-run 93)" ...] In ice-9/boot-9.scm: 2412: 15 [save-module-excursion #<procedure 1125c5a0 at ice-9/eval-string.scm:65:9 ()>] In ice-9/eval-string.scm: 44: 14 [read-and-eval #<input: string 11445cb0> #:lang ...] 37: 13 [lp (gnc:report-run 93)] In report.scm: 784: 12 [gnc:report-run 93] In c-interface.scm: 70: 11 [gnc:backtrace-if-exception #<procedure 1125c4b0 at report.scm:785:5 ()>] 22: 10 [gnc:call-with-error-handling #<procedure 1125c4b0 at report.scm:785:5 ()> ...] In ice-9/boot-9.scm: 160: 9 [catch #t #<procedure 1125c498 at c-interface.scm:23:8 ()> ...] In c-interface.scm: 27: 8 [#<procedure 1125c498 at c-interface.scm:23:8 ()>] In report.scm: 788: 7 [#<procedure 1125c4b0 at report.scm:785:5 ()>] 762: 6 [gnc:report-render-html # #t] In balance-sheet.scm: 731: 5 [balance-sheet-renderer # #f #f] In html-utilities.scm: 748: 4 [gnc:html-make-exchangerates # ...] In srfi/srfi-1.scm: 613: 3 [for-each #<procedure 2890e0a8 at html-utilities.scm:749:7 (commodity)> #] In html-utilities.scm: 750: 2 [#<procedure 2890e0a8 at html-utilities.scm:749:7 (commodity)> #] In commodity-utilities.scm: 848: 1 [gnc:exchange-by-pricedb-nearest # # 1557547199] In unknown file: ?: 0 [gnc-pricedb-convert-balance-nearest-price-t64 # 1 # ...] === Below occurs when I run Finance::Quote by clicking "get quotes" in the pricedb view ==== * 21:46:41 WARN <gnc.gui> [PrintAmountInternal()] Bad numeric: GNC_ERROR_ARG. * 21:46:41 WARN <gnc.gui> [PrintAmountInternal()] Bad numeric: GNC_ERROR_ARG.
May I suggest running the latest maint from https://code.gnucash.org/builds/win32/maint/ -- the "$" display is odd. Secondly try adding a line in commodity-utilities.scm as follows (the + line) and reattach tracefile. (define (gnc:exchange-by-pricedb-nearest foreign domestic date) + (gnc:pk 'gnc:exchange-by-pricedb-nearest foreign domestic date) (and (record? foreign) (gnc:gnc-monetary? foreign) date
Added the extra line and ran Balance Sheet with "show exchange rates" enabled, tracefile outputs this. Downloading the nightly. === * 21:37:30 WARN <gnc.scm> In ice-9/boot-9.scm: 160: 19 [catch #t #<procedure f3c6ea0 at c-interface.scm:23:8 ()> ...] In c-interface.scm: 29: 18 [#<procedure f3c6ea0 at c-interface.scm:23:8 ()>] In unknown file: ?: 17 [eval-string "(gnc:report-run 13)" #<undefined>] ?: 16 [call-with-input-string "(gnc:report-run 13)" ...] In ice-9/boot-9.scm: 2412: 15 [save-module-excursion #<procedure f3c6e58 at ice-9/eval-string.scm:65:9 ()>] In ice-9/eval-string.scm: 44: 14 [read-and-eval #<input: string fac1b60> #:lang ...] 37: 13 [lp (gnc:report-run 13)] In report.scm: 784: 12 [gnc:report-run 13] In c-interface.scm: 70: 11 [gnc:backtrace-if-exception #<procedure f3c6d68 at report.scm:785:5 ()>] 22: 10 [gnc:call-with-error-handling #<procedure f3c6d68 at report.scm:785:5 ()> ...] In ice-9/boot-9.scm: 160: 9 [catch #t #<procedure f3c6d50 at c-interface.scm:23:8 ()> ...] In c-interface.scm: 27: 8 [#<procedure f3c6d50 at c-interface.scm:23:8 ()>] In report.scm: 788: 7 [#<procedure f3c6d68 at report.scm:785:5 ()>] 762: 6 [gnc:report-render-html # #t] In balance-sheet.scm: 731: 5 [balance-sheet-renderer # #f #f] In html-utilities.scm: 748: 4 [gnc:html-make-exchangerates # ...] In srfi/srfi-1.scm: 613: 3 [for-each #<procedure f47d8e8 at html-utilities.scm:749:7 (commodity)> #] In html-utilities.scm: 750: 2 [#<procedure f47d8e8 at html-utilities.scm:749:7 (commodity)> #] In commodity-utilities.scm: 849: 1 [gnc:exchange-by-pricedb-nearest # # 1557633599] In unknown file: ?: 0 [gnc-pricedb-convert-balance-nearest-price-t64 # 1 # ...] ERROR: Throw to key `numerical-overflow' with args `("/" "Numerical overflow" #f #f)'.
Created attachment 373277 [details] Tracefile after GetQuote When I run "Get Quotes" in the PriceDB view, I see this occur for 520 lines: * 21:42:54 WARN <gnc.gui> [PrintAmountInternal()] Bad numeric: GNC_ERROR_ARG. See attached.
Created attachment 373278 [details] #3 Tracefile This tracefile is with gnucash --debug --extra parameters. Actions taken: 1. Start GunCash 2. Run Balance Sheet report with show exchange rate enabled (error) 3. Run Balance Sheet report with show exchange rate disabled (error) * May have reversed the order of #2 and #3 4. Manually edit the pricedb entry for ticker SPY, to retype the value. After doing this, the account tree's value for my SPY account goes from "$" (null) to $3864 5. Run Balance Sheet report with show exchange rate enabled (error) 6. Run Balance Sheet report with show exchange rate disabled (report works) * May have reversed the order of #5 and #6
Created attachment 373279 [details] PriceDB extract [2019.05.11 21:43:47] -*- chris still thinks your pricedb is borked. I think gnc-pricedb-convert-balance-nearest-price-t64 where / is failing [2019.05.11 21:44:17] <CDB-Man> when I run Get Quotes in pricedb view, I see this a million times in tracefile: [2019.05.11 21:44:17] <CDB-Man> * 21:42:54 WARN <gnc.gui> [PrintAmountInternal()] Bad numeric: GNC_ERROR_ARG. [2019.05.11 21:44:44] <CDB-Man> no other error or output [2019.05.11 21:45:44] <fell> CDB-Man: if you run gnucash --debug --extra and look in the tracefile? [2019.05.11 21:47:12] <CDB-Man> will do that next. [2019.05.11 21:49:29] <CDB-Man> hmm, other prices are broken now... [2019.05.11 21:50:41] <CDB-Man> uploading another tracefile [2019.05.11 21:53:22] <CDB-Man> uploaded [2019.05.11 21:53:40] <fell> and mime changed :-) [2019.05.11 21:54:59] <fell> if bugzilla does not recognize the format, it sets octet stream. [2019.05.11 21:55:11] <CDB-Man> i hope my pricedb isnt corrupted like chris thinks... and that if it is corrupted, that its recoverable [2019.05.11 21:55:38] <CDB-Man> 99% of the entries on my pricedb are from finance quote, and the remainder 1% come form transactions where i specified a rate at the transaction level [2019.05.11 21:56:05] <chris> well your step 4 suggests something wrong in your pricedb [2019.05.11 21:56:50] <CDB-Man> yeah, but how to pinpoint? [2019.05.11 21:57:29] <CDB-Man> i've thought about using the remove all old prices option, but then i would like all the historic data [2019.05.11 21:58:38] <chris> I suggest save as XML uncompressed, open in editor, copy all pricedb into new file, save as compressed, and upload. prices won't have your transactional data. and give us a clue which price entry was fixable in your step 4. [2019.05.11 22:00:22] <CDB-Man> hmm, [2019.05.11 22:00:36] <chris> and also upload some history about which builds you were using, and approximately if/when you upgrade your build. [2019.05.11 22:02:10] <CDB-Man> version would be 3.3 where I had no issues, and after thre upgrade to 3.5+ where I started having issues [2019.05.11 22:03:25] <CDB-Man> regarding XML, you're saying tha tthe default .gnucash is compressed XML, so what's the best way to decompressing it for me to extract pricedb? [2019.05.11 22:03:41] <chris> well gtg, do the above. it's gzip - use any freeware eg 7-zip [2019.05.11 22:06:17] <CDB-Man> that worked [2019.05.11 22:15:50] <CDB-Man> hmmm..... [2019.05.11 22:15:51] <CDB-Man> <price:value>690402841721574/4679745419383</price:value> === So it seems that some prices being pulled by Finance::Quote are overly precise, and this might be causing the issue. Attached is the priceDB protion of my .gnucash file, with snippets below of the current prices giving me trouble (SPY, VTI). ==== <price> <price:id type="guid">56864ce89913414296f3cd968117c175</price:id> <price:commodity> <cmdty:space>NYSE</cmdty:space> <cmdty:id>SPY</cmdty:id> </price:commodity> <price:currency> <cmdty:space>CURRENCY</cmdty:space> <cmdty:id>USD</cmdty:id> </price:currency> <price:time> <ts:date>2019-05-10 16:00:00</ts:date> </price:time> <price:source>Finance::Quote</price:source> <price:type>last</price:type> <price:value>1213698188994648/4212767056559</price:value> </price> ==== <price> <price:id type="guid">50657a1b214742068ab17c2bdac082ae</price:id> <price:commodity> <cmdty:space>NYSE</cmdty:space> <cmdty:id>VTI</cmdty:id> </price:commodity> <price:currency> <cmdty:space>CURRENCY</cmdty:space> <cmdty:id>USD</cmdty:id> </price:currency> <price:time> <ts:date>2019-05-10 16:00:00</ts:date> </price:time> <price:source>Finance::Quote</price:source> <price:type>last</price:type> <price:value>690402841721574/4679745419383</price:value> </price>
[2019.05.12 05:13:52] <gncbot> chris: Sent 6 hours and 49 minutes ago: <CDB-Man> uploaded my pricedb extract to the bug report, seems your suspicion about price corruption is right, finance::quote is storing overly precise numbers such as <price:value>690402841721574/4679745419383</price:value> that it seems gnucash is choking on, see https://bugs.gnucash.org/show_bug.cgi?id=797233#c7 [2019.05.12 05:14:35] <chris> CDB-Man: you'll have to ask jralls for a price-scrubbing addition to the file-load mechanism. [2019.05.12 05:37:11] <chris> CDB-Man: if you load a recent win32 build, new pricedb entries should be saner. I don't know what to do about your current pricedb entries; this should be answerable by jralls. Perhaps jralls can provide some insight.
Comment on attachment 373276 [details] Tracefile Hm, bugzilla does not understand MIME text/x-log. CDB-Man, can you adjust the MIME Type to text/plain in the future instead of octet/stream?
@Frank: Noted -- I had relied on the "auto detect" to pick the right file type, looks like that didn't work so well. Thanks for re-uploading.
CDB-Man: while dev resources are in very short supply it would be perhaps best that you fix your pricedb manually meanwhile. 690402841721574/4679745419383 can be replaced to 14753/100 exactly. There was a recent commit to fix something in quote so this bug may have been fixed already: https://github.com/Gnucash/gnucash/commit/95aed3032 but it won't fix existing pricedb entries.
Chris is right that Mike Alexander's commit should prevent goofy numbers like from FQ going forward. Since you're on Windows you can just grab a recent nightly. If you fix the three really big numbers does your file work again? I'm not sure that I can construct a good test to safely scrub prices like that out of a file.
Thanks jralls; after manually scrubbing pricedb of approx 50 entries with unusually long values, all the reports work. I went to maint > https://code.gnucash.org/builds/win32/maint/ And I'll download the following nightly: > gnucash-3.900-2019-04-25-git-3.5-106-g0cf49f124+.setup.exe I'll run Finance::Quote on that and see if it keeps pricedb clean. === On an unrelated matter, can we delete the tracefile attached to message #6? > https://bugs.gnucash.org/show_bug.cgi?id=797233#c6 Since it was ran with --debug --extra it actually contains all my account names and a bunch of other data.
Comment on attachment 373278 [details] #3 Tracefile >> Deleted <<
More specifically https://bugs.gnucash.org/attachment.cgi?id=373278 attachment 373278 [details], some of the GnuCash account names have partial actual account numbers.
Nightly gnucash-3.900-2019-04-25-git-3.5-106-g0cf49f124+.setup.exe still returns goofy numbers such as the below: <price:value>98421868895636/2267262586861</price:value> I'll hold off on running Finance::Quote until we get a bit more clarity on this.
There are newer nightlies. Check the last-modified dates.
I see it now, a bit misleading that the April file is 3.900-xxx where as the May file is 3.5-xxx. I ran: > gnucash-3.5-2019-05-10-git-3.5-170-gd62489409+.setup.exe And it still generated this: ==== <price> <price:id type="guid">fa09f2bf4759469fa6fc84d4afd17cde</price:id> <price:commodity> <cmdty:space>NASDAQ</cmdty:space> <cmdty:id>ATVI</cmdty:id> </price:commodity> <price:currency> <cmdty:space>CURRENCY</cmdty:space> <cmdty:id>USD</cmdty:id> </price:currency> <price:time> <ts:date>2019-05-13 16:00:00</ts:date> </price:time> <price:source>Finance::Quote</price:source> <price:type>last</price:type> <price:value>98421868895636/2267262586861</price:value> </price> ==== Looks like it's still an issue; don't think I'm missing out on a newer version this time. :) ==== > On an unrelated matter, can we delete the tracefile attached to message #6? > https://bugs.gnucash.org/show_bug.cgi?id=797233#c6 > Since it was ran with --debug --extra it actually contains all my account names and a bunch of other data. As a regular user, it looks like I can only mark things obsolete, not delete them.
> I see it now, a bit misleading that the April file is 3.900-xxx where as the May file is 3.5-xxx. That's my fault. I merged a PR that was on master into maint and didn't notice. > As a regular user, it looks like I can only mark things obsolete, not delete them. That's not just you. BZ doesn't allow deleting anything. Mrmph. 43.41. And >>> from decimal import Decimal >>> Decimal(98421868895636)/Decimal(2267262586861) Decimal('43.40999999999999558939486853')
No worries re: the versioning. Looking forward to a later nightly where the fix is in place. Regarding deleting attachments, doing so Google searching, I found this: https://devzing.com/blog/index.php/bugzilla-tip-deleting-attachment/ It details how administrators can be granted deletion rights on Bugzilla.
to help reproduce this bug on recent windows, it *will* be useful to know the exact sources for the quotes used.
All stock prices come from the Yahoo JSON option. Currency pricing from AlphaVantage.
The content of attachment 373278 [details] has been deleted for the following reason: User Request
The problem turns out to be that Mike Alexander misunderstood what Scheme's rationalize and inexact->exact functions do. If the number is floating point the damage is already done, so the fix is to have gnc-fq-helper tag the price with #e. See inexact->exact in https://www.gnu.org/software/guile/manual/guile.html#Exactness. I've pushed the change. If you want to test without getting a new build just change line line 189 in gnc-fq-helper from if($numstr =~ /^\s*(\d+(\.\d+)?([eE][+-]?\d+)?)$/o) { return $1; } else { to if($numstr =~ /^\s*(\d+(\.\d+)?([eE][+-]?\d+)?)$/o) { return "#e" . $1; } else { Chris, is there anywhere else you can think of where we might be inadvertently putting floating-point values into Scheme?
Jralls, that worked. After running FQ again and checking the raw priceDB data, no goofy values were inserted.
@jralls none in my travels across schemeland.
CDB-Man, shall we declare this fixed? Chris, OK. Something of which to be mindful.
Indeed, sounds fixed with the latest commit that you've pushed.
*** Bug 797311 has been marked as a duplicate of this bug. ***