GnuCash
Contact   Instructions
Bug 797659 - Liabilities in budget report no longer calculate correctly
Summary: Liabilities in budget report no longer calculate correctly
Status: RESOLVED FIXED
Alias: None
Product: GnuCash
Classification: Unclassified
Component: Budgets (show other bugs)
Version: 3.9
Hardware: PC All
: Normal normal
Target Milestone: ---
Assignee: core
QA Contact: core
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2020-03-30 15:35 EDT by Alex K
Modified: 2021-06-24 18:24 EDT (History)
10 users (show)

See Also:


Attachments
Special datafile with new budget (56.78 KB, application/x-gnucash)
2020-04-03 07:53 EDT, Christopher Lam
no flags Details
small datafile (14.68 KB, application/x-gnucash)
2020-04-17 11:18 EDT, Christopher Lam
no flags Details
candidate screenshot (61.21 KB, image/png)
2020-04-25 21:37 EDT, Christopher Lam
no flags Details

Description Alex K 2020-03-30 15:35:15 EDT
Since the changes to the budgeting function in 797551, the budget report no longer reports liabilities entries correctly. The budget function used to record reductions to liabilities as negative values, but now expects these numbers to be positive. However, it doesn't look like the budget report expects the numbers to be positive.

For example:
Value of $1000 in Liability line in Budget calculates as an outflow and reduces available budget in the budgeting screen.

On the budget report, once that $1000 is paid down on the liability, the budget report indicates a $2000 difference, rather than $0.
Comment 1 Alex K 2020-03-30 15:36:07 EDT
I should point out that I'm not sure if the right place to change this is in the budget report, or if the budgeting tool itself should be reverted to expect a negative value in the case of a reduction to a liability.
Comment 2 Christopher Lam 2020-03-31 06:06:15 EDT
There was plenty of discussion on https://github.com/Gnucash/gnucash/pull/630

Maybe you can write your concerns there and report whether the Liabilities signs is incorrect?

From my own testing the reports are fine.
Comment 3 Christopher Lam 2020-03-31 09:24:33 EDT
Please be aware the sign behaviour in budgets is dependent on the global Reversed Balance accounts preference. The original developer assumed, wrongly, that everyone used Credit Accounts.

From my testing there's no bug. With Credit Accounts global preference, clicking Estimate on a Liability account will pre-fill the liability budget amounts with a positive number, and this matches the reports.

Please clarify.
Comment 4 Alex K 2020-04-01 23:02:03 EDT
Hi Christopher,

My global reversed balance account preference is set to credit accounts.

So in my case, the particular account I used to identify an issue is a Mortgage liability. In the budget the intention is to record the amount that the liability will be reduced. In the past, I had this entered into the budget editor as a negative number. Both the budget totals reflected accurately, and the budget report accurately compared budget vs. actual.

With the 3.9 release, I needed to change this to a positive number in the budget editor in order to have the budget totals accurately reflect the amount remaining for budgeting. However, this then resulted in the budget report showing the budget as a positive number, the total journal activity as a negative number for the period (representing liability reductions), and the difference as the spread between the two numbers. (e.g. budget 1000, actual -1000, difference 2000).

So I guess the discrepancy is that the budget report displays a liability reduction as a negative number, whereas the budget editor reports a liability reduction as a positive number.
Comment 5 Christopher Lam 2020-04-02 01:14:56 EDT
Thank you Alex. It means the beta testing in PR 630 was incorrect.

The budget *editor* in 3.9 for liabilities then is buggy. This is rather difficult because there are very few beta testers. Are you willing to help fix it?
Comment 6 Christopher Lam 2020-04-02 07:00:29 EDT
@Alex: would you be willing to beta test on a daily build?
Comment 7 Alex K 2020-04-02 21:57:42 EDT
Hi Christopher, absolutely. I'm sorry I know my availability dropped off on that request since our initial interaction there, but my workload has dropped off since then so I'd be happy to work through to resolution on this with you!

Please let me know when you'd like me to test.
Comment 8 Christopher Lam 2020-04-03 07:12:00 EDT
@Alex: https://code.gnucash.org/builds/win32/maint/ will have the daily builds. I have just pushed the proposed fix; you will need to keep your eyes peeled and grab the one after today's 2020-04-03 03:20 build.

To test: budget amounts match actual amounts. Difference columns behave appropriately. If this iteration is correct, the "Outflow to Asset/Equity/Liability" *should* be renamed to "Outflow to Asset/Equity and Inflow from Liability" to be more exact. But then it'd make sense to split it into two rows.
Comment 9 Christopher Lam 2020-04-03 07:53:51 EDT
Created attachment 373623 [details]
Special datafile with new budget

@Alex This is another special datafile -- it is unique because it has a feature "unreversed budgets" which is planned for 4.x onwards. Please download, and delete/recreate accounts and transactions as you wish, and also test the behaviour of budgets.

This special datafile stores budgets differently and triggers a different code path for the budget editor and budget report. It will also behave better when you modify the "Reverse balanced accounts" setting to "None" or "Income & expense". Because it behaves better, it will be the default way to create budgets in 4.x onwards.

I would be *very* grateful if someone else can verify that from tomorrow onwards, the builds will handle budgets for both *current* datafiles (such as your existing file) and *future/featured* datafiles (such as this one).
Comment 10 Christopher Lam 2020-04-07 06:57:46 EDT
P.S. a 7th april build exists on https://code.gnucash.org/builds/win32/maint/ - please test using your current datafile, and using the featured datafile in comment 9 that the budget editor and the budget report both behave reasonably well.
Comment 11 Stefan Kremen 2020-04-09 16:17:59 EDT
Christopher,

I have installed gnucash-3.9-2020-04-07-git-3.9-30-g882fd22ca+.setup.exe, I opened my .gnucash file with my real life budget, which I created back in version 3.7 or even earlier. The budgetted figures in this 3.9 look exacly the same as in 3.7, so I presume the calculations are correct.

Thank you for fixing this.
Stefan
Comment 12 Christopher Lam 2020-04-09 16:50:57 EDT
@Stefan: it's still a candidate fix; need to confirm budgets from all 5 account classes behave appropriately. asset/liability, income/expense, and *equity*.

(IMHO budget-equity-assignment is wrong but someone felt it strongly enough to request this change).

In addition please confirm the budget reports are OK too.

@anyone: the *featured* book in comment 9 also would need some checking please. it handles budget differently (i.e. doesn't break when reversed account balances are changed) so will need to confirm budget+reports are unbroken please.
Comment 13 Christopher Lam 2020-04-09 21:03:47 EDT
@Stefan and @Anyone: because this fix is not yet thoroughly tested, it has been reverted for the upcoming snap release 3.10.

Please all betatest the candidate fix with your datafile and the featured book using the gnucash-3.9-2020-04-07 nightly. It is the *only* build which has the appropriate candidate fix.
Comment 14 Christopher Lam 2020-04-09 22:16:41 EDT
3.10 will be snap-released around GMT 12:00 on 11 April, so unfortunately very little time to beta test.
Comment 15 Adrien 2020-04-10 13:56:53 EDT
I just installed maint-C3.9-30-g882fd22ca-D3.9-11-g43b5edd via flatpak and tested the special data file included here.

I opened the existing budget to give it a once over. Honestly, the numbers aren't making sense. The aggregation of assets/liability/equity is confusing since they use different signs. (when they probably shouldn't) So I decided to go simple with April's column. I changed Income to "54.00" so I would really have something left to budget after the $40 expenses. So far, so good. It showed $14 remaining.

Then I budgeted to pay $10 of liabilities. (had to do so for the main placeholder, putting it on the loan did nothing - another bug, it isn't rolling up) Now it shows -$10.00 as outflows to assets/liabilities/equity and $24.00 left to budget.

The correct amounts would be +$10.00 outflows to assets/liabilities/equity and +$4.00 left to budget.

I'd keep testing, but time is short, and this is definitely not ready for prime time.

If credit accounts is the reverse sign rule, then budgeting $10 to pay a liability (just like budgeting $10 to pay an expense) should be positive and the summary should be the same sign, and it certainly shouldn't ADD to money remaining to be budgeted.

Then there is the issue that child account budgets aren't rolling up to the parent and thus aren't getting included in the summaries.
Comment 16 Adrien 2020-04-10 14:27:33 EDT
Correction: I see now the issue with the parent/child roll-up was that the parent had a specific '0.00' entered rather than being blank. Blanking the parent allows the amount to roll-up properly and be included in the summaries.

The signs still don't make sense to me though for Equity & Liabilities.
Comment 17 Christopher Lam 2020-04-10 23:54:19 EDT
@Adrien - thanks - please be aware the special datafile uses a different path in the budget viewer to calculate totals. The upgraded datafiles are not yet in use yet, so, the priority is (marginally) lower to fix them.

What about the existing datafile budgets - are they now sensible? All asset/liability/income/expense amounts total up correctly?

ISTM 3.9 and 3.10 will not handle 4.x datafile budgets properly in any case.
Comment 18 Christopher Lam 2020-04-11 10:43:54 EDT
My approach to determine whether signs mean inflow or outflow was simple:
1. Input some realistic data into book
2. Click estimate to prefill cells
3. Note the signs for 5 account types

Then after fixing budgets
4. Match the totals strategy to match above meanings for existing book
5. Match the totals strategy for future books.

It would be very useful to see the book (featured or not) and describe where expectations are not met. I still feel that the 7-April build is correct.
Comment 19 Christopher Lam 2020-04-13 21:07:14 EDT
@All:

Let's simplify the beta testing.

1) we can ignore the future datafile as attached above. (*)
2) please review the existing datafile and test budgets are as expected. We have one positive feedback only in comment 11; we really need more positive feedbacks and especially the Budget Report. Also need to confirm Estimate is prefilling the cells correctly, matching the budget report.

(*) the future datafile does simplify budgets by a large margin. We can consider budget columns to be another form of the accounting equation: all cells must total to zero /for each transaction/, which indicates the budget-to-zero is achieved. It will be easiest to see with sign-reversal=none:

- income is negative
- expense is positive
- asset is positive (saving) or negative (spending)
- liability is positive (paying back loan) or negative (when drawing from loan)
- (equity should really not be part of budget)

Perhaps instead of the 4 totals we should have 6 (or 5, ignore equity) totals to include all of above sections. Please test again.
Comment 20 Christopher Lam 2020-04-14 00:01:25 EDT
Oops:

/for each transaction/ should read /for each budget column/

The 5 (or 6) suggested totals would be:
Inflow from Income
Outflow to Expense
Outflow to Asset
Outflow to Liability
Outflow to Equity
Remaining to Budget
Comment 21 crossan007 2020-04-16 09:11:29 EDT
I just upgraded to 3.10, and I believe the budget totals look correct at this time.
Comment 22 Stefan Kremen 2020-04-16 11:57:39 EDT
Hi, I also installed 3.10 on Mac, loaded up my own file with real-world accounts.
I opened my budgets - all numbers from Inflow, Outflow and Remaining add up correctly.
I opened my budget report and again, all numbers from Budgeted, Actual and Diff add up correctly.
The version 3.10 behaves the same as back in 3.7 before this was changed.
It leads me to question why this budgeting tool was tampered with in the first place? :)

Stefan
Comment 23 Christopher Lam 2020-04-16 23:13:30 EDT
Re: 3.10 - thank you for feedback. But I still do not have confirmation that the Liability budget amounts and reports are correctly handled as per bug reporter.

Please confirm that, for Liability accounts (and possibly Equity accounts):

In Budget Viewer/Editor
- Estimate button creates appropriately signed amounts
- Budget Amount totals correctly into totals

In Budget Reports
- Liability Budget Amounts & Actuals & Diffs are not broken
Comment 24 Christopher Lam 2020-04-16 23:14:28 EDT
As to the reasoning for the changes: see thread starting at https://lists.gnucash.org/pipermail/gnucash-devel/2020-April/044823.html
Comment 25 Christopher Lam 2020-04-17 11:18:24 EDT
Created attachment 373648 [details]
small datafile

Here is a small datafile which hopefully confirms that 3.10 is still flawed and the 7-april build handles budgets correctly. In 1 month:

income $100
spend $20
pay back liability $35
remaining to budget should be $55

Beta testers, please confirm the *structure* of budget amounts to liability are similar?
Comment 26 Alex K 2020-04-21 12:44:09 EDT
Hi Christopher, I have the latest nightly I believe ...

Version: 3.900
Build ID: git 3.10-350-g3ca8fa122+(2020-04-13)

However, my budget has the following values:
Asset: $100
Expense: $100
Income: $500
Liability: -$100

I expect a "remaining to budget" of $200, but instead I have $400.

I used a negative value in Liability for years to represent a payment to a liability to reduce its total balance. This is further supported by the budget report interpreting the data this way. I.e. budget report for a liability budgeted at -$400, with a transaction debiting the liability for $400 results in a difference value of $0.
Comment 27 Alex K 2020-04-21 12:46:52 EDT
(In reply to Alex K from comment #26)
> Hi Christopher, I have the latest nightly I believe ...
> 
> Version: 3.900
> Build ID: git 3.10-350-g3ca8fa122+(2020-04-13)
> 
> However, my budget has the following values:
> Asset: $100
> Expense: $100
> Income: $500
> Liability: -$100
> 
> I expect a "remaining to budget" of $200, but instead I have $400.
> 
> I used a negative value in Liability for years to represent a payment to a
> liability to reduce its total balance. This is further supported by the
> budget report interpreting the data this way. I.e. budget report for a
> liability budgeted at -$400, with a transaction debiting the liability for
> $400 results in a difference value of $0.

Sorry Christopher, reading back realizing there is a specific build that has the proposed change, taking a look at the 7 April build now...
Comment 28 Alex K 2020-04-21 13:02:05 EDT
Confirming the "remaining to budget" value in the following build calculates as I would expect it to:

Version: 3.9
Build ID: git 3.9-30-g882fd22ca+(2020-04-06)

One comment I'll add is that the totaling table at the bottom of the budget might be a bit confusing given that asset is not signed the same way as liabilities (regardless of the user's setting) since they will always be opposite to each other, but they are grouped together for the purpose of this table. As a result, the total for this "asset/equity/liability" line is a negative/red value, when truthfully the asset is an increase (positive/black) value.

For this reason, it might make more intuitive sense to separate the Liability and equity, and label it as inflow (since a positive value in these numbers would increase the money available to budget), and separately have "Outflow to Asset" since a positive number here decreases the money available to budget.
Comment 29 Christopher Lam 2020-04-21 21:44:44 EDT
@Alex - agree the Windows 7-April build is the *only* Windows build which has the candidate fix. Very soon we can generate flatpak builds to test, but flatpak beta builds are currently not ready.

I agree the "Outflow to Asset/Liability/Equity" will be misnamed. Hence my suggestion on https://lists.gnucash.org/pipermail/gnucash-devel/2020-April/044890.html that in 3.11 onwards maybe the best totals will be:
Total Income
Total Expense
Total Equity
Total Assets
Total Liability
Remaining to Budget

Thank you for confirming the 7-April handles budget liability correctly for current books. The NEXT test is more tricky but equally needed: take the comment 9 future datafile, create some sample transactions (including income, regular expenses, paying off liability accounts), create budget, use Estimate to prefill cells, test the amounts, try budget reports.
Comment 30 Christopher Lam 2020-04-21 23:30:53 EDT
BTW Regarding the part:

I agree the "Outflow to Asset/Liability/Equity" will be misnamed. Hence my suggestion on https://lists.gnucash.org/pipermail/gnucash-devel/2020-April/044890.html that in 3.11 onwards maybe the best totals will be:
Total Income
Total Expense
Total Equity
Total Assets
Total Liability
Remaining to Budget

Obviously this is TBD...
Comment 31 Christopher Lam 2020-04-23 01:35:52 EDT
Still awaiting feedback on 7-april new-featured budget.

Warlord has now kindly offered automatic flatpak builds from my beta branch -- they will all live in: https://code.gnucash.org/builds/flatpak/christopherlam/beta/

[ ]	gnucash-beta-C3.10-27-g3d99c7520-D3.10-2-g5a09190.flatpakref	2020-04-22 01:42 	1.7K	 
[ ]	gnucash-beta-C3.10-27-g0a0357306-D3.10-2-g5a09190.flatpakref	2020-04-23 01:19 	1.7K	

Both above builds are candidate fixes; note the most useful identifier for flatpak will be the hex string after the first "-g" -- eg 0a0357306 is a build which fixes both current & featured budgets, whereas 3d99c7520 fixes current budgets only. Can beta testers check that budgets are behaving for current datafiles and featured datafiles please!
Comment 32 Christopher Lam 2020-04-25 21:37:52 EDT
Created attachment 373658 [details]
candidate screenshot

will be in next beta build cd195972b soon available at https://code.gnucash.org/builds/flatpak/christopherlam/beta/
Comment 33 Christopher Lam 2020-05-01 06:19:35 EDT
In case anyone is struggling to understand the different beta versions, for *existing* datafiles, the main aim is:

* all beta builds should handle "Remaining to budget" exactly as in 3.7
* different builds handle other total lines with some slight differences
Comment 34 Christopher Lam 2020-05-02 00:12:16 EDT
If there are any beta testers at all, the NEXT flatpak build at https://code.gnucash.org/builds/flatpak/christopherlam/beta/ after 1-May will be the candidate fix. It will restore budget 3.7 behaviour. The commit guid will be f5e2d420c.
Comment 35 Christopher Lam 2020-05-09 17:38:52 EDT
https://code.gnucash.org/builds/win32/maint/ will have builds - 5-May-2020 onwards will have the fix. Please beta test and report back.

Note You need to log in before you can comment on or make changes to this bug.