Created attachment 177864 [details] XML GNUCash file showing reported error Asset Barchart Report (period: month) calculates also transactions from the first day of the next month. For example, if I want to see what is value of my assets at the end of september, I get value of my assets at the end of september including transactions from the first day (expense or income).
Still true in release 2.4.7 for Windows (in Windows XP). Check the Show table option to see the problem more clearly. If the interval is set to month and the first date is December 31, 2009, the next date is January 31, 2010, then following dates are March 3, April 3, etc. In the following year, they continue on the third. If the first date is December 28, the following dates are all the 28th. It seems to be using the previous month's report date instead of the report beginning date to set following dates. There should be an option to select last day of the month or of the accounting period when appropriate. Perhaps the day before the first day of the following period would give even better results. This would still work correctly if hypothetical days were added between accounting periods to solve the 'last day problem' sometimes brought up in other discussions. Not surprisingly, the Liability Barchart report has the same problem. In the case of the net worth barchart, if started on Feb 28, 2011 and ended on the end of the accounting period, it only shows one date, December 31st 2010. It does not seem to be checking for the correct accounting period of the start date either. I do not know whether there may be other reports using the same interval selection code. That should be enough fodder to chew on for this bug report.
I found more. In The net worth report: If the start date is set to February 28, 2009, monthly interval selected, and end of accounting period selected as the last date, then no assets are reported in months before January 2010. Curiously, liabilities are reported. For completeness, I should also mention that I have not changed my accounting period from the program default. David Carlson
Oops, with those start and end end dates chosen, the report shows data through December 2010, which would not be the end of the accounting period containing the start date, but the end of the accounting period before the current period (I think). David
@Andrey: I tried this on 2.5.0 (Ubuntu 12.04) and I can confirm the behaviour reported by David in comment #1. So my guess is that you specified 2011-09-01 until 2011-09-31, and because there is no 2011-09-31 it is translated by GnuCash to 2011-10-01. Is that correct?
In my attached file I have several transactions: 31.12.2010 Income 10000; 31.12.2010 Payed smth 2000; 01.01.2011 Payed smth 4000; 01.01.2011 Income 5000; 09.01.2011 Payed smth 1000. So I make assets report from 01.12.2010 till 31.01.2011 and want to see the following: on 31.12.2010 - 8000; on 31.01.2011 - 8000. But I see on 31.12.2010 - 9000; on 31.01.2011 - 8000. So, I think, yes, there is some problem with how GnuCash translates "1 month" period. It seems that it considers not a 1 calendar month, but smth like 30 days (or somehow like that).
Created attachment 242827 [details] Screenshot GnuCash 2.4.10 (Ubuntu 12.04) I attached a screenshot how it appears to me. Please correct me if I am missing something. As you can see from there, I activated the display of the table that shows which value are used for the graph. It starts with the 01-12-2010, the first day of the reporting period, and it states 0 RUB for the balance, which is correct to me. Then it jumps one month to the 01-01-2011 and shows a balance of 9000 RUB on that day, which looks correct, too. Then it notices, that the next step is smaller than a month, so it takes the last day of the user defined reporting period, here for me 18-01-2011 (for your case it should be 31-01-2011, and reports a balance of 8000 on that day. Also this behaviour appear ok to me. You can see the account that I selected for this report. If I read it correctly, on 31-12-2010 it has a balance of 9000 RUB, achieved by a transfer from the Income account. Why should it show 8000 RUB on the report for that day?
Created attachment 242882 [details] XML GNUCash file showing reported error
Created attachment 242883 [details] Screenshot GNUCash 2.4.11 on Windows 7
All that you have written is correct. But I think it should show 8000 RUB on the second bar, because when I select a period of one month, my opinion is that is should be a calendar month. I tried now another thing. If I make a report which starts on 31.12.2010, then everything seems ok. But then I went further and added several transactions (see new gnucash file and screenshot). For February it shows the date 03.03.2011 and because of that the wrong assets value by the end of February (because it is not the end of February) - am I missing smth? So I think that the main problem is in how GNUCash considers this "month" period - it should be a calendar period, not some amount of days.
I agree with Andrey. If the interval for any report is set to Month, Quarter or year then the starting and ending dates for each interval should land on the same date throughout the chart, even in leap years. If it is set to Week, it should always start/end on the same weekday and only if it is set to days should it actually count days. Technically, I guess it might have a problem before the Gregorian calendar was universally adopted, but most of us do not go that far back in our data.
I did not put my mind completely into gear before typing. In reports that have step size (interval) options I want to see the same date on each interval line unless the date is the last day of each month (or last weekday if that ever becomes an option) when month, quarter, half-year or year step size (interval) are chosen. It is not clear in the options whether the values shown are ending values or beginning values. I would expect that they are values at the end of the local day (i.e. not Universal time) on the dates shown. The report time zone should be shown on the report. This alludes to the issue that I brought up in my comment #1. If one actually wants to see interval ending values, an option needs to be added to the report to show interval end dates and end values instead of end of first day values.
I am currently using GnuCash 2.4.12 svn r22912 in Windows 7
I now believe that this bug should be split into two or three separate bugs. 1. Report should be clear about whether it is reporting starting values or ending values. If starting values is selected, should the identifying dates be first dates of each period or dates preceding the first dates with the possible exception of the last reported date. Similarly, if ending values is selected, should the identifying dates be first dates of each period or dates preceding the first dates with the possible exception of the last reported date. An explanation about how dates are selected could resolve questions about first an last dates shown in reports. 2. Reports should respect the varying lengths of months so that bug 1 can be meaningfully resolved with logically consistent dates evaluated and displayed. This is probably the place to include 'last day of previous period' as a starting option. 3. Report should show the time zone used for it's calculations. Does anyone have something to add to or refute this comment?
I totally agree, and I was about to answer something similar. To 1. This should be an enhancement bug. Reason: As far as I know, there is no real baseline to report a fault against. Also, I am quite ok with what the report is doing right now, so I would expect some discussion about this point. Actually, I am thinking about if Wiki would be good place for this discussion. The reports are sometimes and very shortly described in http://wiki.gnucash.org/wiki/Using_GnuCash. This page could be split into sub-pages, where each page could take one report, and each page can be discussed seperately. (yes, that would mean that each report to be discussed should be properly document beforehand in this Wiki :-) ) To 2. This is in my view exactly what this but is about. Once this fixed, this bug should be closed. To 3. This one should again be an enhancement bug, and I would add: - show a whole date and time string at the buttom of the report, showing the creation date - show a version of the report at the buttom of the report (yes, this means to introduce a version per report) - show the GnuCash version at the buttom of the report. By having this info, it will be easier to handle old saved/exported reports that might be used for historical investigations.
Per this discussion I have opened Bug 699430 detailing a suggested description of this report to be added either to the Wiki or the Help manual. The suggested description refers to this bug report when describing the odd 'date slip' behavior when a step falls on a 'missing' calendar date. Once that is done, the description can be copied to the other barchart report descriptions with appropriate changes.
Created attachment 314918 [details] Patch: Fix selection of intermediate dates that get reported See mailing list thread http://lists.gnucash.org/pipermail/gnucash-user/2015-October/062400.html
Comment on attachment 314918 [details] Patch: Fix selection of intermediate dates that get reported Comment on attachment 314918 [details] Patch: Fix selection of intermediate dates that get reported The current behavior is correct: When constructing an interval for an income, expense, or net-profit bar chart one wants the transactions for the start-date of the interval included, so start-day-time is appropriate. When computing the balance as of a particular day for an asset, liability, or net-worth bar chart one wants the transactions for that day included, so end-day-time is correct. This change would exclude some transactions in the latter case because start-of-day means in the current timezone, and in many countries that changes by an hour twice a year. That means in winter time, all transactions entered during summer time look like they were created at 01:00:00, so they would be left out of the balance.
(In reply to David Carlson from comment #13) > I now believe that this bug should be split into two or three separate bugs. > > 1. Report should be clear about whether it is reporting starting values or > ending values. If starting values is selected, should the identifying dates > be first dates of each period or dates preceding the first dates with the > possible exception of the last reported date. Similarly, if ending values > is selected, should the identifying dates be first dates of each period or > dates preceding the first dates with the possible exception of the last > reported date. An explanation about how dates are selected could resolve > questions about first an last dates shown in reports. > > 2. Reports should respect the varying lengths of months so that bug 1 can be > meaningfully resolved with logically consistent dates evaluated and > displayed. This is probably the place to include 'last day of previous > period' as a starting option. > > 3. Report should show the time zone used for it's calculations. > > Does anyone have something to add to or refute this comment? Asset, Liability, and Net Worth reports include transactions on the day indicated in the report. This is documented in e.g. http://gnucash.org/docs/v2.6/C/gnucash-help/report-assets.html: "The first bar shows the selected values at the end of the day on the first date chosen." Time zone is the one in effect on the computer at the time the report is created. This is a bit of a problem with the way transactions are recorded because if GnuCash is used in a different time zone from the one transactions in which transactions were entered the transaction date can change, see bug 137017. The problem isn't that the intervals are wrong, it's that when the reports were designed the author assumed that reports would always start at the beginning of a month so it always uses the number of days of the current month to calculate the date in the next month, which blows up for dates at the end of months. The interval function needs to notice that condition and add the number of days in the *next* month to the interval date in that case. But that's still not sufficient, because if you give 28 Feb as a start date GnuCash has to decide whether you want the 28th of each month or the last day of each month, so we'd need either to assume that if you start on the last day of a month you mean last day of every month or provide a control to specify. I'd be inclined to just assume.
I see that David correctly analyzed the situation in comment 1. (In reply to David Carlson from comment #2) > I found more. > > In The net worth report: > > If the start date is set to February 28, 2009, monthly interval selected, > and end of accounting period selected as the last date, then no assets are > reported in months before January 2010. Curiously, liabilities are reported. That's because the first transaction is after 28 Dec 2010. When I check the net-worth chart in 2.6.9 I don't get any liabilities in the first period. (In reply to David Carlson from comment #3) > Oops, with those start and end end dates chosen, the report shows data > through December 2010, which would not be the end of the accounting period > containing the start date, but the end of the accounting period before the > current period (I think). The accounting period selectors are just shorthand for dates; they have the advantage of being updated every time you run the report so that when you run a saved report using them you don't need to fiddle the dates.
(In reply to John Ralls from comment #18) > (In reply to David Carlson from comment #13) > > I now believe that this bug should be split into two or three separate bugs. > > > > 1. Report should be clear about whether it is reporting starting values or > > ending values. If starting values is selected, should the identifying dates > > be first dates of each period or dates preceding the first dates with the > > possible exception of the last reported date. Similarly, if ending values > > is selected, should the identifying dates be first dates of each period or > > dates preceding the first dates with the possible exception of the last > > reported date. An explanation about how dates are selected could resolve > > questions about first an last dates shown in reports. > > > > 2. Reports should respect the varying lengths of months so that bug 1 can be > > meaningfully resolved with logically consistent dates evaluated and > > displayed. This is probably the place to include 'last day of previous > > period' as a starting option. > > > > 3. Report should show the time zone used for it's calculations. > > > > Does anyone have something to add to or refute this comment? > > Asset, Liability, and Net Worth reports include transactions on the day > indicated in the report. This is documented in e.g. > http://gnucash.org/docs/v2.6/C/gnucash-help/report-assets.html: "The first > bar shows the selected values at the end of the day on the first date > chosen." > > Time zone is the one in effect on the computer at the time the report is > created. This is a bit of a problem with the way transactions are recorded > because if GnuCash is used in a different time zone from the one > transactions in which transactions were entered the transaction date can > change, see bug 137017. > > The problem isn't that the intervals are wrong, it's that when the reports > were designed the author assumed that reports would always start at the > beginning of a month so it always uses the number of days of the current > month to calculate the date in the next month, which blows up for dates at > the end of months. The interval function needs to notice that condition and > add the number of days in the *next* month to the interval date in that > case. But that's still not sufficient, because if you give 28 Feb as a start > date GnuCash has to decide whether you want the 28th of each month or the > last day of each month, so we'd need either to assume that if you start on > the last day of a month you mean last day of every month or provide a > control to specify. I'd be inclined to just assume. Agreed on the last-day-of-month assumption. As for the interval calculation I'd be tempted to take the day-of-month as base for each subsequent interval. If that day-of-month doesn't exist in one of the following periods, take the last-day-of-month for that period only. That's slightly different from adding the number of days from the *next* month.
For that to work you have to put the last day of the month on the list but pass the originally calculated date to the next recursive call.
FWIW the 'slipping day' problem will be fixed in 3.5 onwards: https://github.com/Gnucash/gnucash/commit/65bfeaf5de8c52c77ba0f4e8f4d5d6ceeb45b33e The 'new' regular-dates or regular-intervals will assume that the day specified is the desired one. monthly with a start-date of 28-feb, will generate dates 28-mar 28-apr etc. monthly with a start-date of 31-dec will leads to dates 31-jan 28/29-feb 31-mar 30-apr etc. the algorithm is the simplest one I could think of: 1. from original DD/MM/YY add required delta (+/-1 for monthly, +/-3 for quarterly etc) to MM to make MMM 2. try make new date DD/MMM/YY 3. if the new date causes a date slip, its new-month will not be as predicted; try with a reduced DD until the new-month is as expected. Can this bug be closed?
I am concerned that users may want the monthly reports to start on the last day of every month, but the first month happens to be February. How could that user get the subsequent months to be reported on the last day?
Well, 28-feb is an unfortunate casualty in fixing this genuine bug *without* needing to create yet another option and fixing *all* relevant reports. I'll have to say sorry, there's no other easy way. If they want monthly end-of-month reports and *must* include 28/29-february, the solution is now to choose 31-jan and choose monthly intervals. To have any other solution is to add many new options in all reports. My approach to most of my bugfixes is, if there are no perfect solution, to choose the 'least-intrusive' or 'least-surprising' among a number of limited options. We cannot be 100% comprehensive for all possible combinations. See discussion onwards at: https://code.gnucash.org/logs/2019/02/02.html#T06:44:51
I'm inclined to agree the solution is better than the bug. On the other hand I also think assuming last day of month whenever possible would be even more intuitive. I believe it to be much more likely to want a report at end of month than on the 28th of each month if the start month happens to be February. I do understand this would likely mean backwards incompatible changes. And I don't know if the added effort is worth it. Our user base will no doubt tell us once 3.5 gets released.
Ok here's an algorithm for 'guessing the end-of-month' cases. 1. (define (end-of-month? time64) a. break out time64 into d1/m1/y1 b. create new date (d1+1)/m1/y1 c. break out new date into d2/m2/y2 d. return (m1 != m2) 2. start with start-date DD/MM/YY 3. if (end-of-month? start-date ) then switch on a flag. 4(a). if flag OFF then use my algorithm above; this produces dates 29-nov 29-dec, 29-jan, 28/29-feb, 29-mar, 29-apr etc 5(b). if flag ON then use a different algorithm. from starting-date 30-nov, try 30-dec (which fails end-of-month therefore use DD+1/MM+1/YY to produce 31-dec), 30-jan (which fails end-of-month therefore use DD+1/MM+2/YY to produce 31-jan), try 30-feb (which fails thanks to my original algorithm implemented therefore try DD-1/MM+3/YY = 29-feb, which *may* fail therefore try 28-feb) --- I would still *strongly* prefer not to implement the above, because (1) it is much more 'unsurprising' to me that DD is pinned as far as possible than for software to guess that '28/29/30/31' means 'end-of-month' and requires special handling. But this is just my opinion. (2) the code for above will be done correctly \m/ but I'm sure will still be ***undecipherable*** to the next hacker...
End of month(s) might be more easily found as day of year for the first of the following month minus one, then compared to day of year for next proposed calculation date. If that is also very complicated, take Geert's advice and comment the h*** in the code so it is easier to follow.
Chris, I believe this is a matter of disconnect between cold logic and human psychology :) I'm not sure how to explain this very well, but this is how I understand it: if the human brain can choose between seeing a repetitive pattern on the 28th of each month or at the end of the month (like when it starts on February), it will see end of month first. This is because to our brain month boundaries have more meaning than a random date within the month. Catering for this indeed requires more complex coding, our mind is not that logical as we would like to believe. In addition it does still make an assumption that is not always correct (though from the human psychology point of view more likely so than the simpler algorithm). Having said that, I don't insist on the implementation of this more complex algorithm. As mentioned before, what you have now is more consistent than what we had before. The dates no longer jump past the end of month at random (well, not so random). In addition there's a slightly clunky workaround to force end of month in the February 28 case: make the report start one month earlier on January 31st. That will ensure all subsequent data points will be at end of month at the expense of one extra data point the chart. The extra data point is what makes it a bit clunky IMO. Yet another (long-term) approach to fix this would be to rethink our date input widget. It now allows to enter either an absolute date or a relative period marker. While refactoring the options code in the future we may to reconsider this to also allow users to enter something like "Last day of month X" where "X" is input separately as a month/year combination. If done well that same month/year combination widget could then serve to solve our current issue with recurring accounting periods that don't start/end on any of the hard-coded period definitions. But that's another story.
Ok human psychology trumps cold logic, so, the date list generator has been amended: (1) slipping months are now fixed; if a month-interval causes month slip, the day will reduce until no month slip. i.e. 31-nov, 31-dec, 31-jan, 31->30->29->28-feb (2) if the original date is a month-end we assume all dates must be month-ends. if the user specifies 28-feb on non-leap year then month-end is respected. if they choose 28-feb-leapyear then all days will be 28. The other aspects are unchanged and NOTABUG or NOTRELEVANT (a) should Networth or Profit&Loss start from begin or end of dates? Well GnuCash convention states Networth=end-of-date, and profit&loss = start-of-begin-date to end-of-begin-date... no need to change this. (b) should report specify TZ? Heck, no. We want to eliminate time in GnuCash. I believe that a report being launched in *any* timezone will dutifully line up date comparison algorithms so that D/M/Y as above is respected. I think there's some bug whereby you save-report, relaunch with different TZ, and load-report may cause TZ slip but this is not this bug, and will not be fixed by report-TZ.
(a) should Networth or Profit&Loss start from begin or end of dates? Well GnuCash convention states Networth=end-of-date, and profit&loss = start-of-begin-date to (*typo)end-of-end-date... no need to change this. relevant commit for this bugfix is https://github.com/Gnucash/gnucash/commit/d711cc35
