GnuCash
Contact   Instructions
Bug 796925 - Database connections not closed by session.end()/session.destroy() when is_new=True
Summary: Database connections not closed by session.end()/session.destroy() when is_ne...
Status: NEW
Alias: None
Product: GnuCash
Classification: Unclassified
Component: Python Bindings (show other bugs)
Version: git-maint
Hardware: PC Linux
: Normal minor
Target Milestone: ---
Assignee: core
QA Contact: core
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2018-10-24 15:06 EDT by Tom Lofts
Modified: 2018-10-25 14:27 EDT (History)
1 user (show)

See Also:


Attachments
Example Python script reproducing issue (1.05 KB, text/x-python)
2018-10-24 15:06 EDT, Tom Lofts
no flags Details

Description Tom Lofts 2018-10-24 15:06:34 EDT
Created attachment 373036 [details]
Example Python script reproducing issue

I've been writing some integration tests which call the following code multiple times:

session = gnucash.Session('mysql://root:oxford@localhost/test', is_new=True, ignore_lock=False)
session.end()
session.destroy()

After this runs a large number of times I get an error 'Too many connections' because Gnucash is not closing the database connection when the session ends/is destroyed. The connections are only closed when the script finishes.

I think the problem might be that two database connections are originally opened, as if session.end()/session.destroy() is not called, two connections are created each time session = gnucash.Session is called, but only one is cleaned up by session.end()/session.destroy() leaving an additional connection open after this set of 3 commands.

Digging deeper, here's part of the MySQL query log for session = gnucash.Session

2018-10-24T13:00:10.997511Z     7 Connect   root@localhost on test using Socket
2018-10-24T13:00:10.997735Z     7 Query SET NAMES 'utf8'
2018-10-24T13:00:10.997937Z     7 Query SELECT @@sql_mode
2018-10-24T13:00:10.998749Z     7 Query SET sql_mode='ONLY_FULL_GROUP_BY,STRICT_TRANS_TABLES,NO_ZERO_IN_DATE,ERROR_FOR_DIVISION_BY_ZERO,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION'
2018-10-24T13:00:10.998934Z     7 Query CREATE TEMPORARY TABLE numtest ( test_int BIGINT, test_unsigned BIGINT, test_double FLOAT8 )
2018-10-24T13:00:11.000065Z     7 Query INSERT INTO numtest VALUES (-9223372036854775807, 9223372036854775807, 1.79769213486e+307)
2018-10-24T13:00:11.000440Z     7 Query SELECT * FROM numtest
2018-10-24T13:00:11.000777Z     7 Query DROP TABLE numtest
2018-10-24T13:00:11.028168Z     7 Query SHOW TABLES FROM `test`

2018-10-24T13:00:11.029106Z     8 Connect   root@localhost on test using Socket
2018-10-24T13:00:11.029276Z     8 Query SET NAMES 'utf8'
2018-10-24T13:00:11.029424Z     8 Query SELECT @@sql_mode
2018-10-24T13:00:11.029664Z     8 Query SET sql_mode='ONLY_FULL_GROUP_BY,STRICT_TRANS_TABLES,NO_ZERO_IN_DATE,ERROR_FOR_DIVISION_BY_ZERO,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION'
2018-10-24T13:00:11.029780Z     8 Query CREATE TEMPORARY TABLE numtest ( test_int BIGINT, test_unsigned BIGINT, test_double FLOAT8 )
2018-10-24T13:00:11.030637Z     8 Query INSERT INTO numtest VALUES (-9223372036854775807, 9223372036854775807, 1.79769213486e+307)
2018-10-24T13:00:11.030847Z     8 Query SELECT * FROM numtest
2018-10-24T13:00:11.031036Z     8 Query DROP TABLE numtest

2018-10-24T13:00:11.032583Z     8 Query BEGIN
2018-10-24T13:00:11.032757Z     8 Query SHOW TABLES FROM `test` LIKE 'gnclock'
....

You can see that two connections are opened - 7 and 8, but connection 7 is never closed and they also run the conn_test_dbi_library functions twice.

I think this narrows the issue down to GncDbiBackend<Type>::session_begin in gnc-backend-dbi.cpp, specifically dbi_conn_connect is called twice in this function: once for the tests (connection 7 above) and once to create a new book (connection 8) - it's possible this issues exists outside of the Python bindings and is a bug in the DBI backend, but I haven't confirmed this though the GUI.

Please find a test script attached (which assumes MySQL credentials) which runs the above commands 5 times and counts the open connections after each run.

I hope this is enough to investigate the issue further, but if you have any other queries please let me know.

Kind regards,

Tom
Comment 1 Tom Lofts 2018-10-25 14:25:13 EDT
Hi All,

I've done some more research here and I think the problem is occurring in qof_commit_edit_part2 in qofinstance.cpp.

The book instance should be have it's priv->infant set to false here:

https://github.com/Gnucash/gnucash/blob/dddc278851eb1bff3ec5c35cea282a69a3e4d05c/libgnucash/engine/qofinstance.cpp#L1039

But this line is never called as qof_book_get_backend(priv->book) returns a backend which is commited on line 1023.

This commit then runs on the book with infant set to true causing the error and exists the function before priv->infant is set to false.

Looking at the code in qof_commit_edit_part2 I'm not sure exactly the best way of resolving this, but hopefully someone more familiar with the Gnucash internals than me can investigate further.

Thanks,

Tom
Comment 2 Tom Lofts 2018-10-25 14:27:50 EDT
Sorry, please ignore the above comment #1 - it should have been posted in bug #795839 but I don't see a way to remove it.

Note You need to log in before you can comment on or make changes to this bug.