(will insert later)
The release notes for 2.7.7 mention that gnucash now supports python-3.x. I don't know how to test this (please tell me!) but while trying to run random files installed by gnucash with python-3.6, I saw a lot of errors: > python3 /usr/pkg/share/gnucash/python/pycons/shell.py File "/usr/pkg/share/gnucash/python/pycons/shell.py", line 156 print `r` ^ SyntaxError: Missing parentheses in call to 'print'. Did you mean print(print `r`)? > python3 /usr/pkg/share/gnucash/python/pycons/ishell.py File "/usr/pkg/share/gnucash/python/pycons/ishell.py", line 23 except Exception,e: ^ SyntaxError: invalid syntax > python3 /usr/pkg/share/gnucash/python/pycons/console.py File "/usr/pkg/share/gnucash/python/pycons/console.py", line 82 def seek(self, a): raise IOError, (29, 'Illegal seek') ^ SyntaxError: invalid syntax These errors indicate that the files only support python-2.x, not python-3.x. Is this module still used? Then it should support python-3.x. One other problem: > python3 /usr/pkg/lib/python3.6/site-packages/gnucash/gnucash_business.py * 13:51:30 WARN <gnc.engine> failed to load gncmod-backend-dbi from relative path dbi * 13:51:30 CRIT <gnc.engine> required library gncmod-backend-dbi not found. * 13:51:30 WARN <gnc.engine> failed to load gncmod-backend-xml from relative path xml * 13:51:30 CRIT <gnc.engine> required library gncmod-backend-xml not found. The backends are installed in the default location: /usr/pkg/lib/gnucash/libgncmod-backend-xml.so /usr/pkg/lib/gnucash/libgncmod-backend-dbi.so Why are they not found?
The Python bindings were changed to python3 and I changed enough of the pycons demo to get it to not have any errors at GnuCash startup but didn't have time to check the rest of it. I don't know if anyone uses pycons; I actually rather doubt it, but it's not a lot of work to fix it up to work with Py3. /usr/pkg is a default path? What distro is that? One doesn't run code installed in lib/pythonx directly. If any code in there even has a __main__ it's for testing that module. You'll get further with starting python, importing gnucash_business, and then calling functions added by the module. There are a bunch of example scripts in bindings/python/examples in the source directory. They're not installed anywhere. My first guess is that by trying to run gnucash_business.py directly you've not executed some setup code that enables GnuCash's binreloc code to find its modules and that regardless of whether /usr/pkg/lib is the default libdir for your distro that's not what was passed to CMake at build time (the fallback default location).
I'm packaging for pkgsrc, the default packaging system for NetBSD. /usr/pkg is the default there. I tried running the simple test: bindings/python/example_scripts> python3.6 simple_test.py * 16:44:13 WARN <gnc.engine> failed to load gncmod-backend-dbi from relative path dbi * 16:44:13 CRIT <gnc.engine> required library gncmod-backend-dbi not found. * 16:44:13 WARN <gnc.engine> failed to load gncmod-backend-xml from relative path xml * 16:44:13 CRIT <gnc.engine> required library gncmod-backend-xml not found. Traceback (most recent call last): File "simple_test.py", line 10, in <module> session = Session("xml://%s" % FILE_1, is_new=True) File "/usr/pkg/lib/python3.6/site-packages/gnucash/gnucash_core.py", line 107, in __init__ self.begin(book_uri, ignore_lock, is_new, force_new) File "/usr/pkg/lib/python3.6/site-packages/gnucash/gnucash_core.py", line 153, in new_function self.raise_backend_errors(function.__name__) File "/usr/pkg/lib/python3.6/site-packages/gnucash/gnucash_core.py", line 131, in raise_backend_errors errors ) gnucash.gnucash_core.GnuCashBackendException: call to begin resulted in the following errors, ERR_BACKEND_NO_HANDLER gnucash is setup with -DCMAKE_INSTALL_PREFIX:PATH=/usr/pkg -DCMAKE_INSTALL_LIBDIR:PATH=lib. What else do I need to set?
-DWITH_PYTHON=ON for a start. What does CMakeCache.txt have for LIBDIR?
I hadn't mentioned it, but I do set -DWITH_PYTHON=ON. CMAKE_INSTALL_LIBDIR:PATH=lib LIBDIR:STRING=lib
I realize now that simply running `python simple_test.py` from the command line doesn't access the cmake setup so it doesn't actually matter what's in CMAKE_INSTALL_LIBDIR. You need to set the paths in the environment so that dlopen can find the loadable modules. LD_LIBRARY_PATH and GNC_LIBRARY_PATH are usually sufficient, and you can point them at either the installation path or the build path. You don't have to do that for gnucash itself because it reads a file named etc/gnucash/environment that has the paths, but loading it isn't exposed to the bindings.
(In reply to John Ralls from comment #6) > You don't have to do that for gnucash itself because it reads a file named > etc/gnucash/environment that has the paths, but loading it isn't exposed to > the bindings. It used to be the python bindings also loaded the environment file. If this is no longer the case, that's a regression IMO. I did add code to the python binding's init block at some point for this. I'm not at my development machine until Tuesday, so I can't check before that time.
Ah, you're right: It's exported to python via core-utils/gnc-environment.c. I thought that that was buried in gnucash-bin.c. But the bindings don't run that on their own and there's no evidence in history that they ever did. Should they? It's a one-line addition to bindings/python/__init__.py.
I think we should. The python bindings depend on libgnucash. As it currently stands libgnucash still contains code relying on guile. And it provides/uses libgnc-mod. Both won't work if the environment adjustments from etc/environment haven't been set. Perhaps in the future this can be removed, but in the current context it seems required.
OK, pushed. It will be in tomorrow's release.
And reverted. Aside from it bothering Arch on Travis I found that it gets called as part of the %init in bindings/python/gnucash_core.i so calling it again from Python shouldn't be necessary. I think the real problem is that it's not actually finding gnucash/environment because gnc_path_get_pkgsysconfdir() relies on binreloc and falls back on a compiled-in SYSCONFDIR. Binreloc isn't going to get the right answer from python so it goes to the fall-back, which is set from ${CMAKE_INSTALL_SYSCONFDIR} and that is "etc". But there seems to be more: If I run python3 /path/to/source/bindings/python/examples/simple_test.py from the build dir it works. If I cd to the source directory and try from there it fails to find the backend modules, even if I specify GNC_MODULE_PATH and LD_LIBRARY_PATH on the command line.
(In reply to John Ralls from comment #11) > And reverted. Aside from it bothering Arch on Travis I found that it gets > called as part of the %init in bindings/python/gnucash_core.i so calling it > again from Python shouldn't be necessary. > Ah, I *thought* I had added it somewhere. Glad you found it. > I think the real problem is that it's not actually finding > gnucash/environment because gnc_path_get_pkgsysconfdir() relies on binreloc > and falls back on a compiled-in SYSCONFDIR. Binreloc isn't going to get the > right answer from python so it goes to the fall-back, which is set from > ${CMAKE_INSTALL_SYSCONFDIR} and that is "etc". > Let me write this out a bit more elaborately, as that helps me clear my head: Binreloc as we use it indeed takes the base path to the running gnucash executable as the basis for all other paths. If gnucash itself is not running, this will fallback to paths defined at compile time. For linux like systems that will mostly work because on such systems the final installation path will already be defined at compile time. Not so on Windows and OS X. On these systems the installation path is only determined at installation time. Sidenote: we currently only support python bindings on linux and similar systems. I have added the note on Windows and OS X mainly for binreloc completeness. Obviously when using the gnucash core (which we're slowly reworking into libgnucash) via the python bindings, there is no running executable. So at that point binreloc always falls back to the fallback compile time paths. That is bound to fail. One solution would be to adjust our binreloc code to use the base path of libgnc-core-utils instead to determine the main prefix. The comment in https://github.com/Gnucash/gnucash/blob/unstable/libgnucash/core-utils/binreloc.c#L258 suggests binreloc used to have an alternative initialization routine to work from a library instead of an executable. I have not yet searched the net for this and don't know whether this works in a platform independent way without extra tweaks (which we also did to get it working on Windows and OS X). Alternatively I also found https://github.com/gpakosz/whereami which looks like a modernized version of binreloc. In addition to what binreloc provides is has these extras: - it also supports ios and android which we hope to port to ourselves one day - it can detect a library as well as an executable so we can use it to base our path initialization on libgnc-core-utils (or wherever our path initialization will eventually end up). More generally this issue is one more reminder of how complicated path handling is when the application needs to support multiple platforms. There are many possible variations which we still may not be covering. And on the other hand it also reminds we of how scattered our path handling is in gnucash and of the many (sometimes false) assumptions it makes. Perhaps we should define a separate task to clean this up throughout the whole of gnucash. As part of this task we can decide on the best way forward. Probably not for gnucash 3.0 though. > But there seems to be more: If I run > python3 /path/to/source/bindings/python/examples/simple_test.py > from the build dir it works. If I cd to the source directory and try from > there it fails to find the backend modules, even if I specify > GNC_MODULE_PATH and LD_LIBRARY_PATH on the command line. I can't reproduce this (yet). When I try python3 /home/janssege/Development/gnucash/unstable/bindings/python/example_scripts/simple_test.py I get this error Traceback (most recent call last): File "/home/janssege/Development/gnucash/unstable/bindings/python/example_scripts/simple_test.py", line 6, in <module> from gnucash import Session, Account, Transaction, Split, GncNumeric ImportError: cannot import name 'Session' That's with today's unstable and after a clean rebuild. It looks like this error comes earlier than the errors about gncmodule loading. I'd need to figure this out first before I can comment sensibly on the difference in behaviour you describe.
MacOS actually needs to determine the load paths at runtime, which is what binreloc is all about. And there is an executable when using the python bindings: The python interpreter. The problem, of course, is that it's not necessarily in the same hierarchy as the gnucash libraries. The compiled-in paths are relative, e.g. SYSCONFDIR is just "etc/gnucash", so they're independent of the eventual location of the hierarchy... though CMake does some funny things with paths when the install prefix is rooted in either /usr or /opt, so some care is needed. Just to add one more layer of complication, Thomas is trying to run from DESTDIR, so the compiled in paths (e.g. ${CMAKE_INSTALL_FULL_SYSCONFDIR} is going to be wrong. For your import problem, it's probably not finding the right "gnucash". Try adding ${builddir}/lib/python3.x/site-packages to PYTHONPATH.
No, I'm not trying to run from DESTDIR - gnucash is properly installed.
> I'm packaging for pkgsrc, the default packaging system for NetBSD. /usr/pkg is the default there. Sorry, I took that to mean that /usr/pkg was the default DESTDIR for your packaging system. https://www.freebsd.org/cgi/man.cgi?query=hier&sektion=7&manpath=freebsd-release-ports doesn't say anything about /usr/pkg.
That's the FreeBSD man page :-) Here's the NetBSD one: https://bugzilla.gnome.org/show_bug.cgi?id=794526 (DESTDIR is usually something like /tmp/gnucash/dest/usr/pkg/)
Really? This bug is a NetBSD manpage? ;-) I think you meant http://netbsd.gw.com/cgi-bin/man-cgi?hier+7+NetBSD-current.
Oops. Yes, that one :-)
(In reply to John Ralls from comment #13) > MacOS actually needs to determine the load paths at runtime, which is what > binreloc is all about. Indeed. And so does our Windows package. And possibly the same may be true for newer package formats on linux such as flatpak and similar. It's safe to assume gnucash requires a runtime path discovery mechanism. This can be binreloc though to solve scripting support we should use the alternative form to extract a prefix from a library path rather than a binary. As this is not part of our gnucash copy of binreloc I searched the web for the original sources. I found someone imported the original binreloc sources into github in 2014 but otherwise hasn't made any updates to it: https://github.com/limbahq/binreloc The README for this repo refers to the listaller documentation which in turn suggests they created binreloc. However the liststaller's git repo reveals the binreloc code was last touched in 2012: https://gitorious.org/listaller/listaller/source/master:?p=listaller:listaller.git;a=history;f=contrib/binreloc;hb=HEAD So current binreloc has two issues: - it appears to be unmaintained - it only targets linux and Windows (according to the liststaller documentation) Especially the lack of OS X support is troubling. We patched our own binreloc instance to work from the executable path. I don't know if we OS X also provides similar functions to extract the full path for a shared library. Well, it probably does because whoami has figured it out. So the alternative to binreloc would be whoami. This only handles the first part though: it extracts a full path for a binary or library (and for scripting support the latter is what we need). From there we still need to add functions to provide specific subdirectories of this path, like binreloc does. As said before I like whoami supports our (longterm) future target platforms as well as our current ones. So for 4.0 I would definitely be interested in integrating it into libgnucash. For 3.0 that may be too radical to still accept now. But perhaps we can adjust our current binreloc sources to - use br_init_lib rather than br_init - extract the part of whoami that detects a library path on OS X and insert in in our binreloc code. I haven't checked feasibility of this. I'm just looking for sensible ways forward this close to 3.0 release with minimal code impact. For 4.0 here's one more consideration: binreloc assumes all relevant directories are subdirectories of the discovered prefix. In theory this is not guaranteed as one can arbitrarily override these directories at configure time. We have set up our Windows and OS X builds to adhere to this assumption for simplicity so on these platforms that's a non-issue. I don't know though whether there are linux like platforms that do override any directories and whether we should continue to support this. I believe currently this works if binreloc is disabled. > And there is an executable when using the python > bindings: The python interpreter. The problem, of course, is that it's not > necessarily in the same hierarchy as the gnucash libraries. > Indeed. I didn't mention it. > The compiled-in paths are relative, e.g. SYSCONFDIR is just "etc/gnucash", > so they're independent of the eventual location of the hierarchy... though > CMake does some funny things with paths when the install prefix is rooted in > either /usr or /opt, so some care is needed. > And that explains why the script call from the build dir works. The relative paths at that point match a valid install tree (the build tree) in which the environment file can be found. > Just to add one more layer of complication, Thomas is trying to run from > DESTDIR, so the compiled in paths (e.g. ${CMAKE_INSTALL_FULL_SYSCONFDIR} is > going to be wrong. > > For your import problem, it's probably not finding the right "gnucash". Try > adding ${builddir}/lib/python3.x/site-packages to PYTHONPATH. Right, that was it. PYTHONPATH is amended in etc/gnucash/environment, but as that file isn't interpreted yet when the python script initializes my python path was wrong. Setting it properly makes the test script work. This PYTHONPATH initialization will always be required for gnucash instances not installed in the default path and one we can't fix from within gnucash.
Our binreloc diverges rather a lot from the "official" one. Christian Stimming added Windows support to it when he set up GnuCash to use it in 2006 and I added Mac bundle support when I did the mac integration work in 2009. There's been some cleaning and fixing since as well. It's too late to get into 3.0 but we can surely replace it during 3.x, it has no affect whatever on compatibility. Actually, we can fix pythonpath inside of gnucash, at least for the example scripts. In each example script: import sys sys.path += GNUCASH_LIBRARY_PATH from gnucash import ... Of course that requires converting the scripts to foo.py.in and running cmake's configure_file on them to insert the path.
*** Bug 795359 has been marked as a duplicate of this bug. ***
At least part of this is that we're chasing the wrong problem. The backend libraries are loaded by gnc_engine_init_part2 at https://github.com/Gnucash/gnucash/blob/maint/libgnucash/engine/gnc-engine.c#L67. Until cdcb230 that I just pushed it was passing "dbi" and "xml" as subdirectories for the respective backends; that's clearly a leftover from the old autotools builds. With that in place I can run simple_test.py from an arbitrary directory as long as I give python3 the path. To run from the build directory instead set GNC_BUILDDIR to point there and GNC_UNINSTALLED=1.
(In reply to John Ralls from comment #22) > At least part of this is that we're chasing the wrong problem. The backend > libraries are loaded by gnc_engine_init_part2 at > https://github.com/Gnucash/gnucash/blob/maint/libgnucash/engine/gnc-engine. > c#L67. Until cdcb230 that I just pushed it was passing "dbi" and "xml" as > subdirectories for the respective backends; that's clearly a leftover from > the old autotools builds. Wow, that's a good find. > > With that in place I can run simple_test.py from an arbitrary directory as > long as I give python3 the path. This I don't understand. Which path do you give to python3: the path to simple_test.py ? Or something else ?
python3 /path/to/simple_test.py should work if everything is installed on the normal system paths. In my MacOS jhbuild environment using a jhbuild shell is sufficient setup, binreloc understands $PREFIX; from outside of the jhbuild shell I need only add PREFIX=/path/to/installation_prefix to the environment and say $PREFIX/bin/python3... If GnuCash isn't installed one can instead set GNC_UNINSTALLED=1 GNC_BUILDDIR=/path/to/build_directory in the environment and binreloc will find the right paths.
@Thomas Klausner please pull the latest maint and try as outlined above.
Created attachment 372753 [details] Dockerfile reproducing issue after cdcb230 commit Hi John, I've attempted to try your advice with the latest maint release, but I'm still seeing the same error - it's possible I'm doing something wrong - I've attached a Dockerfile which builds from maint and runs 'import gnucash' as per your instructions but it still errors as per Thomas Klausner's original response. If I'm doing something wrong or if you'd prefer I provide further information or a testcase in another format here please let me know. Kind regards, Tom
Sorry I haven't tested this yet. Is this included in the 3.2 release?
Yes.
(In reply to Tom Lofts from comment #26) > I've attempted to try your advice with the latest maint release, but I'm > still seeing the same error - it's possible I'm doing something wrong - I've > attached a Dockerfile which builds from maint and runs 'import gnucash' as > per your instructions but it still errors as per Thomas Klausner's original > response. No, you didn't follow my instructions. ln -s /build/lib/python3/dist-packages/gnucash /usr/lib/python3/dist-packages/gnucash is not equivalent to ninja install Since GnuCash *isn't* installed, you need to define GNC_UNINSTALLED and GNC_BUILDDIR: ENV GNC_UNINSTALLED 1 ENV GNC_BUILDDIR /build You'll get several rounds of WARN <gnc.core-utils> no backend loaded, or the backend doesn't define register_cb, returning 0 that you can ignore, then "OK". For a more interesting test, substitute RUN python3 /gnucash/bindings/python/example_scripts/simple_book.py for your very simple test.
So I tried again with gnucash-3.2. I installed it first, then I ran: bindings/python/example_scripts> python3 simple_test.py * 10:50:26 WARN <gnc.engine> failed to load gncmod-backend-dbi from relative path * 10:50:26 CRIT <gnc.engine> required library gncmod-backend-dbi not found. * 10:50:26 WARN <gnc.engine> failed to load gncmod-backend-xml from relative path * 10:50:26 CRIT <gnc.engine> required library gncmod-backend-xml not found. Traceback (most recent call last): File "simple_test.py", line 10, in <module> session = Session("xml://%s" % FILE_1, is_new=True) File "/usr/pkg/lib/python3.6/site-packages/gnucash/gnucash_core.py", line 107, in __init__ self.begin(book_uri, ignore_lock, is_new, force_new) File "/usr/pkg/lib/python3.6/site-packages/gnucash/gnucash_core.py", line 153, in new_function self.raise_backend_errors(function.__name__) File "/usr/pkg/lib/python3.6/site-packages/gnucash/gnucash_core.py", line 131, in raise_backend_errors errors ) gnucash.gnucash_core.GnuCashBackendException: call to begin resulted in the following errors, ERR_BACKEND_NO_HANDLER Looking at the system calls, I see that it is trying to open relative paths -- ${PREFIX} is missing: 19079 1 python3.6 NAMI "lib/gnucash/libgncmod-backend-dbi.so" 19079 1 python3.6 NAMI "lib/gnucash/libgncmod-backend-dbi.dylib" 19079 1 python3.6 NAMI "lib/gnucash/libgncmod-backend-dbi.dylib.so" 19079 1 python3.6 NAMI "lib/gnucash/libgncmod-backend-dbi.dylib.la" 19079 1 python3.6 NAMI "lib/gnucash/libgncmod-backend-dbi.dylib" ... 19079 1 python3.6 NAMI "lib/gnucash/libgncmod-backend-xml.so" 19079 1 python3.6 NAMI "lib/gnucash/libgncmod-backend-xml.dylib" 19079 1 python3.6 NAMI "lib/gnucash/libgncmod-backend-xml.dylib.so" 19079 1 python3.6 NAMI "lib/gnucash/libgncmod-backend-xml.dylib.la" 19079 1 python3.6 NAMI "lib/gnucash/libgncmod-backend-xml.dylib" I would have expected /usr/pkg/lib/gnucash/libgncmod-backend-dbi.so, which exists in the file system.
Try PREFIX=/usr/pkg python3 simple_test.py
What is the full path to the python3 executable on NetBSD?
No change: PREFIX=/usr/pkg python3 simple_test.py * 23:31:49 WARN <gnc.engine> failed to load gncmod-backend-dbi from relative path * 23:31:49 CRIT <gnc.engine> required library gncmod-backend-dbi not found. * 23:31:49 WARN <gnc.engine> failed to load gncmod-backend-xml from relative path * 23:31:49 CRIT <gnc.engine> required library gncmod-backend-xml not found. Traceback (most recent call last): File "simple_test.py", line 10, in <module> session = Session("xml://%s" % FILE_1, is_new=True) File "/usr/pkg/lib/python3.6/site-packages/gnucash/gnucash_core.py", line 107, in __init__ self.begin(book_uri, ignore_lock, is_new, force_new) File "/usr/pkg/lib/python3.6/site-packages/gnucash/gnucash_core.py", line 153, in new_function self.raise_backend_errors(function.__name__) File "/usr/pkg/lib/python3.6/site-packages/gnucash/gnucash_core.py", line 131, in raise_backend_errors errors ) gnucash.gnucash_core.GnuCashBackendException: call to begin resulted in the following errors, ERR_BACKEND_NO_HANDLER python3 is /usr/pkg/bin/python3
Are you disabling binreloc in cmake? If not, it seems broken and you're going to have to debug it.
Created attachment 372884 [details] Updated Dockerfile with issue resolved after cdcb230 commit (In reply to John Ralls from comment #29) Hi John, Thanks for your help with this - I haven't used the cmake/ninja build system much and must have missed the fact I hadn't run an install step. I've now updated the Dockerfile to include an install step and I can confirm the issue no longer occurs for me on the 'import gnucash' line. Thanks for getting the bug resolved and your continued development on Gnucash. Kind regards, Tom
(In reply to John Ralls from comment #34) > Are you disabling binreloc in cmake? If not, it seems broken and you're > going to have to debug it. I'm not changing any binreloc setting AFAIK. When debugging this, what should I look for?
While refreshing my memory by looking at the binreloc code again so that I could answer that question I realized that it only knows about $PREFIX on MacOS (by way of a gtkosx_application call). The bindings don't call gnc_gbr_init() so when not on MacOS and GNC_BUILDIR and GNC_UNINSTALLED aren't set gnc_gbr_find_prefix returns NULL. That's consistent with what you're seeing. So first experiment, try GNC_UNINSTALLED=1 GNC_BUILDDIR=usr/pkg python3 simple_test.py Next let's try ensuring that binreloc is initialized. Apply the following and recompile: --- a/libgnucash/engine/qof-backend.cpp +++ b/libgnucash/engine/qof-backend.cpp @@ -29,6 +29,7 @@ extern "C" #include "qof.h" #include <gnc-path.h> #include "gncla-dir.h" +#include <binreloc.h> } #include <string> @@ -92,6 +93,14 @@ QofBackend::register_backend(const char* directory, const char* module_name) return false; } + if (!gnc_gbr_find_exe(NULL)) //Make sure binreloc is initialized + { + GError *error = nullptr; + if (!gnc_gbr_init(&error)) + PERR("Failed to initialize binreloc: %s", error->message); + g_clear_error(&error); + } + auto absdir = directory; if (!absdir || !g_path_is_absolute(absdir)) absdir = gnc_path_get_pkglibdir ();
I've got the same issue. I downloaded the source using git. I use a local installation to ~/.local. I read through this bug report and tried to understand the issue through the source. I came to understand that the path for the libs is derived from the executable path. This works when they are in the same directory structure. It doesn't work when that is not the case. Starting gnucash as ~/.local/bin/gnucash finds the libs. Starting python like 'python3 -c "import gnucash"' doesn't work. Python3 is /usr/bin/python3 so the libraries are unsuccessfully being searched in /usr/lib64. It works when I put python to ~/.local/bin/python3. But that's a bit unconvenient :) That's not all. I just realized that the situation I described only happens when the BinReloc has been initialized. I added the line gnc_gbr_init(); to gnucash_core.i. I thought it doesn't change anything. That's not true. It changes the outcome only in the way that the abovementioned is possible. Without the this even relocating the python binary doesn't work.
I found a place in the environment where I could put the path. When debugging libgnucash/core-utils/gnc-environment.c (see https://github.com/Gnucash/gnucash/commit/fb6fd8a1eac16a065e3d375349961d8b47253e52) I found that PREFIX contains the right path. It is being set in gncla-dir.h. The call to gnc_path_get_prefix() in gnc-path.c uses this as default as it calls gnc_gbr_find_prefix (PREFIX). This is not the case for the other values. No further use of PREFIX. I changed binreloc.c: (https://github.com/c-holtermann/gnucash/commit/d102abd4c098a691dee674fa80480b89bb4a98a1) which doesn't seem completely right as PREFIX is being used a level higher (gnc-path.c). Should the other pathes contain the PREFIX? gncla-dir.h looks like this: #define PREFIX "/home/MYUSER/.local" #define DATADIR "share" #define SYSCONFDIR "etc" #define BINDIR "bin" #define LIBDIR "lib64" should DATADIR contain /home/MYUSER/.local ?
Your change to binreloc.c makes the calls to gnc_gbr_find_prefix consistent with the one in gnc-file.c, so I think that it's correct. Does it get the bindings working? It doesn't matter if DATADIR and friends include PREFIX because they're filtered through e.g. datadir = gnc_file_path_relative_part(PREFIX, DATADIR); and that would just take it out. The only caveat is that gncla-dir.h is being set up by cmake, so if one uses a DESTDIR installation the bindings won't work from the DESTDIR.
I think the changes in binreloc may theoretically do more harm than good. When binreloc is initialized (hence enabled) all paths are calculated relative to the gnucash binary (exe variable in gnc_gbr_prefix) and PREFIX will be ignored completely. When binreloc is disabled gnc_gbr_find_prefix will return NULL in the code as it currently is. As a consequence the value of default_data_dir/default_lib_dir etc are returned. However when you always pass PREFIX to gnc_gbr_find_prefix, it will return prefix in that case. If DATADIR is a subdirectory (or lower sibling) of PREFIX it will give the same result, albeit via a slower code path. That is because default_data_dir is also set to DATADIR. In case DATADIR is *not* a sibling of PREFIX, the result will be completely wrong, because the code will return PREFIX/DATADIR regardless. The harm is theoretical because I haven't heard of an installation where DATADIR and friends are not siblings of PREFIX. However both autotools and cmake do allow overrides of several installation paths and we try to follow this. So our (disabled) binreloc code should be able to handle it.
In that case we should change the defines in gncla-dir.h to contain CMAKE_FULL paths so that they're absolute. The current result with them being relative, presumably to PREFIX, also won't work in your hypothetical installation.
(In reply to John Ralls from comment #40) > Your change to binreloc.c makes the calls to gnc_gbr_find_prefix consistent > with the one in gnc-file.c, so I think that it's correct. Does it get the > bindings working? I've put PREFIX in and for some days the python bindings are running fine for me without the error message. It seems rather hacky though and to fit my specific situation. I'm happy if both of you figure out a more general approach fitting all possible scenarios.
(In reply to John Ralls from comment #11) > And reverted. Aside from it bothering Arch on Travis I found that it gets > called as part of the %init in bindings/python/gnucash_core.i so calling it > again from Python shouldn't be necessary. I don't see where and that binreloc gets called in %init. I put an early call to gnc_gbr_init in ( https://github.com/c-holtermann/gnucash/commit/fd9c09555e2e3def95b28caf523ea67494e3462f ). gnc_gbr_find_exe(NULL) returns true in that case. The path of the python executable gets found (as I described in comment 38) and my suggested workaround using PREFIX fails. As described before it works again if I put the python executable in the same tree as the libs. It doesn't seem to make sense to rely on binreloc in a python scenario as in figuring out a path starting with the executable, does it ? It seems to me that the python bindings did not got that way as there is or was no call to gnc_gbr_init. Can't find that call in the bindings at least. git grep gnc_gbr_init $(git rev-list --all -- src/optional) -- src/optional git grep gnc_gbr_init $(git rev-list --all -- bindings/python) -- bindings/python call may be in a subroutine, though. But right now when I call gnc_gbr_find_exe at the end of the %init it returns false so it doesn't look like gnc_gbr_init has been called anywhere.
Comment 11 was about gnc_environment_setup(), not gnc_gbr_init(). It doesn't make sense to depend on binreloc in its initialized state, no. Initializing binreloc negates your providing PREFIX as default_prefix to gnc_gbr_find_prefix() because if binreloc *is* initialized then exe *isn't* NULL and control skips down to calling g_path_get_dirname twice on the path to exe. Geert's concern in comment 41 is that someone might configure the build to arrange the various directories in a way that doesn't fit the prefix model and that GnuCash should still work in that case. To support that we need to change e.g. gnc_gbr_find_data_dir to do something like: gnc_gbr_find_data_dir(const char *default_data_dir) { char *prefix = NULL, *dir = NULL; if (DATADIR && g_path_is_absolute (DATADIR)) { prefix = gnc_gbr_find_prefix (NULL); if (prefix == NULL) return g_strdup (default_data_dir ? default_data_dir : DATADIR); } else { prefix = gnc_gbr_find_prefix (PREFIX); } /* prefix is NULL only if something's wrong with PREFIX so log an error. * Also no need to test default_data_dir, g_strdup will do the right * thing. */ g_return_val_if_fail (prefix != NULL), g_strdup (default_data_dir)); /* DATADIR isn't absolute so we don't need to get its relative part. */ dir = g_build_filename (prefix, DATADIR, NULL); g_free (prefix); return dir; } The comments are about differences to the existing code, I wouldn't include them in the real function. It occurs to me that if we're going to rewrite binreloc to this extent we should do it in C++ and use boost::filesystem instead of glib.
I've done something like comment 45 at https://github.com/jralls/gnucash/commit/f219bc45aa355a9f194bcaedcd72f511eb22fbcf Please test it.
(In reply to John Ralls from comment #46) > I've done something like comment 45 at > https://github.com/jralls/gnucash/commit/ > f219bc45aa355a9f194bcaedcd72f511eb22fbcf > > Please test it. with cmake -D CMAKE_INSTALL_PREFIX=$HOME/.local -D WITH_PYTHON=ON . compiling and installation as well as importing and running the python bindings work fine for me.
Very good, merged and pushed. It will be in GnuCash 3.3.
Thanks !
