Bug 817554
| Summary: | UsrMove broke Py_Initialize() | ||||||
|---|---|---|---|---|---|---|---|
| Product: | [Fedora] Fedora | Reporter: | Jan Safranek <jsafrane> | ||||
| Component: | python | Assignee: | Dave Malcolm <dmalcolm> | ||||
| Status: | CLOSED RAWHIDE | QA Contact: | Fedora Extras Quality Assurance <extras-qa> | ||||
| Severity: | high | Docs Contact: | |||||
| Priority: | unspecified | ||||||
| Version: | 17 | CC: | dFyD0d4hFT0P, dmalcolm, info, ivazqueznet, jonathansteffan, tomspur | ||||
| Target Milestone: | --- | ||||||
| Target Release: | --- | ||||||
| Hardware: | Unspecified | ||||||
| OS: | Unspecified | ||||||
| Whiteboard: | |||||||
| Fixed In Version: | python-2.7.3-34.fc19 | Doc Type: | Bug Fix | ||||
| Doc Text: | Story Points: | --- | |||||
| Clone Of: | |||||||
| : | 903985 (view as bug list) | Environment: | |||||
| Last Closed: | 2013-03-06 22:34:13 UTC | Type: | Bug | ||||
| Regression: | --- | Mount Type: | --- | ||||
| Documentation: | --- | CRM: | |||||
| Verified Versions: | Category: | --- | |||||
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |||||
| Cloudforms Team: | --- | Target Upstream Version: | |||||
| Embargoed: | |||||||
| Bug Depends On: | |||||||
| Bug Blocks: | 903985 | ||||||
| Attachments: |
|
||||||
Thanks for filing this bug report. I'm able to reproduce this, and am investigating.
Note to self:
This is happening because sys.path is erroneously getting initialized to:
['lib64/python27.zip', '/lib64/python2.7/', '/lib64/python2.7/plat-linux2', '/lib64/python2.7/lib-tk', '/lib64/python2.7/lib-old', '/lib64/python2.7/lib-dynload']
rather than to the "/usr/"-prefixed paths.
The initial value for sys.path comes from calculate_path(), in Modules/getpath.c
Summary of the logic in calculate_path():
if there's a slash in Py_GetProgramName()
then:
it uses Py_GetProgramName() as argv0_path; get the containing dir as argv0_path
else:
it tries to find Py_GetProgramName() within $PATH
if found:
it uses the abspath it finds; get the containing dir as argv0_path
else: (not found on $PATH):
argv0_path == ""
if PYTHONHOME not set (step 1):
if argv0_path is not in the build directory (step 2):
Step 3: search for "/lib64/python2.7/os.py" in argv0_path:
(*) the bug is happening due to it finds this; previously it wouldn't have
Step 4: search within PREFIX/EXEC_PREFIX from preprocessor (which are set to /usr at build time by the Makefile)
(**) previously it was using this
So it appears that this bug will happen when:
(1) there is no slash in Py_GetProgramName() OR it is a path to within the root directory (defaults to "python")
AND
(2) Py_GetProgramName() is *not* found in $PATH (defaults to "python", so this *is* normally found, albeit as/bin/python, but since /bin/lib64 doesn't exist the test at step 3 fails)
AND
(3) $PYTHONHOME is not set
It looks like previously this combination would have fallen back to "Step 4", using the PREFIX and EXEC_PREFIX from the Makefile, but now it's getting a false success at "Step 3".
Any progress with this bug? I am still able to reproduce it with python-2.7.3-13.fc18.x86_64 on Fedora 18 pre-alpha. Same for me, I have to symlink /include to /usr/include to make the Python config work. Candidate workaround committed to git as: http://pkgs.fedoraproject.org/cgit/python.git/commit/?id=4e8ad3c0992c75b72400b3a99c7886d275142ebc Building python-2.7.3-33.fc19 for rawhide Task info: http://koji.fedoraproject.org/koji/taskinfo?taskID=5086641 (In reply to comment #4) > Candidate workaround committed to git as: > http://pkgs.fedoraproject.org/cgit/python.git/commit/ > ?id=4e8ad3c0992c75b72400b3a99c7886d275142ebc > > Building python-2.7.3-33.fc19 for rawhide > Task info: http://koji.fedoraproject.org/koji/taskinfo?taskID=5086641 Failed test_sys and test_subprocess when run from the build directory. Revised candidate workaround committed to git as: http://pkgs.fedoraproject.org/cgit/python.git/commit/?id=1b2a274a94e296f929715acd189bbe6c028861b1 Building python-2.7.3-34.fc19 for rawhide Task info: http://koji.fedoraproject.org/koji/taskinfo?taskID=5086849 Build of python-2.7.3-34.fc19 succeeded Is this going to be fixed for fc18 as well? |
Created attachment 581187 [details] reproducer Description of problem: I have a daemon with embedded python. When the daemon is started via systemd (->working directory is /), following code fails: Py_SetProgramName("cmpi_swig"); Py_Initialize(); The code is not mine, it's from cmpi-bindings project and was working perfectly in older Fedora. Version-Release number of selected component (if applicable): python-2.7.3-4.fc18.x86_64 How reproducible: always Steps to Reproduce: 1. compile attached reproducer: gcc -o reproduce reproduce.c -lpython2.7 -I/usr/include/python2.7 2. cd / 3. $OLDPWD/reproduce Actual results: Traceback (most recent call last): File "/lib64/python2.7/site.py", line 567, in <module> main() File "/lib64/python2.7/site.py", line 549, in main known_paths = addusersitepackages(known_paths) File "/lib64/python2.7/site.py", line 278, in addusersitepackages user_site = getusersitepackages() File "/lib64/python2.7/site.py", line 253, in getusersitepackages user_base = getuserbase() # this will also set USER_BASE File "/lib64/python2.7/site.py", line 243, in getuserbase USER_BASE = get_config_var('userbase') File "/lib64/python2.7/sysconfig.py", line 521, in get_config_var return get_config_vars().get(name) File "/lib64/python2.7/sysconfig.py", line 420, in get_config_vars _init_posix(_CONFIG_VARS) File "/lib64/python2.7/sysconfig.py", line 299, in _init_posix raise IOError(msg) IOError: invalid Python installation: unable to open //include/python2.7/pyconfig-64.h (No such file or directory) Expected results: no error reported Additional info: Looking at strace output, python tries to 'find itself'. On Fedora16 (=before usrmove): stat("/usr/lib64/qt-3.3/bin/cmpi_swig", 0x7fffc439d0e0) = -1 ENOENT (No such file or directory) stat("/usr/local/bin/cmpi_swig", 0x7fffc439d0e0) = -1 ENOENT (No such file or directory) stat("/usr/bin/cmpi_swig", 0x7fffc439d0e0) = -1 ENOENT (No such file or directory) stat("/bin/cmpi_swig", 0x7fffc439d0e0) = -1 ENOENT (No such file or directory) stat("/usr/games/cmpi_swig", 0x7fffc439d0e0) = -1 ENOENT (No such file or directory) stat("/usr/local/sbin/cmpi_swig", 0x7fffc439d0e0) = -1 ENOENT (No such file or directory) stat("/usr/sbin/cmpi_swig", 0x7fffc439d0e0) = -1 ENOENT (No such file or directory) stat("/sbin/cmpi_swig", 0x7fffc439d0e0) = -1 ENOENT (No such file or directory) stat("/home/jsafrane/bin/cmpi_swig", 0x7fffc439d0e0) = -1 ENOENT (No such file or directory) stat("/opt/eclipse/cmpi_swig", 0x7fffc439d0e0) = -1 ENOENT (No such file or directory) readlink("", 0x7fffc439f190, 4096) = -1 ENOENT (No such file or directory) stat("Modules/Setup", 0x7fffc439d030) = -1 ENOENT (No such file or directory) getcwd("/", 4096) = 2 stat("/lib64/python2.7/os.py", 0x7fffc439d020) = -1 ENOENT (No such file or directory) stat("/lib64/python2.7/os.pyc", 0x7fffc439d020) = -1 ENOENT (No such file or directory) stat("/usr/lib64/python2.7/os.py", {st_mode=S_IFREG|0644, st_size=26300, ...}) = 0 stat("Modules/Setup", 0x7fffc439d030) = -1 ENOENT (No such file or directory) getcwd("/", 4096) = 2 stat("/lib64/python2.7/lib-dynload", 0x7fffc439d030) = -1 ENOENT (No such file or directory) stat("/usr/lib64/python2.7/lib-dynload", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0 -> it *does not* find itself in / (current working directory) and continues with /usr/ On Rawhide: stat("/usr/local/sbin/cmpi_swig", 0x7fffd54784f0) = -1 ENOENT (No such file or directory) stat("/sbin/cmpi_swig", 0x7fffd54784f0) = -1 ENOENT (No such file or directory) stat("/bin/cmpi_swig", 0x7fffd54784f0) = -1 ENOENT (No such file or directory) stat("/usr/sbin/cmpi_swig", 0x7fffd54784f0) = -1 ENOENT (No such file or directory) stat("/usr/bin/cmpi_swig", 0x7fffd54784f0) = -1 ENOENT (No such file or directory) stat("/root/bin/cmpi_swig", 0x7fffd54784f0) = -1 ENOENT (No such file or directory) readlink("", 0x7fffd5479590, 4096) = -1 ENOENT (No such file or directory) stat("Modules/Setup", 0x7fffd5478440) = -1 ENOENT (No such file or directory) getcwd("/", 4096) = 2 stat("/lib64/python2.7/os.py", {st_mode=S_IFREG|0644, st_size=25769, ...}) = 0 stat("Modules/Setup", 0x7fffd5478440) = -1 ENOENT (No such file or directory) getcwd("/", 4096) = 2 stat("/lib64/python2.7/lib-dynload", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0 -> it *finds* itself in / (/lib64/python2.7/lib-dynload) and expect //include/python2.7/pyconfig-64.h exists -> error