Bug 1087615 - Update to python 3.4
Summary: Update to python 3.4
Keywords:
Status: CLOSED RAWHIDE
Alias: None
Product: Fedora
Classification: Fedora
Component: python3
Version: rawhide
Hardware: All
OS: Linux
unspecified
medium
Target Milestone: ---
Assignee: Matej Stuchlik
QA Contact: Fedora Extras Quality Assurance
URL: https://fedoraproject.org/wiki/Change...
Whiteboard:
: 1095069 (view as bug list)
Depends On: 1088233
Blocks:
TreeView+ depends on / blocked
 
Reported: 2014-04-14 20:12 UTC by Orion Poplawski
Modified: 2016-02-01 02:15 UTC (History)
8 users (show)

Fixed In Version:
Clone Of:
Environment:
Last Closed: 2014-06-16 07:23:00 UTC
Type: Bug
Embargoed:


Attachments (Terms of Use)
strace of hanging test_subprocess_send_signal (51.97 KB, text/plain)
2014-04-15 20:05 UTC, Orion Poplawski
no flags Details


Links
System ID Private Priority Status Summary Last Updated
Python 21247 0 None None None Never

Description Orion Poplawski 2014-04-14 20:12:06 UTC
Description of problem:

Let's update to python 3.4.  I see a python3.4 branch in git, is this the most recent work?  Looks like there may be changes done to master that should get merged in.  I updated that locally from 3.4rc2 to 3.4 final and did some scratch builds:

http://koji.fedoraproject.org/koji/taskinfo?taskID=6736790
http://koji.fedoraproject.org/koji/taskinfo?taskID=6736794

but they seem to be hanging in a test, last message:

test_subprocess_kill (test.test_asyncio.test_events.EPollEventLoopTests) ... ok

Willing to help out if needed.

Comment 1 Orion Poplawski 2014-04-14 20:42:33 UTC
I'm blind:

http://copr-fe.cloud.fedoraproject.org/coprs/bkabrda/python-3.4/

Comment 2 Matej Stuchlik 2014-04-15 07:42:16 UTC
Hey, as you've noticed we already have that COPR repo and even a Koji side tag [0], I'll be sending a mail to fedora/python devel about all that today, hang tight! :)

Matt

[0] https://fedorahosted.org/rel-eng/ticket/5884

Comment 3 Orion Poplawski 2014-04-15 14:33:20 UTC
I see that your koji build is hanging in the same place mine are :(.  I wonder what is different between the koji builders and the copr builders in this regard.  I can't reproduce locally either.

Comment 4 Orion Poplawski 2014-04-15 20:05:34 UTC
Created attachment 886645 [details]
strace of hanging test_subprocess_send_signal

It's hanging in the new asyncio tests - specifically test_subprocess_send_signal in test_asyncio/test_events.py.  Attached a relevant strace, looks like the parent process is stuck in epoll_wait().  Could this be caused by older kernels on the koji builders?

Comment 5 Orion Poplawski 2014-04-15 21:35:06 UTC
Filed upstream.

Comment 6 Orion Poplawski 2014-04-15 22:38:30 UTC
Looks like in the koji builds, the SIGHUP sigaction is set to SIG_IGN, which causes the processes that the python tests are trying to kill with SIGHUP not to die.  Perhaps the koji builders should not be doing that, perhaps the python tests should reset the SIGHUP sigaction to SIG_DFL.

Comment 7 Thomas Spura 2014-04-16 07:06:22 UTC
(In reply to Orion Poplawski from comment #6)
> Looks like in the koji builds, the SIGHUP sigaction is set to SIG_IGN, which
> causes the processes that the python tests are trying to kill with SIGHUP
> not to die.  Perhaps the koji builders should not be doing that, perhaps the
> python tests should reset the SIGHUP sigaction to SIG_DFL.

We have had a similar problem within pexpect. The workaround is to redefine the correct SIGHUB behavior:
https://github.com/pexpect/pexpect/commit/1fbfddf33d196fd1f211fb95efdaa810b8b5dad3

Comment 8 Marcela Mašláňová 2014-04-16 07:30:33 UTC
Interesting! I have also some strange builds, which pass in koji, but not in Copr.

Comment 9 Matej Stuchlik 2014-04-16 07:44:33 UTC
(In reply to Orion Poplawski from comment #3)
> I see that your koji build is hanging in the same place mine are :(.  I
> wonder what is different between the koji builders and the copr builders in
> this regard.  I can't reproduce locally either.

Yup, I initially thought this is a different bug I already had a workaround for :(

I figured this would be related to SIGHUP, since when you disable the test_subprocess_kill, the test suite still fails with

======================================================================
FAIL: test_send_signal (test.test_asyncio.test_subprocess.SubprocessFastWatcherTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/builddir/build/BUILD/Python-3.4.0/Lib/test/test_asyncio/test_subprocess.py", line 118, in test_send_signal
    self.assertEqual(-signal.SIGHUP, returncode)
AssertionError: -1 != 0

Comment 10 Matej Stuchlik 2014-04-16 09:19:29 UTC
Looking at the Koji repo, in koji/__init__.py there's the following:

def daemonize():
    [...]
    signal.signal(signal.SIGHUP, signal.SIG_IGN)
    [...]

Signal dispositions are inherited by child processes [0], but but Koji never seems to change SIGHUP back to SIG_DFL. Methinks this is a bug in Koji, right?

[0] http://man7.org/linux/man-pages/man2/sigaction.2.html#NOTES

Comment 11 Thomas Spura 2014-04-16 09:25:31 UTC
When having the problems with pexpect, I opened a ticked for koji:
https://fedorahosted.org/koji/ticket/270

Maybe opening a bug here at bugzilla would be a better option?

Comment 12 Matej Stuchlik 2014-04-16 09:28:10 UTC
(In reply to Thomas Spura from comment #11)
> When having the problems with pexpect, I opened a ticked for koji:
> https://fedorahosted.org/koji/ticket/270
> 
> Maybe opening a bug here at bugzilla would be a better option?

I imagine so, I'll do it.

Comment 13 Matej Stuchlik 2014-04-17 08:28:29 UTC
Seeing as resolving rhbz#1088233/pybt#21247 is probably going to take a while and since COPR and local builds are just fine, I'm going to go for the simplest solution: disable the affected tests for now and build without them.

Comment 14 Orion Poplawski 2014-04-18 19:34:05 UTC
Sounds good to me.

Comment 15 Donald Stufft 2014-04-30 17:03:36 UTC
It looks like ensurepip doesn't function at all if pip isn't already installed into the system. It fails with:

[root@test ~]# python3 -m ensurepip
Traceback (most recent call last):
  File "/usr/lib64/python3.4/runpy.py", line 171, in _run_module_as_main
    "__main__", mod_spec)
  File "/usr/lib64/python3.4/runpy.py", line 86, in _run_code
    exec(code, run_globals)
  File "/usr/lib64/python3.4/ensurepip/__main__.py", line 4, in <module>
    ensurepip._main()
  File "/usr/lib64/python3.4/ensurepip/__init__.py", line 233, in _main
    default_pip=args.default_pip,
  File "/usr/lib64/python3.4/ensurepip/__init__.py", line 97, in bootstrap
    dep_records = rewheel.find_system_records([p[0] for p in _PROJECTS])
  File "/usr/lib64/python3.4/ensurepip/rewheel/__init__.py", line 51, in find_system_records
    records.append(os.path.join(path, 'RECORD'))
  File "/usr/lib64/python3.4/posixpath.py", line 83, in join
    path += b
TypeError: unsupported operand type(s) for +=: 'NoneType' and 'str'

Comment 16 Matej Stuchlik 2014-05-06 09:54:02 UTC
Yes, you're right, I forgot to add pip and setuptools to Requires. Thanks for letting me know. :)

(In reply to Donald Stufft from comment #15)
> It looks like ensurepip doesn't function at all if pip isn't already
> installed into the system. It fails with:
> 
> [root@test ~]# python3 -m ensurepip
> Traceback (most recent call last):
>   File "/usr/lib64/python3.4/runpy.py", line 171, in _run_module_as_main
>     "__main__", mod_spec)
>   File "/usr/lib64/python3.4/runpy.py", line 86, in _run_code
>     exec(code, run_globals)
>   File "/usr/lib64/python3.4/ensurepip/__main__.py", line 4, in <module>
>     ensurepip._main()
>   File "/usr/lib64/python3.4/ensurepip/__init__.py", line 233, in _main
>     default_pip=args.default_pip,
>   File "/usr/lib64/python3.4/ensurepip/__init__.py", line 97, in bootstrap
>     dep_records = rewheel.find_system_records([p[0] for p in _PROJECTS])
>   File "/usr/lib64/python3.4/ensurepip/rewheel/__init__.py", line 51, in
> find_system_records
>     records.append(os.path.join(path, 'RECORD'))
>   File "/usr/lib64/python3.4/posixpath.py", line 83, in join
>     path += b
> TypeError: unsupported operand type(s) for +=: 'NoneType' and 'str'

Comment 17 Bohuslav "Slavek" Kabrda 2014-05-07 13:30:05 UTC
*** Bug 1095069 has been marked as a duplicate of this bug. ***

Comment 18 Onyeibo Oku 2014-06-16 01:05:47 UTC
Python3.4 has landed on Rawhide repos.
Shouldn't this be closed?

Comment 19 Matej Stuchlik 2014-06-16 07:23:00 UTC
(In reply to Onyeibo from comment #18)
> Python3.4 has landed on Rawhide repos.
> Shouldn't this be closed?

It should. :)


Note You need to log in before you can comment on or make changes to this bug.