Bug 1851622 - sshuttle crashes on "from shutil import which"
Summary: sshuttle crashes on "from shutil import which"
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Fedora
Classification: Fedora
Component: sshuttle
Version: 32
Hardware: x86_64
OS: Linux
unspecified
high
Target Milestone: ---
Assignee: Kevin Fenzi
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
: 1851744 (view as bug list)
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2020-06-27 15:27 UTC by Marcin Steć
Modified: 2020-11-16 20:28 UTC (History)
9 users (show)

Fixed In Version: sshuttle-1.0.4-1.fc32 sshuttle-1.0.4-1.el8 sshuttle-1.0.4-1.fc33
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2020-09-07 17:13:06 UTC
Type: Bug
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Github sshuttle sshuttle issues 469 0 None closed Removing python2 support makes sshuttle fail with old remotes 2020-11-14 22:42:07 UTC

Description Marcin Steć 2020-06-27 15:27:32 UTC
Description of problem:

After last bunch of Fedora updates, sshuttle stopped working. Displays this:

ImportError: cannot import name which
c : fatal: server died with error code 1


Version-Release number of selected component (if applicable):

sshuttle-1.0.1-1.fc32.noarch

How reproducible:
Every time I start the command.

Steps to Reproduce:
1.run this command

sshuttle -vv -r some.host.com 10.250.49.0/24 172.17.0.0/16

2.
3.

Actual results:

Starting sshuttle proxy.
firewall manager: Starting firewall with Python version 3.8.3
firewall manager: ready method name nat.
IPv6 enabled: None
UDP enabled: False
DNS enabled: False
User enabled: False
Binding redirector: 12300
TCP redirector listening on ('127.0.0.1', 12300).
TCP redirector listening with <socket.socket fd=5, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('127.0.0.1', 12300)>.
Starting client with Python version 3.8.3
c : connecting to server...
c : executing: ['ssh', '-p', '22', 'root@tsm1lz', '--', '/bin/sh -c \'P=python3; $P -V 2>/dev/null || P=python; exec "$P" -c \'"\'"\'import sys, os; verbosity=2; sys.stdin = os.fdopen(0, "rb"); exec(compile(sys.stdin.read(1275), "assembler.py", "exec"))\'"\'"\'\'']
c :  > channel=0 cmd=PING len=7 (fullness=0)
server: assembling u'sshuttle' (7 bytes)
server: assembling u'sshuttle.cmdline_options' (66 bytes)
server: assembling u'sshuttle.helpers' (902 bytes)
server: assembling u'sshuttle.ssnet' (5734 bytes)
server: assembling u'sshuttle.hostwatch' (2463 bytes)
server: assembling u'sshuttle.server' (3673 bytes)
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "assembler.py", line 25, in <module>
  File "sshuttle.server", line 9, in <module>
ImportError: cannot import name which
c : fatal: server died with error code 1


Expected results:

Client connected.

Additional info:

I checked the line 9 in server.py file on github. It is:

from shutil import which

I ran this line directly in default OS's Python (currently 3.8.3) and it reported no errors.

Comment 1 Kevin Fenzi 2020-06-27 17:52:45 UTC
So, this is an unfortunate fallout from upstream switching to all python3. ;( 

It seems the target server also needs to have python3 on it. 

If you install python3 on the target machine does it work?

Comment 2 Marcin Steć 2020-06-28 13:50:13 UTC
Yes, it works if target host is python3 capable.
I got only this warrning:

assembler.py:3: DeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses

However, I need to connect to quite old systems (SLES 11 SP4) where I have absolutely no administrative rights. 
Any chance for backward compatibility with Python 2 ? Or may be there is a way to use custom, home dir based python 3 installation?

Comment 3 Kevin Fenzi 2020-06-28 22:47:42 UTC
*** Bug 1851744 has been marked as a duplicate of this bug. ***

Comment 4 Kevin Fenzi 2020-06-28 22:52:52 UTC
(In reply to Marcin Steć from comment #2)
> Yes, it works if target host is python3 capable.
> I got only this warrning:
> 
> assembler.py:3: DeprecationWarning: the imp module is deprecated in favour
> of importlib; see the module's documentation for alternative uses
> 
> However, I need to connect to quite old systems (SLES 11 SP4) where I have
> absolutely no administrative rights. 
> Any chance for backward compatibility with Python 2 ?

It looks not too likely right now, but they are willing to accept PR's...
See: https://github.com/sshuttle/sshuttle/issues/469

>  Or may be there is a
> way to use custom, home dir based python 3 installation?

You can pass '--python /path/to/python' so that might work?

Comment 5 Yaroslav Sidlovsky 2020-07-01 09:23:00 UTC
Works for me after applying this 2 patches:
1. sshuttle-1.0.1-cannot-import-name-which.patch: https://gist.github.com/ZaWertun/67e86106d503380d45ddb2ac80962c49
2. sshuttle-1.0.1-no-attribute-set_blocking.patch: https://gist.github.com/ZaWertun/c587b0c3d9e2c2c4215b02a4efdda423

Comment 6 Kevin Fenzi 2020-07-01 16:34:13 UTC
Cool. Are those going to get submitted upstream?

Comment 7 Marcin Steć 2020-07-07 11:06:37 UTC
(In reply to Kevin Fenzi from comment #4)
> (In reply to Marcin Steć from comment #2)
> > Yes, it works if target host is python3 capable.
> > I got only this warrning:
> > 
> > assembler.py:3: DeprecationWarning: the imp module is deprecated in favour
> > of importlib; see the module's documentation for alternative uses
> > 
> > However, I need to connect to quite old systems (SLES 11 SP4) where I have
> > absolutely no administrative rights. 
> > Any chance for backward compatibility with Python 2 ?
> 
> It looks not too likely right now, but they are willing to accept PR's...
> See: https://github.com/sshuttle/sshuttle/issues/469
> 
> >  Or may be there is a
> > way to use custom, home dir based python 3 installation?
> 
> You can pass '--python /path/to/python' so that might work?

Hi!

I managed to try it out with remote SLES12 SP5 (Power9 Little Endian) host. It has python 3.4.10 on board. It looks I hit second bug @Yaroslaw Sidovsky mentioned. 

Here is what I get:

Starting sshuttle proxy.
firewall manager: Starting firewall with Python version 3.8.3
firewall manager: ready method name nat.
IPv6 enabled: None
UDP enabled: False
DNS enabled: False
User enabled: False
TCP redirector listening on ('127.0.0.1', 12300).
Starting client with Python version 3.8.3
c : connecting to server...
Starting server with Python version 3.4.10
 s: latency control setting = True
 s: auto-nets:False
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "assembler.py", line 41, in <module>
  File "sshuttle.server", line 395, in main
  File "sshuttle.ssnet", line 603, in runonce
  File "sshuttle.ssnet", line 493, in callback
  File "sshuttle.ssnet", line 442, in flush
AttributeError: 'module' object has no attribute 'set_blocking'
c : Connected.
c : fatal: other end: ConnectionResetError(104, 'Connection reset by peer')


Forking sshuttle-legacy to reach old jump-hosts seems to be a good idea.

Comment 8 Kevin Fenzi 2020-08-30 19:50:35 UTC
Upstream has landed a patch to bring back python2 compat on the remote side. 

Look for a update here in a few...

Comment 9 Fedora Update System 2020-08-30 20:36:26 UTC
FEDORA-2020-1892c7d4d3 has been submitted as an update to Fedora 32. https://bodhi.fedoraproject.org/updates/FEDORA-2020-1892c7d4d3

Comment 10 Fedora Update System 2020-08-30 20:36:26 UTC
FEDORA-2020-ae4b9b4efb has been submitted as an update to Fedora 33. https://bodhi.fedoraproject.org/updates/FEDORA-2020-ae4b9b4efb

Comment 11 Fedora Update System 2020-08-30 20:36:27 UTC
FEDORA-EPEL-2020-4684533f8f has been submitted as an update to Fedora EPEL 8. https://bodhi.fedoraproject.org/updates/FEDORA-EPEL-2020-4684533f8f

Comment 12 Fedora Update System 2020-08-31 15:55:14 UTC
FEDORA-2020-1892c7d4d3 has been pushed to the Fedora 32 testing repository.
In short time you'll be able to install the update with the following command:
`sudo dnf upgrade --enablerepo=updates-testing --advisory=FEDORA-2020-1892c7d4d3`
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2020-1892c7d4d3

See also https://fedoraproject.org/wiki/QA:Updates_Testing for more information on how to test updates.

Comment 13 Fedora Update System 2020-08-31 16:08:46 UTC
FEDORA-EPEL-2020-4684533f8f has been pushed to the Fedora EPEL 8 testing repository.

You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-EPEL-2020-4684533f8f

See also https://fedoraproject.org/wiki/QA:Updates_Testing for more information on how to test updates.

Comment 14 Fedora Update System 2020-08-31 18:57:12 UTC
FEDORA-2020-ae4b9b4efb has been pushed to the Fedora 33 testing repository.
In short time you'll be able to install the update with the following command:
`sudo dnf upgrade --enablerepo=updates-testing --advisory=FEDORA-2020-ae4b9b4efb`
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2020-ae4b9b4efb

See also https://fedoraproject.org/wiki/QA:Updates_Testing for more information on how to test updates.

Comment 15 Fedora Update System 2020-09-07 17:13:06 UTC
FEDORA-2020-1892c7d4d3 has been pushed to the Fedora 32 stable repository.
If problem still persists, please make note of it in this bug report.

Comment 16 Fedora Update System 2020-09-15 15:22:46 UTC
FEDORA-EPEL-2020-4684533f8f has been pushed to the Fedora EPEL 8 stable repository.
If problem still persists, please make note of it in this bug report.

Comment 17 Fedora Update System 2020-09-25 16:39:20 UTC
FEDORA-2020-ae4b9b4efb has been pushed to the Fedora 33 stable repository.
If problem still persists, please make note of it in this bug report.

Comment 18 Jamshid Afshar 2020-11-14 22:07:04 UTC
This breaks sshuttle => centos/rh 7 out of the box. Is there an alternative to "yum install python3"? This is a container so trying to avoid adding. Guess I could upgrade the container image to centos/rh 8.
Btw I have a "brew install" of sshuttle --version 1.0.4, this will be fixed in an upcoming patch?

Comment 19 Kevin Fenzi 2020-11-14 22:50:08 UTC
(In reply to Jamshid Afshar from comment #18)
> This breaks sshuttle => centos/rh 7 out of the box. Is there an alternative

It shouldn't. From what host version?

> to "yum install python3"? This is a container so trying to avoid adding.
> Guess I could upgrade the container image to centos/rh 8.
> Btw I have a "brew install" of sshuttle --version 1.0.4, this will be fixed
> in an upcoming patch?

It was already fixed in 1.0.4.

Comment 20 Jamshid Afshar 2020-11-15 17:59:14 UTC
Sorry I'm running sshuttle on my macos catalina, trying to connect to a centos 7.8 container running sshd that does not have python3. Installing python3 into the container fixes this error:

```
% uname -a
Darwin macbook 19.6.0 Darwin Kernel Version 19.6.0: Thu Oct 29 22:56:45 PDT 2020; root:xnu-6153.141.2.2~1/RELEASE_X86_64 x86_64

% brew list |grep -E 'sshuttle|python'
python
python
sshuttle

% sshuttle --version      
1.0.4

% sshuttle  --dns --no-latency-control --ssh-cmd "ssh -vv -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null" --remote root.1.50:4222  172.18.0.0/16          
OpenSSH_8.1p1, LibreSSL 2.7.3
debug1: Reading configuration data /Users/jamshid/.ssh/config
debug1: /Users/jamshid/.ssh/config line 13: Applying options for 192.168.1.50
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: /etc/ssh/ssh_config line 47: Applying options for *
debug2: resolve_canonicalize: hostname 192.168.1.50 is address
debug2: ssh_connect_direct
debug1: Connecting to 192.168.1.50 [192.168.1.50] port 4222.
debug1: Connection established.
debug1: identity file /Users/jamshid/.ssh/id_rsa type 0
debug1: identity file /Users/jamshid/.ssh/id_rsa-cert type -1
debug1: Local version string SSH-2.0-OpenSSH_8.1
debug1: Remote protocol version 2.0, remote software version OpenSSH_7.4
debug1: match: OpenSSH_7.4 pat OpenSSH_7.0*,OpenSSH_7.1*,OpenSSH_7.2*,OpenSSH_7.3*,OpenSSH_7.4*,OpenSSH_7.5*,OpenSSH_7.6*,OpenSSH_7.7* compat 0x04000002
debug2: fd 3 setting O_NONBLOCK
debug1: Authenticating to 192.168.1.50:4222 as 'root'
debug1: SSH2_MSG_KEXINIT sent
debug1: SSH2_MSG_KEXINIT received
debug2: local client KEXINIT proposal
debug2: KEX algorithms: curve25519-sha256,curve25519-sha256,ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group16-sha512,diffie-hellman-group18-sha512,diffie-hellman-group14-sha256,diffie-hellman-group14-sha1,ext-info-c
debug2: host key algorithms: ecdsa-sha2-nistp256-cert-v01,ecdsa-sha2-nistp384-cert-v01,ecdsa-sha2-nistp521-cert-v01,ssh-ed25519-cert-v01,rsa-sha2-512-cert-v01,rsa-sha2-256-cert-v01,ssh-rsa-cert-v01,ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521,ssh-ed25519,rsa-sha2-512,rsa-sha2-256,ssh-rsa
debug2: ciphers ctos: chacha20-poly1305,aes128-ctr,aes192-ctr,aes256-ctr,aes128-gcm,aes256-gcm
debug2: ciphers stoc: chacha20-poly1305,aes128-ctr,aes192-ctr,aes256-ctr,aes128-gcm,aes256-gcm
debug2: MACs ctos: umac-64-etm,umac-128-etm,hmac-sha2-256-etm,hmac-sha2-512-etm,hmac-sha1-etm,umac-64,umac-128,hmac-sha2-256,hmac-sha2-512,hmac-sha1
debug2: MACs stoc: umac-64-etm,umac-128-etm,hmac-sha2-256-etm,hmac-sha2-512-etm,hmac-sha1-etm,umac-64,umac-128,hmac-sha2-256,hmac-sha2-512,hmac-sha1
debug2: compression ctos: none,zlib,zlib
debug2: compression stoc: none,zlib,zlib
debug2: languages ctos: 
debug2: languages stoc: 
debug2: first_kex_follows 0 
debug2: reserved 0 
debug2: peer server KEXINIT proposal
debug2: KEX algorithms: curve25519-sha256,curve25519-sha256,ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group16-sha512,diffie-hellman-group18-sha512,diffie-hellman-group-exchange-sha1,diffie-hellman-group14-sha256,diffie-hellman-group14-sha1,diffie-hellman-group1-sha1
debug2: host key algorithms: ssh-rsa,rsa-sha2-512,rsa-sha2-256
debug2: ciphers ctos: chacha20-poly1305,aes128-ctr,aes192-ctr,aes256-ctr,aes128-gcm,aes256-gcm,aes128-cbc,aes192-cbc,aes256-cbc,blowfish-cbc,cast128-cbc,3des-cbc
debug2: ciphers stoc: chacha20-poly1305,aes128-ctr,aes192-ctr,aes256-ctr,aes128-gcm,aes256-gcm,aes128-cbc,aes192-cbc,aes256-cbc,blowfish-cbc,cast128-cbc,3des-cbc
debug2: MACs ctos: umac-64-etm,umac-128-etm,hmac-sha2-256-etm,hmac-sha2-512-etm,hmac-sha1-etm,umac-64,umac-128,hmac-sha2-256,hmac-sha2-512,hmac-sha1
debug2: MACs stoc: umac-64-etm,umac-128-etm,hmac-sha2-256-etm,hmac-sha2-512-etm,hmac-sha1-etm,umac-64,umac-128,hmac-sha2-256,hmac-sha2-512,hmac-sha1
debug2: compression ctos: none,zlib
debug2: compression stoc: none,zlib
debug2: languages ctos: 
debug2: languages stoc: 
debug2: first_kex_follows 0 
debug2: reserved 0 
debug1: kex: algorithm: curve25519-sha256
debug1: kex: host key algorithm: rsa-sha2-512
debug1: kex: server->client cipher: chacha20-poly1305 MAC: <implicit> compression: none
debug1: kex: client->server cipher: chacha20-poly1305 MAC: <implicit> compression: none
debug1: expecting SSH2_MSG_KEX_ECDH_REPLY
debug1: Server host key: ssh-rsa SHA256:4otFXL8eRvE82d/wHvxiESCVDfbmQmstEQ88F26PZFw
debug1: checking without port identifier
Warning: Permanently added '[192.168.1.50]:4222' (RSA) to the list of known hosts.
debug2: set_newkeys: mode 1
debug1: rekey out after 134217728 blocks
debug1: SSH2_MSG_NEWKEYS sent
debug1: expecting SSH2_MSG_NEWKEYS
debug1: SSH2_MSG_NEWKEYS received
debug2: set_newkeys: mode 0
debug1: rekey in after 134217728 blocks
debug1: Will attempt key: /Users/jamshid/.ssh/id_rsa RSA SHA256:16O5Al/pfJueCQ9vKvf7dXxCk9iqLCzsxKmmkHR3hVU explicit
debug2: pubkey_prepare: done
debug1: SSH2_MSG_EXT_INFO received
debug1: kex_input_ext_info: server-sig-algs=<rsa-sha2-256,rsa-sha2-512>
debug2: service_accept: ssh-userauth
debug1: SSH2_MSG_SERVICE_ACCEPT received
debug1: Authentications that can continue: publickey,password
debug1: Next authentication method: publickey
debug1: Offering public key: /Users/jamshid/.ssh/id_rsa RSA SHA256:16O5Al/pfJueCQ9vKvf7dXxCk9iqLCzsxKmmkHR3hVU explicit
debug2: we sent a publickey packet, wait for reply
debug1: Authentications that can continue: publickey,password
debug2: we did not send a packet, disable method
debug1: Next authentication method: password
root.1.50's password: 
debug2: we sent a password packet, wait for reply
debug1: Authentication succeeded (password).
Authenticated to 192.168.1.50 ([192.168.1.50]:4222).
debug2: fd 4 setting O_NONBLOCK
debug1: channel 0: new [client-session]
debug2: channel 0: send open
debug1: Requesting no-more-sessions
debug1: Entering interactive session.
debug1: pledge: network
debug1: client_input_global_request: rtype hostkeys-00 want_reply 0
debug2: channel_input_open_confirmation: channel 0: callback start
debug2: fd 3 setting TCP_NODELAY
debug2: client_session2_setup: id 0
debug1: Sending environment.
debug1: Sending env LANG = en_US.UTF-8
debug2: channel 0: request env confirm 0
debug1: Sending command: /bin/sh -c 'P=python3; $P -V 2>/dev/null || P=python; exec "$P" -c '"'"'import sys, os; verbosity=0; sys.stdin = os.fdopen(0, "rb"); exec(compile(sys.stdin.read(1279), "assembler.py", "exec"))'"'"''
debug2: channel 0: request exec confirm 1
debug2: channel_input_open_confirmation: channel 0: callback done
debug2: channel 0: open confirm rwindow 0 rmax 32768
debug2: channel 0: rcvd adjust 2097152
debug2: channel_input_status_confirm: type 99 id 0
debug2: exec request accepted on channel 0
debug2: channel 0: rcvd ext data 74
debug2: channel 0: rcvd ext data 44
debug2: channel 0: rcvd ext data 68
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "assembler.py", line 18, in <module>
TypeError: module.__init__() argument 1 must be string, not unicode
debug2: channel 0: written 186 to efd 6
debug1: client_input_channel_req: channel 0 rtype exit-status reply 0
debug1: client_input_channel_req: channel 0 rtype eow reply 0
debug2: channel 0: rcvd eow
debug2: channel 0: chan_shutdown_read (i0 o0 sock -1 wfd 4 efd 6 [write])
debug2: channel 0: input open -> closed
debug2: channel 0: rcvd eof
ebug2: channel 0: output open -> drain
debug2: channel 0: obuf empty
debug2: channel 0: chan_shutdown_write (i3 o1 sock -1 wfd 5 efd 6 [write])
debug2: channel 0: output drain -> closed
debug2: channel 0: rcvd close
debug2: channel 0: almost dead
debug2: channel 0: gc: notify user
debug2: channel 0: gc: user detached
debug2: channel 0: send close
debug2: channel 0: is dead
debug2: channel 0: garbage collecting
debug1: channel 0: free: client-session, nchannels 1
debug1: fd 0 clearing O_NONBLOCK
Transferred: sent 16792, received 2824 bytes, in 0.0 seconds
Bytes per second: sent 442756.9, received 74460.8
debug1: Exit status 1
client: fatal: expected server init string b'SSHUTTLE0001'; got b''
```

Comment 21 Kevin Fenzi 2020-11-16 20:28:38 UTC
No idea here. I have never used osx... I'd suggest reporting it upstream?


Note You need to log in before you can comment on or make changes to this bug.