Bug 2082270 - [RFE] ncat: change default behavior and close immediately when the server is done sending
Summary: [RFE] ncat: change default behavior and close immediately when the server is ...
Keywords:
Status: CLOSED WONTFIX
Alias: None
Product: Fedora
Classification: Fedora
Component: nmap
Version: rawhide
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: ---
Assignee: Martin Osvald 🛹
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2022-05-05 17:17 UTC by Martin Osvald 🛹
Modified: 2022-05-10 15:52 UTC (History)
5 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2022-05-10 15:52:47 UTC
Type: Bug


Attachments (Terms of Use)

Description Martin Osvald 🛹 2022-05-05 17:17:36 UTC
This bug was initially created as a copy of Bug #1890475

I am copying this bug because: 

Bring this feature into Fedora first


3. What is the nature and description of the request?

- Customer is using ncat to transfer files between two hosts by using rsync
- when copy operation is finished server closed socket for writing by calling shutdown(sd, SHUT_WR).
- client sees this and continues, because it can still send data (if you do not use --recv-only option)
- client socket keeps open (CLOSE_WAIT) even server has send shutdown
- when server socket is closed, ncat can't distinguish whether it's closed or half closed, until it tries to send something.


4. Why does the customer need this? (List the business requirements here)

- cp files/data between two sockets and terminate also client when server is done


5. How would the customer like to achieve this? (List the functional requirements here)

a) change the default behavior, so that client also performs shutdown after receiving server shutdown
b) Allowing the -k option to bring back the (current) keep alive behavior in client mode


6. For each functional requirement listed, specify how Red Hat and the customer can test to confirm the requirement is successfully implemented.

Example:

a)

root@server# useradd user1
root@server# su - user1
user1@server# ncat -4 --allow <client IP> -l -e /bin/bash

root@client# ncat <server IP>
exit

root@client# lsof -p <ncat pid> | grep CLOSE_WAIT
root@client#


b)

root@server# useradd user1
root@server# su - user1
root@server# ncat -4 --allow <client IP> -k -l -e /bin/bash

root@client# ncat <server IP>
exit

root@client# lsof -p <ncat pid> | grep CLOSE_WAIT
COMMAND    PID USER   FD   TYPE DEVICE SIZE/OFF      NODE NAME
ncat    <pid> root    3u  IPv4 468854      0t0       TCP <client>:50546-><server>:31337 (CLOSE_WAIT)


7. Is there already an existing RFE upstream or in Red Hat Bugzilla?

- found RHBZ#1078244 -> it is confirmed that the actual behavior is not a bug and BZ is closed with WONTFIX
- found upstream mailing list [https://seclists.org/nmap-dev/2017/q2/134] where upstream maintainer suggest to change default behavior and implement parameter -k
- found [https://github.com/nmap/nmap/issues/1779] where an extra argument --exit-on-close is requested to change the current behavior


8. Does the customer have any specific timeline dependencies and which release would they like to target (i.e. RHEL5, RHEL6)?

- no, asap


9. Is the sales team involved in this request and do they have any additional input?

- no


10. List any affected packages or components.

- nmap-ncat


11. Would the customer be able to assist in testing this functionality if implemented?

- yes

Comment 2 Fedora Update System 2022-05-06 08:28:58 UTC
FEDORA-2022-de7fea2a3f has been submitted as an update to Fedora 36. https://bodhi.fedoraproject.org/updates/FEDORA-2022-de7fea2a3f

Comment 3 Fedora Update System 2022-05-06 09:38:12 UTC
FEDORA-2022-20b1146027 has been submitted as an update to Fedora 35. https://bodhi.fedoraproject.org/updates/FEDORA-2022-20b1146027

Comment 4 Fedora Update System 2022-05-06 11:02:20 UTC
FEDORA-2022-de7fea2a3f has been pushed to the Fedora 36 testing repository.
Soon you'll be able to install the update with the following command:
`sudo dnf upgrade --enablerepo=updates-testing --advisory=FEDORA-2022-de7fea2a3f`
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2022-de7fea2a3f

See also https://fedoraproject.org/wiki/QA:Updates_Testing for more information on how to test updates.

Comment 5 Fedora Update System 2022-05-08 03:26:54 UTC
FEDORA-2022-20b1146027 has been pushed to the Fedora 35 testing repository.
Soon you'll be able to install the update with the following command:
`sudo dnf upgrade --enablerepo=updates-testing --advisory=FEDORA-2022-20b1146027`
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2022-20b1146027

See also https://fedoraproject.org/wiki/QA:Updates_Testing for more information on how to test updates.

Comment 6 Lyosha Alexeev 2022-05-08 21:18:24 UTC
This change caused several of my automated tasks that relied on this feature to fail.
Couldn't this be handled in a different way than changing the default behavior when there isn't even a minor version bump involved?

It took some time figuring out it was Ncat's fault and not some server side fluke. There isn't even a manpage entry for "--no-terminate".

Comment 7 Martin Osvald 🛹 2022-05-09 04:17:51 UTC
Hi Lyosha,

Thank you for this valuable feedback. Just increased positive karma needed to get this update into the stable to hopefully gather more.

Please, do you think you could share some bits and pieces of your tasks script(s) which are using ncat and failed without using --no-terminate option so that I can see the use case where this change in default behavior fails?

Comment 8 Fedora Update System 2022-05-09 12:49:38 UTC
FEDORA-2022-de7fea2a3f has been pushed to the Fedora 36 testing repository.
Soon you'll be able to install the update with the following command:
`sudo dnf upgrade --enablerepo=updates-testing --advisory=FEDORA-2022-de7fea2a3f`
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2022-de7fea2a3f

See also https://fedoraproject.org/wiki/QA:Updates_Testing for more information on how to test updates.

Comment 9 Lyosha Alexeev 2022-05-09 14:01:25 UTC
(In reply to Martin Osvald from comment #7)

> Please, do you think you could share some bits and pieces of your tasks
> script(s) which are using ncat and failed without using --no-terminate
> option so that I can see the use case where this change in default behavior
> fails?


- A rudimentary container health check where I send a magic string to a service in order for it to return the exact version running, which then gets parsed. Without --no-terminate, the result is empty.

  echo VERSION | ncat 127.0.0.1 9999
  This correctly sends VERSION to the service but doesn't await the response.

  E.g. checking response from redis: PONG=$(echo 'PING' | ncat 127.0.0.1 6379)

- Using a wrapper around a tcp pastebin service like unsha.re or termbin.com, e.g. echo foo | ncat unsha.re 9999
  This worked fine previously and now requires --no-terminate or using something like socat to wait for the resulting URL.

Basically everything that requires to wait for an answer after piping in some string. Which is, as I understand it, the desired behavior. But breaking the default behavior is unfortunate.

Comment 10 Ed Santiago 2022-05-10 14:39:12 UTC
This completely breaks common podman use cases:

    # podman run -d --name foo  -p 127.0.0.1:52000:52000 quay.io/libpod/testimage:20210610 nc -l -n -v -p 52000
    [containerid]
    # echo hi | nc 127.0.0.1 52000
    # podman logs foo
    [does not show "hi"]

Running 'nc -v -v -v -v' on a good (f35) and bad (f37) system shows that the broken nc (i.e. this one) is missing:

    libnsock nsock_trace_handler_callback(): Callback: READ SUCCESS for EID 26 [peer unspecified] (3 bytes): hi.
    libnsock event_new(): event_new (IOD #1) (EID #35)
    libnsock nsock_write(): Write request for 3 bytes to IOD #1 EID 35 [127.0.0.1:52000]
    ...and a few more...

The new flag is not acceptable: it is impossible to go back and fix all versions of podman tests in the field and in current maintenance branches. It is also impossible even to add that flag to current CI, because obviously the flag will not be recognized on older versions of nc.

I urge you to please back this out.

Comment 11 Martin Osvald 🛹 2022-05-10 15:52:47 UTC
Thank you for the very valuable feedback. I canceled the updates for f35, f36, and will be reverting the changes for them including rawhide.

I WONTFIX this for Fedora and will for RHEL too.


Note You need to log in before you can comment on or make changes to this bug.