Bug 1270166

Summary: UDP packet checksum is not converted from 0x0000 to 0xffff with Qemu e1000 emulation.
Product: Red Hat Enterprise Linux 7 Reporter: Yuki Shibuya <shibuya.yk>
Component: qemu-kvmAssignee: Philippe Mathieu-Daudé <philmd>
Status: CLOSED ERRATA QA Contact: Quan Wenli <wquan>
Severity: medium Docs Contact:
Priority: medium    
Version: 7.1CC: aadam, ailan, andbartl, areis, chayang, ebohn, jasowang, juzhang, knoel, maurizio.antillon, pezhang, philmd, rkhan, shibuya.yk, toyoshima.ts, virt-maint, xfu
Target Milestone: rc   
Target Release: ---   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: qemu-kvm-1.5.3-168.el7 Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2020-03-31 20:02:00 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Yuki Shibuya 2015-10-09 07:38:11 UTC
Description of problem:

UDP packet checksum is not converted from 0x0000 to 0xffff with Qemu e1000 emulation.

* HostOS: RHEL7.1
$ uname -a
Linux cge4nfv01 3.10.0-229.7.2.el7.x86_64 #1 SMP Fri May 15 21:38:46 EDT 2015 x86_64 x86_64 x86_64 GNU/Linux

* QEMU: 1.5.3-86.el7_1.5.x86_64
$ rpm -qa | grep qemu
qemu-img-1.5.3-86.el7_1.5.x86_64
qemu-kvm-1.5.3-86.el7_1.5.x86_64
libvirt-daemon-driver-qemu-1.2.8-16.el7_1.3.x86_64
qemu-kvm-tools-1.5.3-86.el7_1.5.x86_64
ipxe-roms-qemu-20130517-6.gitc4bce43.el7.noarch
qemu-kvm-common-1.5.3-86.el7_1.5.x86_64

* Host CPU: Haswell
 
* GUEST OS: CentOS 7
# uname -a
Linux demovm 3.10.0-229.4.2.el7.x86_64 #1 SMP Wed May 13 10:06:09 UTC 2015 x86_64 x86_64 x86_64 GNU/Linux

* Use e1000 model NIC on GUEST

* Send udp packet from GUEST VM to destination server (that is physical server and not host server).

# ethtool -i ens7
driver: e1000
version: 7.3.21-k8-NAPI
firmware-version:
bus-info: 0000:00:07.0
supports-statistics: yes
supports-test: yes
supports-eeprom-access: yes
supports-register-dump: yes
supports-priv-flags: no

* Enable tx-checksumming offload
# ethtool -k ens7
Features for ens7:
rx-checksumming: off
tx-checksumming: on
        tx-checksum-ipv4: off [fixed]
        tx-checksum-ip-generic: on
        tx-checksum-ipv6: off [fixed]
        tx-checksum-fcoe-crc: off [fixed]
        tx-checksum-sctp: off [fixed]
scatter-gather: on
        tx-scatter-gather: on
        tx-scatter-gather-fraglist: off [fixed]
tcp-segmentation-offload: on
        tx-tcp-segmentation: on
        tx-tcp-ecn-segmentation: off [fixed]
        tx-tcp6-segmentation: off [fixed]

* UDP packet checksum is not converted from 0x0000 to 0xffff with Qemu e1000 emulation.

I sent a udp packet that computed checksum is zero from GUEST CentOS7 to target destination server.
I captured udp packets on destination server, and found a udp packet having checksum=0.

09:57:22.290053 IP 192.168.100.220.50000 > octopusFREE.italk: UDP, length 4
        0x0000:  4500 0020 38b4 4000 4011 b718 c0a8 64dc  E...8.@.@.....d.
        0x0010:  c0a8 64d3 c350 3039 000c 0000 c14b 0000  ..d..P09.....K..
                                          ~~~~checksum is zero.
        0x0020:  0000 0000 0000 0000 0000 0000 0000

In RFC 768(https://www.ietf.org/rfc/rfc768.txt), it is described as follows 
about such udp packet.
 
[If the computed  checksum  is zero,  it is transmitted  as all ones (the
 equivalent  in one's complement  arithmetic).   An all zero  transmitted
 checksum  value means that the transmitter  generated  no checksum  (for
 debugging or for higher level protocols that don't care).]

When tx-checksumming offload was disabled, I got a following caputure of same packet.

# ethtool -k ens7
Features for ens7:
rx-checksumming: off
tx-checksumming: off
        tx-checksum-ipv4: off [fixed]
        tx-checksum-ip-generic: off
        tx-checksum-ipv6: off [fixed]
        tx-checksum-fcoe-crc: off [fixed]
        tx-checksum-sctp: off [fixed]
scatter-gather: on
        tx-scatter-gather: on
        tx-scatter-gather-fraglist: off [fixed]
tcp-segmentation-offload: off
        tx-tcp-segmentation: off [requested on]
        tx-tcp-ecn-segmentation: off [fixed]
        tx-tcp6-segmentation: off [fixed]

09:52:29.316050 IP 192.168.100.220.50000 > octopusFREE.italk: UDP, length 4
        0x0000:  4500 0020 3032 4000 4011 bf9a c0a8 64dc  E...02@.@.....d.
        0x0010:  c0a8 64d3 c350 3039 000c ffff c14b 0000  ..d..P09.....K..
                                          ~~~~checksum is 0xffff.
        0x0020:  0000 0000 0000 0000 0000 0000 0000       ..............

Version-Release number of selected component (if applicable):

$ uname -a
Linux cge4nfv01 3.10.0-229.7.2.el7.x86_64 #1 SMP Fri May 15 21:38:46 EDT 2015 x86_64 x86_64 x86_64 GNU/Linux

$ rpm -qa | grep qemu
qemu-img-1.5.3-86.el7_1.5.x86_64
qemu-kvm-1.5.3-86.el7_1.5.x86_64
libvirt-daemon-driver-qemu-1.2.8-16.el7_1.3.x86_64
qemu-kvm-tools-1.5.3-86.el7_1.5.x86_64
ipxe-roms-qemu-20130517-6.gitc4bce43.el7.noarch
qemu-kvm-common-1.5.3-86.el7_1.5.x86_64

How reproducible:

To reproduce this, we need two physical servers.
One is host server to create VM, the other one is destination server.

Steps to Reproduce:
1. Create CentOS 7 VM.
2. Set e1000 NIC tyep to created VM.
3. Enable tx-checksumming offload on VM
4. Execute tcpdump to capture udp packets on destination server.
5. Send a udp packet that computed checksum is zero to destination server.

I created a udp packet that computed checksum is zero as follows.

- 1. Send a udp packet having zero message by using following test program,
     and capture the packet on destination server. 

This test program is executed as follows:
# ./a.out <destination address>

===================== test program =======================
#include <sys/types.h>
#include <sys/socket.h>
#include <netinet/in.h>
#include <errno.h>
#include <stdio.h>

int main(int argc, char *argv[])
{
        int sock, ret = 0;
        int msg;
        struct sockaddr_in addr, s_addr;

        if (argc !=2) {
                printf("usage: %s <IPv4 addr> \n", argv[0]);
                return 0;
        }

        sock = socket(AF_INET, SOCK_DGRAM, 0);

        s_addr.sin_family = AF_INET;
        s_addr.sin_port = htons(50000);

        if(bind(sock, (struct sockaddr *)&s_addr, sizeof(s_addr)) < 0) {
                printf("bind() errno = %d(%s)\n", errno, strerror(errno));
        }

        addr.sin_family = AF_INET;
        addr.sin_port = htons(12345);
        addr.sin_addr.s_addr = inet_addr(argv[1]);

        /*
         * data = 0 or
         * udp checksum is converted into host byte order
         */
        msg = 0;

        if (sendto(sock, &msg, sizeof(msg), 0, (struct sockaddr *)&addr, sizeof(addr)) < 0)
                printf("sendto() errno = %d(%s)\n", errno, strerror(errno));

        close(sock);

        return 0;
}
===================== test program =======================

- 2. Check checksum for zero message, and rebuild test program to
     send a udp packet that computed checksum is zero.

We will get following capture on destination server.

09:56:49.583568 IP 192.168.100.220.50000 > octopusFREE.italk: UDP, length 4
        0x0000:  4500 0020 38b3 4000 4011 b719 c0a8 64dc  E...8.@.@.....d.
        0x0010:  c0a8 64d3 c350 3039 000c c14b 0000 0000  ..d..P09...K....
                                          ~~~~checksum for zero message
        0x0020:  0000 0000 0000 0000 0000 0000 0000       ..............

We convert this checksum from network byte order to host byte order.
We set the result as message of test program, and rebuild test program.
In this case, checksum for zero message is c14b. Therefore, we set 
4bc1 to the message of test program as follows:

msg = 0x4bc1;

- 3. Execute test program after rebuild on same GUEST and same destination address.

Actual results:

When computed UDP packet checksum is 0x0000, this checksum is not converted from 
0x0000 to 0xffff with Qemu e1000 emulation.

Expected results:

When computed UDP packet checksum is 0x0000, this checksum is converted from 
0x0000 to 0xffff with Qemu e1000 emulation.

Additional info:

I consider that a putting checksum to packets is implemented at following code.
I think, in this process, the case of that computed checksum is zero based on RFC 768
is not considered.

hw/net/e1000.c
static void
putsum(uint8_t *data, uint32_t n, uint32_t sloc, uint32_t css, uint32_t cse)
{
    uint32_t sum;

    if (cse && cse < n)
        n = cse + 1;
    if (sloc < n-1) {
        sum = net_checksum_add(n-css, data+css);
        cpu_to_be16wu((uint16_t *)(data + sloc),
                      net_checksum_finish(sum));
    }
}

and above function is called at the following point.

hw/net/e1000.c
static void
xmit_seg(E1000State *s)
{
(snip)

    if (tp->sum_needed & E1000_TXD_POPTS_TXSM)
        putsum(tp->data, tp->size, tp->tucso, tp->tucss, tp->tucse);
        ~~~~~~ for udp and tcp checksum
    if (tp->sum_needed & E1000_TXD_POPTS_IXSM)
        putsum(tp->data, tp->size, tp->ipcso, tp->ipcss, tp->ipcse);
    if (tp->vlan_needed) {
        memmove(tp->vlan, tp->data, 4);
        memmove(tp->data, tp->data + 4, 8);
        memcpy(tp->data + 8, tp->vlan_header, 4);
        e1000_send_packet(s, tp->vlan, tp->size + 4);
    } else
        e1000_send_packet(s, tp->data, tp->size);
(snip)

Comment 4 jason wang 2017-11-17 06:49:55 UTC
Corner case.

Defer to 7.6.

I've queued the fix upstream.

Comment 5 Yuki Shibuya 2017-11-20 02:19:06 UTC
Thank you.

I understand that the fix will be included in RHEL 7.6.

Comment 6 Quan Wenli 2019-04-15 03:04:23 UTC
We will not add this test case since it's a corner case, will verify it as a bug fix.

Comment 13 Quan Wenli 2019-08-22 10:28:22 UTC
Reproduce it with qemu-kvm-1.5.3-167.el7.

Steps:
1. Create RHEL 7 VM.
2. Set e1000 NIC tyep to created VM.
    -device e1000,mac=9a:37:37:37:37:6e,id=idPK7OFU,netdev=id3XU8h2,bus=pci.0,addr=0x6  \
    -netdev tap,id=id3XU8h2 
3. confrim tx-checksumming offload on VM
[root@bootp-73-11-207 ~]# ethtool -k eth0
Features for eth0:
rx-checksumming: off
tx-checksumming: on
	tx-checksum-ipv4: off [fixed]
	tx-checksum-ip-generic: on
	tx-checksum-ipv6: off [fixed]
	tx-checksum-fcoe-crc: off [fixed]
	tx-checksum-sctp: off [fixed]
4. Send a udp packet that computed checksum is zero to destination server. using a.c reproduce in comment 0.
[root@bootp-73-11-207 ~]# gcc a.c
[root@bootp-73-11-207 ~]# ./a.c 10.73.8.2
5. Execute tcpdump to capture udp packets on destination server.
[root@dell-per730-06 ~]# tcpdump  -i eno1  udp -vv -X -c 1
tcpdump: listening on eno1, link-type EN10MB (Ethernet), capture size 262144 bytes
06:15:01.902199 IP (tos 0x0, ttl 64, id 33775, offset 0, flags [DF], proto UDP (17), length 32)
    bootp-73-11-207.lab.eng.pek2.redhat.com.50000 > dell-per730-06.lab.eng.pek2.redhat.com.italk: [udp sum ok] UDP, length 4
	0x0000:  4500 0020 83ef 4000 4011 8e7b 0a49 0bcf  E.....@.@..{.I..
	0x0010:  0a49 0802 c350 3039 000c e3e9 0000 0000  .I...P09........
	0x0020:  0000 0000 0000 0000 0000 0000 0000       ..............

getting checksum is e3e9, replace "msg = 0xe9e3" in a.c and re-build it. 
6. run again with steps 4 and step5.
[root@dell-per730-06 ~]# tcpdump  -i eno1  udp -vv -X -c 1
tcpdump: listening on eno1, link-type EN10MB (Ethernet), capture size 262144 bytes
06:15:01.902199 IP (tos 0x0, ttl 64, id 33775, offset 0, flags [DF], proto UDP (17), length 32)
    bootp-73-11-207.lab.eng.pek2.redhat.com.50000 > dell-per730-06.lab.eng.pek2.redhat.com.italk: [udp sum ok] UDP, length 4
	0x0000:  4500 0020 83ef 4000 4011 8e7b 0a49 0bcf  E.....@.@..{.I..
	0x0010:  0a49 0802 c350 3039 000c e3e9 0000 0000  .I...P09........
	0x0020:  0000 0000 0000 0000 0000 0000 0000       ..............



Verified it with qemu-kvm-1.5.3-168.el7
Same steps with 1-6, after running " tcpdump  -i eno1  udp -vv -X -c 1" on destination server, it has been change to "ffff e3e9" in tcpdump results.

root@dell-per730-06 ~]# tcpdump  -i eno1  udp -vv -X -c 1
tcpdump: listening on eno1, link-type EN10MB (Ethernet), capture size 262144 bytes
06:13:18.891520 IP (tos 0x0, ttl 64, id 33035, offset 0, flags [DF], proto UDP (17), length 32)
    bootp-73-11-207.lab.eng.pek2.redhat.com.50000 > dell-per730-06.lab.eng.pek2.redhat.com.italk: [udp sum ok] UDP, length 4
	0x0000:  4500 0020 810b 4000 4011 915f 0a49 0bcf  E.....@.@.._.I..
	0x0010:  0a49 0802 c350 3039 000c ffff e3e9 0000  .I...P09........
	0x0020:  0000 0000 0000 0000 0000 0000 0000       ..............


Base on above, set it to verified.

Comment 15 errata-xmlrpc 2020-03-31 20:02:00 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2020:1116

Comment 18 Ademar Reis 2020-05-11 17:37:30 UTC
Clearing needinfo which seems to have been added by accident.