Description of problem: Repeatedly errors: client_io->complete_request() returned Input/output error Feb 20 16:34:59 overcloud-controller-0 journal: 2020-02-20 16:34:59.347929 7fafd55a3700 1 ====== starting new request req=0x7fafd559cf70 ===== Feb 20 16:34:59 overcloud-controller-0 journal: 2020-02-20 16:34:59.349152 7fafd55a3700 0 ERROR: client_io->complete_request() returned Input/output error Feb 20 16:34:59 overcloud-controller-0 docker: 2020-02-20 16:34:59.347929 7fafd55a3700 1 ====== starting new request req=0x7fafd559cf70 ===== Feb 20 16:34:59 overcloud-controller-0 docker: 2020-02-20 16:34:59.349152 7fafd55a3700 0 ERROR: client_io->complete_request() returned Input/output error Feb 20 16:34:59 overcloud-controller-0 docker: 2020-02-20 16:34:59.349173 7fafd55a3700 1 ====== req done req=0x7fafd559cf70 op status=0 http_status=200 ====== Feb 20 16:34:59 overcloud-controller-0 docker: 2020-02-20 16:34:59.349209 7fafd55a3700 1 civetweb: 0x55d75524e000: 172.18.0.11 - - [20/Feb/2020:16:34:59 +0100] "HEAD / HTTP/1.0" 200 182 - - Feb 20 16:34:59 overcloud-controller-0 journal: 2020-02-20 16:34:59.349173 7fafd55a3700 1 ====== req done req=0x7fafd559cf70 op status=0 http_status=200 ====== Feb 20 16:34:59 overcloud-controller-0 journal: 2020-02-20 16:34:59.349209 7fafd55a3700 1 civetweb: 0x55d75524e000: 172.18.0.11 - - [20/Feb/2020:16:34:59 +0100] "HEAD / HTTP/1.0" 200 182 - - Feb 20 16:34:59 overcloud-controller-0 ovs-vswitchd: ovs|347528|netdev_tc_offloads(revalidator603)|ERR|Dropped 17 log messages in last 1 seconds (most recently, 1 seconds ago) due to excessive rate Feb 20 16:34:59 overcloud-controller-0 ovs-vswitchd: ovs|347529|netdev_tc_offloads(revalidator603)|ERR|dump_create: failed to get ifindex for tapda1682c1-76: Operation not supported Feb 20 16:35:00 overcloud-controller-0 su: (to rabbitmq) root on none Version-Release number of selected component (if applicable): How reproducible: This environment Steps to Reproduce: 1. Deploy overcloud with ceph_rgw 2. 3. Actual results: Error logs like above Expected results: No error logs Additional info: ~~~ if $ceph_rgw { ::tripleo::haproxy::endpoint { 'ceph_rgw': public_virtual_ip => $public_virtual_ip, internal_ip => hiera('ceph_rgw_vip', $controller_virtual_ip), service_port => $ports[ceph_rgw_port], ip_addresses => hiera('ceph_rgw_node_ips', $controller_hosts_real), server_names => hiera('ceph_rgw_node_names', $controller_hosts_names_real), public_ssl_port => $ports[ceph_rgw_ssl_port], service_network => $ceph_rgw_network, listen_options => merge($default_listen_options, { 'option' => [ 'httpchk HEAD /' ] }), <================================ } } ~~~ in puppet-tripleo/manifests/haproxy.pp That would be caused by this commit: commit dc52f322eaa1bf3048408e5719f77e456a357ef6 Author: Alex Schultz <aschultz> Date: Tue Feb 28 16:46:47 2017 +0000 Add httpchk for http services The httpchk health check option should help reduce the situtations where haproxy thinks the service is up but the service is only listening and not actively serving http requests. Change-Id: I13cc5dcf2eea53731e756d078586ab9a97340912 Closes-Bug: #1629052 which was around since 2017. So the problem is not on the openstack configuration side of things and would probably be a haproxy issue OR a ceph-rgw issue while handling HEAD / requests.
Hi, whats the status in of this BZ? Thanks Sasan
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory (Red Hat OpenStack Platform 13.0 director bug fix advisory), and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2020:5575