RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.
Bug 1442477 - latest httpd update 2.4.6-45.el7_3.4 is breaking Spacewalk
Summary: latest httpd update 2.4.6-45.el7_3.4 is breaking Spacewalk
Keywords:
Status: CLOSED NOTABUG
Alias: None
Product: Red Hat Enterprise Linux 7
Classification: Red Hat
Component: httpd
Version: 7.3
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: rc
: ---
Assignee: Luboš Uhliarik
QA Contact: BaseOS QE - Apps
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2017-04-14 22:19 UTC by Leonid Kanter
Modified: 2017-05-09 15:00 UTC (History)
5 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2017-05-09 15:00:09 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Bugzilla 1445428 0 unspecified CLOSED httpd: HTTP Error 400 - Bad Request 2021-02-22 00:41:40 UTC

Internal Links: 1445428

Description Leonid Kanter 2017-04-14 22:19:20 UTC
Description of problem:

I'm running two instances of Spacewalk - 2.4 and 2.6. On both installations after the httpd update clients stopped to receive yum metadata and rpm packages with error 400. Rolling back this update fixed this problem immediately.

Version-Release number of selected component (if applicable):

2.4.6-45.el7_3.4

How reproducible: 

always

Steps to Reproduce:
1. update httpd on Spacewalk server
2. run yum update on spacewalk client system

Actual results:

On client you will have:

failed to retrieve repodata/repomd.xml from epel-x86_64-7
error was [Errno 14] HTTPS Error 400 - Bad Request

In Spacewalk server logs you will have:

192.168.1.78 - - [14/Apr/2017:17:53:11 +0000] "GET /XMLRPC//GET-REQ/centos-x86_64-7/repodata/repomd.xml HTTP/1.1" 400 226

Expected results:

spacewalk client should be able to get yum repodata and updates

Additional info:

Turnend on debug, I found following in httpd logs:

[Fri Apr 14 18:11:12.193177 2017] [core:debug] [pid 6486] protocol.c(956): (22)Invalid argument: [client 192.168.1.78:42834] Failed to read request header line X-RHN-Auth-User-Id:
[Fri Apr 14 18:11:12.193220 2017] [core:debug] [pid 6486] protocol.c(1309): [client 192.168.1.78:42834] AH00567: request failed: error reading the headers

Since the policy of header processing was changed in patch httpd-2.4.6-CVE-2016-8743.patch, I tried to recompile httpd with this patch commented out, and it fixed the problem.

Comment 2 Luboš Uhliarik 2017-04-18 15:47:51 UTC
Have you also updated yum-rhn-plugin ?

Comment 3 Luboš Uhliarik 2017-04-18 16:04:07 UTC
Leonid,

I just spoke to SW developers, and they told me, fix for this issue will be included in SW 2.7. 

So you have 2 options:

a) roll back httpd on your SW instance

b) apply the changes from fdf2063e14fc2539f09efcb7e72dd85060ab9ae5 yourself

Comment 4 Leonid Kanter 2017-04-18 18:40:08 UTC
Hello Luboš,

Yes, I verified that yum-rhn-plugin-2.0.1-6.1.el7_3.noarch works with new httpd, but it's unavailable for el6 and most of my clients are using yum-rhn-plugin-2.6.3-1. I found other solution: instead of rolling back httpd I sat an option


HttpProtocolOptions Unsafe

in /etc/httpd/conf.d/zz-spacewalk-server-wsgi.conf

It restored behavior of the header parser.

Comment 5 Grant Gainey 2017-04-26 13:35:44 UTC
So what's happening here is the result of an update to httpd, that enforces strict whitespace rules, that uncovered a bug in yum-rhn-plugin that was introduced in 2010.

(The spacewalk commit that started us down this path, for anyone who is interested, is 213b97e38dd75007281b37e3180dc68b736f2fac )

yum-rhn-plugin has been fixed, but a client without the fix yet won't be able to call home to *get* the fix, if the server has the new httpd.

There are two workarounds available:

One is to downgrade httpd

The other is to add an httpd config option that *turns off" strict-whitespace-enforcing until all your clients have a corrected yum-rhn-plugin. You can do that with the following:

# echo “HTTPProtocolOptions unsafe” >> /etc/httpd/conf.d/123unsafe.conf; systemctl restart httpd

That will put httpd back to its pre-2.4.25 behavior.

spacewalk-nightly deploys the config-option reliably for Fedora and RHEL6. We're still working on finding a way to deploy that config-change reliably on RHEL7. The issue is that RHEL7's httpd change is in the release, not the version, so <IfVersion> can't be used to 'protect' the change - and trying to use HttpProtocolOptions on an httpd that doesn't support it, causes httpd to fail and exit. Which is rude.

Comment 6 Grant Gainey 2017-04-26 13:36:34 UTC
Setting “HTTPProtocolOptions unsafe” in /etc/httpd/conf.d/zz-spacewalk-server-wsgi.conf also works :)

Comment 12 Joe Orton 2017-05-09 15:00:09 UTC
We'll be able to improve the workaround for future Spacewalk + RHEL7 httpd, but I'm closing this out since it's not a bug, Grant's commentary in comment 5 covers the options/workarounds.


Note You need to log in before you can comment on or make changes to this bug.