Bug 1750857

Summary: EPEL7 nginx package contains multiple CVEs
Product: [Fedora] Fedora EPEL Reporter: Andrew J. Caines <A.J.Caines+RedHat>
Component: nginxAssignee: Felix Kaechele <felix>
Status: CLOSED ERRATA QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: high Docs Contact:
Priority: unspecified    
Version: epel7CC: adam.winberg, affix, athmanem, bperkins, felix, jeremy, jkaluza, jorton, luhliari, orion, pavel.lisy, peter.borsa, rich.campbell, samuel.dejong, tadej.j, wtogami
Target Milestone: ---Keywords: Security, Upgrades
Target Release: ---   
Hardware: All   
OS: Linux   
Whiteboard:
Fixed In Version: nginx-1.16.1-1.el7 Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2019-10-19 16:52:28 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Andrew J. Caines 2019-09-10 15:49:09 UTC
EPEL7 currently includes nginx 1.12.2, which is vulnerable to multiple CVS including CVE-2019-9511, CVE-2019-9513, CVE-2019-9516 and which will not be patched upstream due to being EOL. Version 1.16.1 is packaged in current Fedora.

Version-Release number of selected component (if applicable):

nginx-1.12.2-3

How reproducible:

Always

Steps to Reproduce:

1. yum install nginx

Actual results:

nginx is version 1.12.2

Expected results:

nginx should be version 1.16.1

Comment 1 Warren Togami 2019-09-15 09:47:20 UTC
I haven't been involved here for many years but I noticed this ticket. As the old EPEL7 version is no longer patched upstream we may have no choice but to upgrade it to match Fedora. That is assuming nobody really has time to go through the effort of backporting the CVE patches and doing careful QA testing.

Question is if this is safe for the existing EPEL7 deployed nginx to upgrade into this as it's several years of changes they were not expecting? 

I have a production EPEL7 system running very basic nginx. Rebuilding Fedora 31's nginx-1.16.1-1 does work on EL7.

error: Failed dependencies:
	system-logos-httpd is needed by nginx-1:1.16.1-1.el7.x86_64

The equivalent to this is already provided in the centos-logos and centos-indexhtml. I'm proposing conditionals for the nginx.spec to use these automatically.

Sep 15 09:07:36 drone systemd[1]: Starting The nginx HTTP and reverse proxy server...
Sep 15 09:07:36 drone nginx[6695]: nginx: [emerg] module "/usr/lib64/nginx/modules/ngx_http_geoip_module.so" version 1012002 instead of 1016001 in /usr/share/n...eoip.conf:1
Sep 15 09:07:36 drone nginx[6695]: nginx: configuration file /etc/nginx/nginx.conf test failed
Sep 15 09:07:36 drone systemd[1]: nginx.service: control process exited, code=exited status=1
Sep 15 09:07:36 drone systemd[1]: Failed to start The nginx HTTP and reverse proxy server.
Sep 15 09:07:36 drone systemd[1]: Unit nginx.service entered failed state.
Sep 15 09:07:36 drone systemd[1]: nginx.service failed.

# rpm -qf /usr/lib64/nginx/modules/ngx_http_geoip_module.so
nginx-mod-http-geoip-1.12.2-3.el7.x86_64

I didn't install this explicitly on EPEL7, it was a dependency. nginx-1.16 RPM would need to Obsoletes: 1:1.12 that sub-package.

https://src.fedoraproject.org/rpms/nginx/blob/master/f/nginx.spec
	
* Mon Jul 30 2018 Luboš Uhliarik <luhliari> - 1:1.12.1-13
- don't build with geoip by default


https://src.fedoraproject.org/rpms/nginx/pull-request/6
Here's my proposed changes that would allow the EPEL7 nginx to be maintained in parallel with modern Fedora.

https://kojipkgs.fedoraproject.org/scratch/wtogami/task_37669661/
Here's a scratch build. I modified this scratch build's Release to '0' so this package will auto-upgrade into -1 when nginx-1:1.16.1 is pushed to EPEL7. I think scratch builds are auto-deleted in a week.

I tested this on my EPEL7 x86-64 machine with:

yum remove 'nginx*'
yum install nginx -y
systemctl restart nginx
# Check both default indexhtml http://IPADDRESS and custom https://DOMAINAME
yum update 'nginx*.rpm' -y
systemctl restart nginx
# Check both default indexhtml http://IPADDRESS and custom https://DOMAINAME

TESTING NEEDED!
Note: My /etc/nginx/nginx.conf is unmodified thus the new RPM replaced it. My only custom config is within /etc/nginx/conf.d/*
Other people who had modified their nginx.conf need to test this upgrade path.

Comment 2 Orion Poplawski 2019-09-18 17:54:09 UTC
So far so good here with light testing. Got a deprecation warning:

nginx: [warn] the "ssl" directive is deprecated, use the "listen ... ssl" directive instead in /etc/nginx/conf.d/forms-orion.conf:17

but it's just a warning.

Comment 3 Orion Poplawski 2019-09-19 15:16:22 UTC
Another option would be to use the spec from the nginx 1.14 stream in RHEL8 - https://git.centos.org/rpms/nginx/tree/c8-stream-1.14 which was recently updated to fix various CVEs

Comment 4 Warren Togami 2019-09-19 23:21:35 UTC
After posting the above I looked deeper into the history of EPEL7 nginx. We had upgraded the version several times before. Upgrading again is clearly the right plan.

The reality is the Fedora package is best maintained so if we want EPEL7 to be well maintained nginx.spec should be kept in sync with Fedora master branch. It was very easy for me to adapt the f32 nginx.spec to work properly on EPEL7.

Upgrading past the RHEL8 version is not a good argument against doing it as the RHEL8 version is outside of our control. Fedora/EPEL are within our control thus we should do what's best for it.

Thoughts?

Comment 5 Orion Poplawski 2019-09-20 02:14:48 UTC
I honestly don't have much of a horse in this race and would be fine with either, but:

- EPEL tends to value stability over latest features.  Presumably new versions of nginx may eventually remove support for current configurations.  If RedHat is maintaining 1.14 perhaps it's worth piggy backing on that work.

But I'm not going to be maintaining nginx, and will be happy to use the latest and greatest.

Comment 6 Warren Togami 2019-09-20 21:31:05 UTC
> EPEL tends to value stability over latest features.

Agreed. Although some upstream projects also value stability, which seems to be the case here.

Comment 7 Warren Togami 2019-09-23 18:42:49 UTC
https://fedorapeople.org/~wtogami/temp/2019/nginx-epel7-scratch/
Moved the koji scratch build here since the original will expire soon.

Comment 8 samuel.dejong 2019-10-15 05:21:15 UTC
Is this being worked on? Currently there are systems that are showing this vulnerability. Two options at this point. Either the NGINX in EPEL 7 gets updated as this thread seems to indicate or migrate to the official NGINX repo. Not sure about using the Fedora repo. This thread seems to indicate there are plans for the NGINX package to be updated in EPEL 7. Can someone confirm?

Comment 9 Fedora Update System 2019-10-15 16:23:07 UTC
FEDORA-EPEL-2019-6ca75fc39a has been submitted as an update to Fedora EPEL 7. https://bodhi.fedoraproject.org/updates/FEDORA-EPEL-2019-6ca75fc39a

Comment 10 samuel.dejong 2019-10-15 17:24:09 UTC
Thanks. What's the normal timeframe for this being available? Looks like 7 days since last ccmment. What's WFM mean?

Comment 11 Felix Kaechele 2019-10-15 17:44:51 UTC
WFM means "Works for me". The usual timeframe for EPEL packages to sit in testing is 14 days (from the time it lands in testing).
The above link even includes the information about when it is expected to land in stable: "days to stable: 4".

Comment 12 samuel.dejong 2019-10-15 17:46:41 UTC
Thanks. Sorry just a bit new as to the process. I saw that stable thing afterwards but still wasn't sure if it meant it would be available.

Comment 13 Orion Poplawski 2019-10-15 17:57:24 UTC
You can install it now by enabling the epel-testing repo:

yum --enablerepo=epel-testing upgrade nginx

Please report your experience on the update page.  If you do it with a registered account and give it positive feedback you can hasten its arrival in the normal epel repository.

Comment 14 samuel.dejong 2019-10-15 18:06:18 UTC
Good point. I can do that and then disable the repo as it should be moving to the stable repo soon.

Comment 15 Fedora Update System 2019-10-19 16:52:28 UTC
nginx-1.16.1-1.el7 has been pushed to the Fedora EPEL 7 stable repository. If problems still persist, please make note of it in this bug report.