Bug 1480085

Summary: nagios-plugins-http 2.2.1-4git breaks http result content check
Product: [Fedora] Fedora EPEL Reporter: Peter Bieringer <pb>
Component: nagios-pluginsAssignee: Stephen John Smoogen <smooge>
Status: CLOSED ERRATA QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: high Docs Contact:
Priority: unspecified    
Version: epel7CC: adam.wilbraham, athmanem, b.heden, info, kmf, markus.frosch, ondrejj, paul, public, redhat, smooge, smooge, svene, swilkerson
Target Milestone: ---Keywords: Reopened
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: nagios-plugins-2.2.1-9git5c7eb5b9.el7 nagios-plugins-2.2.1-15.20180725git3429dad.el6 Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2019-04-06 20:23:02 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:

Description Peter Bieringer 2017-08-10 06:18:52 UTC
Description of problem:
After upgrade from 2.1.4 to 2.2.1-4git the http content check is broken

Version-Release number of selected component (if applicable):
nagios-plugins-http-2.2.1-4git.el7.x86_64

How reproducible:
always

Steps to Reproduce:
1. run check
/tmp/check_http-2.2.1 -H ipv4.bieringer.de -s ipv4.bieringer.de




Actual results:
 /tmp/check_http-2.2.1 -H ipv4.bieringer.de -s ipv4.bieringer.de
HTTP CRITICAL: HTTP/1.1 200 OK - string 'ipv4.bieringer.de' not found on 'http://ipv4.bieringer.de:80/' - 26089 bytes in 0,066 second response time |time=0,066241s;;;0,000000 size=26089B;;;0

no wonder, if used with -v:

...
<body>

<center>

<h1>
19ipv4.bieringer
HTTP OK: HTTP/1.1 200 OK - 26081 bytes in 0,065 second response time |time=0,065444s;;;0,000000 size=26081B;;;0

(oops: strange "19ipv4.bieringer")


Expected results:

/tmp/check_http-2.1.4 -H ipv4.bieringer.de -s ipv4.bieringer.de
HTTP OK: HTTP/1.1 200 OK - 26089 bytes in 0,067 second response time |time=0,067280s;;;0,000000 size=26089B;;;0

with -v it is seen that the whole HTML page is returned


Additional info:

as requested already: check_http (and potential others) need urgently a proper test framework during build


Because of the 5+ issues found with check_http in the short past during minor upgrades imho a major rewrite is necessary...

Comment 1 Sven 2017-08-15 11:47:26 UTC
I'm seeing a similar issue with string/regex matching in 2.2.1-4git, where it can't find a string after the first 1K or so of headers + content. 

In the example above, "<h1>ipv4.bieringer.de</h1>" is also just after the 1K mark.

Comment 2 Dan Pritts 2017-08-22 22:33:49 UTC
I just manually compiled 2.2.1 on RHEL6 and it doesn't have the problem. 

There are a lot of changes to check_http.c in the 4git patch file but it's not obvious which one is the culprit.

Comment 3 Adam Wilbraham 2017-08-30 11:52:19 UTC
I've also encountered issues with this version of the plugin, specifically using the SSL expiry check functionality.

From "check_http --help":

"Examples:
 CHECK CONTENT: check_http -w 5 -c 10 --ssl -H www.verisign.com

 When the 'www.verisign.com' server returns its content within 5 seconds,
 a STATE_OK will be returned. When the server returns its content but exceeds
 the 5-second threshold, a STATE_WARNING will be returned. When an error occurs,
 a STATE_CRITICAL will be returned.

 CHECK CERTIFICATE: check_http -H www.verisign.com -C 14

 When the certificate of 'www.verisign.com' is valid for more than 14 days,
 a STATE_OK is returned. When the certificate is still valid, but for less than
 14 days, a STATE_WARNING is returned. A STATE_CRITICAL will be returned when
 the certificate is expired."

This no longer works if the response not a 2x or 3x as the plugin is now also doing response code verification, eg:

 /usr/lib64/nagios/plugins/check_http -H api.google.com -C30
SSL OK - Certificate 'www.google.com' will expire on 2017-11-14 16:29 +0000/GMT. HTTP WARNING: HTTP/1.1 404 Not Found - 1786 bytes in 0.127 second response time |time=0.126712s;;;0.000000 size=1786B;;;0


Previously it would just respond like:

"OK - Certificate 'www.google.com' will expire on Tue 14 Nov 2017 16:29:00 GMT."

...unless you specifically passed in additional parameters to check_http.

Comment 4 Florian Paul Hoberg 2017-09-18 13:46:26 UTC
Unfortunately we ran into the same behaviour like Adam after upgrading from "2.1.4" to "2.2.1-4git" that SSL certificate only checks without any other options will fail when the http response code is not 2x/3x. On the one hand it sounds legit for a http check, on the other hand such a (silent) change may have a bigger impact for some people.

Comment 5 hansmi 2017-10-10 11:30:50 UTC
I can reliably reproduce memory corruption in check_http when talking over TLS to an HTTP server which returns nothing but "ok" (an OpenShift API server). There have been a couple seemingly related fixes on the upstream repository (https://github.com/nagios-plugins/nagios-plugins/commits/maint/plugins/check_http.c). While I can't publicise the hostname I can give Valgrind and GDB outputs.

---
$ gdb --args /usr/lib64/nagios/plugins/check_http -v -H HOSTNAME -I HOSTNAME -S -p 443 -u /healthz -R '^ok$'
…
Starting program: /usr/lib64/nagios/plugins/check_http -v -H HOSTNAME -I HOSTNAME -S -p 443 -u /healthz -R \^ok\$
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib64/libthread_db.so.1".
SSL initialized
GET /healthz HTTP/1.1
User-Agent: check_http/v2.2.1 (nagios-plugins 2.2.1)
Connection: close
Host: HOSTNAME
Accept: */*


https://HOSTNAME:443/healthz is 162 characters
STATUS: HTTP/1.1 200 OK

Breakpoint 1, check_http () at check_http.c:1235
1235        printf ("**** HEADER ****\n%s\n**** CONTENT ****\n%s\n", header,
(gdb) p page
$2 = 0x5555557b6c50 "ok\352\366\377\177"
---

---
$ valgrind --track-origins=yes /usr/lib64/nagios/plugins/check_http -v -H HOSTNAME -I HOSTNAME -S -p 443 -u /healthz
…
GET /healthz HTTP/1.1
User-Agent: check_http/v2.2.1 (nagios-plugins 2.2.1)
Connection: close
Host: HOSTNAME
Accept: */*


https://HOSTNAME:443/healthz is 162 characters
STATUS: HTTP/1.1 200 OK
**** HEADER ****
Cache-Control: no-store
Date: Tue, 10 Oct 2017 10:55:27 GMT
Content-Length: 2
Content-Type: text/plain; charset=utf-8
Connection: close
**** CONTENT ****
==41432== Conditional jump or move depends on uninitialised value(s)
==41432==    at 0x5DA46E4: vfprintf (vfprintf.c:1635)
==41432==    by 0x5E69767: __printf_chk (printf_chk.c:36)
==41432==    by 0x10CB72: UnknownInlinedFun (stdio2.h:104)
==41432==    by 0x10CB72: check_http (check_http.c:1235)
==41432==    by 0x10B733: main (check_http.c:183)
==41432==  Uninitialised value was created by a heap allocation
==41432==    at 0x4C2BB78: realloc (vg_replace_malloc.c:785)
==41432==    by 0x10C775: check_http (check_http.c:1146)
==41432==    by 0x10B733: main (check_http.c:183)
==41432==
ok
HTTP OK: HTTP/1.1 200 OK - 162 bytes in 1.615 second response time |time=1.615004s;;;0.000000 size=162B;;;0

==41432==
==41432== HEAP SUMMARY:
==41432==     in use at exit: 103,924 bytes in 3,464 blocks
==41432==   total heap usage: 5,987 allocs, 2,523 frees, 424,412 bytes allocated
==41432==
==41432== LEAK SUMMARY:
==41432==    definitely lost: 733 bytes in 15 blocks
==41432==    indirectly lost: 0 bytes in 0 blocks
==41432==      possibly lost: 0 bytes in 0 blocks
==41432==    still reachable: 103,191 bytes in 3,449 blocks
==41432==         suppressed: 0 bytes in 0 blocks
==41432== Rerun with --leak-check=full to see details of leaked memory
==41432==
==41432== For counts of detected and suppressed errors, rerun with: -v
==41432== ERROR SUMMARY: 1 errors from 1 contexts (suppressed: 0 from 0)
---

---
$ curl --trace tr https://HOSTNAME/healthz
# cat tr
== Info: About to connect() to HOSTNAME port 443 (#0)
…
=> Send header, 102 bytes (0x66)
0000: 47 45 54 20 2f 68 65 61 6c 74 68 7a 20 48 54 54 GET /healthz HTT
0010: 50 2f 31 2e 31 0d 0a 55 73 65 72 2d 41 67 65 6e P/1.1..User-Agen
0020: 74 3a 20 63 75 72 6c 2f 37 2e 32 39 2e 30 0d 0a t: curl/7.29.0..
0030: 48 6f 73 74 3a 20 72 61 73 63 68 2d 6f 73 2d 63 Host: HOSTNAMEHO
0040: 6f 6e 73 6f 6c 65 2e 61 6d 61 7a 65 65 69 6f 2e STNAMEHOSTNAMEH.
0050: 63 6c 6f 75 64 0d 0a 41 63 63 65 70 74 3a 20 2a cloud..Accept: *
0060: 2f 2a 0d 0a 0d 0a                               /*....
<= Recv header, 17 bytes (0x11)
0000: 48 54 54 50 2f 31 2e 31 20 32 30 30 20 4f 4b 0d HTTP/1.1 200 OK.
0010: 0a                                              .
<= Recv header, 25 bytes (0x19)
0000: 43 61 63 68 65 2d 43 6f 6e 74 72 6f 6c 3a 20 6e Cache-Control: n
0010: 6f 2d 73 74 6f 72 65 0d 0a                      o-store..
<= Recv header, 37 bytes (0x25)
0000: 44 61 74 65 3a 20 54 75 65 2c 20 31 30 20 4f 63 Date: Tue, 10 Oc
0010: 74 20 32 30 31 37 20 31 31 3a 32 38 3a 35 37 20 t 2017 11:28:57 
0020: 47 4d 54 0d 0a                                  GMT..
<= Recv header, 19 bytes (0x13)
0000: 43 6f 6e 74 65 6e 74 2d 4c 65 6e 67 74 68 3a 20 Content-Length: 
0010: 32 0d 0a                                        2..
<= Recv header, 41 bytes (0x29)
0000: 43 6f 6e 74 65 6e 74 2d 54 79 70 65 3a 20 74 65 Content-Type: te
0010: 78 74 2f 70 6c 61 69 6e 3b 20 63 68 61 72 73 65 xt/plain; charse
0020: 74 3d 75 74 66 2d 38 0d 0a                      t=utf-8..
<= Recv header, 2 bytes (0x2)
0000: 0d 0a                                           ..
<= Recv data, 2 bytes (0x2)
0000: 6f 6b                                           ok
== Info: Connection #0 to host HOSTNAME left intact
---

Comment 6 Fedora Update System 2017-11-21 17:54:46 UTC
nagios-plugins-2.2.1-9git5c7eb5b9.el7 has been submitted as an update to Fedora EPEL 7. https://bodhi.fedoraproject.org/updates/FEDORA-EPEL-2017-95028b0cf2

Comment 7 Fedora Update System 2017-11-21 18:13:02 UTC
nagios-plugins-2.2.1-9.el6 has been submitted as an update to Fedora EPEL 6. https://bodhi.fedoraproject.org/updates/FEDORA-EPEL-2017-e0c04ced36

Comment 8 Fedora Update System 2017-11-21 18:29:24 UTC
nagios-plugins-2.2.1-9git5c7eb5b9.el6 has been submitted as an update to Fedora EPEL 6. https://bodhi.fedoraproject.org/updates/FEDORA-EPEL-2017-5c6c3ab0a8

Comment 9 Fedora Update System 2017-11-21 19:33:27 UTC
nagios-plugins-2.2.1-9git5c7eb5b9.el7 has been pushed to the Fedora EPEL 7 testing repository. If problems still persist, please make note of it in this bug report.
See https://fedoraproject.org/wiki/QA:Updates_Testing for
instructions on how to install test updates.
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-EPEL-2017-95028b0cf2

Comment 10 Fedora Update System 2017-11-25 05:22:47 UTC
nagios-plugins-2.2.1-9git5c7eb5b9.el6 has been pushed to the Fedora EPEL 6 testing repository. If problems still persist, please make note of it in this bug report.
See https://fedoraproject.org/wiki/QA:Updates_Testing for
instructions on how to install test updates.
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-EPEL-2017-5c6c3ab0a8

Comment 11 Fedora Update System 2018-04-22 00:41:36 UTC
nagios-plugins-2.2.1-9git5c7eb5b9.el7 has been pushed to the Fedora EPEL 7 stable repository. If problems still persist, please make note of it in this bug report.

Comment 12 Peter Bieringer 2018-04-22 09:12:05 UTC
Sorry, did oversee that there was a version in testing...but this particular version won't fix the issue:

# ./check_http-2.2.1-9git5c7eb5b9.el7 -H ipv4.bieringer.de -s ipv4.bieringer.de -v
GET / HTTP/1.1
User-Agent: check_http/v2.2.1.git (nagios-plugins 2.2.1)
Connection: close
Host: ipv4.bieringer.de
Accept: */*


http://ipv4.bieringer.de:80/ is 26089 characters
STATUS: HTTP/1.1 200 OK
**** HEADER ****
Date: Sun, 22 Apr 2018 09:08:27 GMT
Server: Apache/2
Vary: Accept-Encoding,User-Agent
Connection: close
Transfer-Encoding: chunked
Content-Type: text/html; charset=UTF-8
**** CONTENT ****
<!doctype html public "-//w3c//dtd html 4.0 transitional//en">
<html>
<head>
   <meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1">
   <meta name="AUTHOR" content="Dr. Peter Bieringer">
   <meta name="KeyWords" content="test, ip, ipv4, ipv6, geo, address, ipv4 address, ipv6 address, ip address">
   <meta name="description" content="IP Address Information Test">
   <link href="/favicon.ico" rel="SHORTCUT ICON">
   <title>ip.bieringer.de - IP Address Information Test Page</title>
</head>
<body>

<center>

<h1>
19ipv4.bieringer
HTTP CRITICAL: HTTP/1.1 200 OK - string 'ipv4.bieringer.de' not found on 'http://ipv4.bieringer.de:80/' - 26089 bytes in 0,086 second response time |time=0,085571s;;;0,000000 size=26089B;;;0

Comment 13 Fedora Update System 2018-07-25 22:24:37 UTC
nagios-plugins-2.2.1-14.20180725git3429dad.el6 has been submitted as an update to Fedora EPEL 6. https://bodhi.fedoraproject.org/updates/FEDORA-EPEL-2018-c191412de6

Comment 14 Fedora Update System 2018-07-26 14:52:05 UTC
nagios-plugins-2.2.1-14.20180725git3429dad.el6 has been pushed to the Fedora EPEL 6 testing repository. If problems still persist, please make note of it in this bug report.
See https://fedoraproject.org/wiki/QA:Updates_Testing for
instructions on how to install test updates.
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-EPEL-2018-c191412de6

Comment 15 info 2018-08-30 16:28:40 UTC
Problem experienced by us (probably the same Peter Bieringer had) related to errnous processed chunks beginning with linebreaks is solved in nagios-plugins-2.2.1-14.20180725git3429dad.el6.
it would be nice if this version makes it into epel for both el6 and el7 asap.

Comment 16 Peter Bieringer 2018-12-12 08:39:56 UTC
nagios-plugins-http-2.2.1-14.20180725git3429dad.el7.x86_64 is still somehow broken regarding the result message

/usr/lib64/nagios/plugins/check_http -S -H $host -w 5 -c 10 -p '443' -u '$uri' -e 'HTTP/1.1 200' -v
SSL initialized
GET $uri HTTP/1.1
User-Agent: check_http/v2.2.1.git (nagios-plugins 2.2.1)
Connection: close
Host: $host
Accept: */*


https://$host:443/$uri is 612 characters
STATUS: HTTP/1.1 503 Service Temporarily Unavailable
**** HEADER ****
Date: Wed, 12 Dec 2018 08:36:20 GMT
Content-Length: 442
Connection: close
Content-Type: text/html; charset=iso-8859-1
**** CONTENT ****
<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
<html><head>
<title>503 Service Temporarily Unavailable</title>
</head><body>
<h1>Service Temporarily Unavailable</h1>
<p>The server is temporarily unable to service your
request due to maintenance downtime or capacity
problems. Please try again later.</p>
<p>Additionally, a 403 Forbidden
error was encountered while trying to use an ErrorDocument to handle the request.</p>
</body></html>

Status line output matched "HTTP/1.1 200" -
HTTP CRITICAL - Status line output matched "HTTP/1.1 200" - [...]

=> HTTP CRITICAL - Status line output matched "HTTP/1.1 200" - 

Result CRITICAL is ok, but the output is wrong, the status line is not matching, also the final line break is missing + some others


If I change to: -e 'HTTP/1.1 503' -v

last lines are:

Status line output matched "HTTP/1.1 503" -
HTTP OK: Status line output matched "HTTP/1.1 503" - 612 bytes in 0.081 second response time |time=0.080803s;5.000000;10.000000;0.000000 size=612B;;;0

Comment 17 Fedora Update System 2019-03-12 13:41:42 UTC
nagios-plugins-2.2.1-15.20180725git3429dad.el6 has been submitted as an update to Fedora EPEL 6. https://bodhi.fedoraproject.org/updates/FEDORA-EPEL-2019-7bf33722b6

Comment 18 Fedora Update System 2019-03-12 23:01:58 UTC
nagios-plugins-2.2.1-15.20180725git3429dad.el6 has been pushed to the Fedora EPEL 6 testing repository. If problems still persist, please make note of it in this bug report.
See https://fedoraproject.org/wiki/QA:Updates_Testing for
instructions on how to install test updates.
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-EPEL-2019-7bf33722b6

Comment 19 Fedora Update System 2019-04-06 20:23:02 UTC
nagios-plugins-2.2.1-15.20180725git3429dad.el6 has been pushed to the Fedora EPEL 6 stable repository. If problems still persist, please make note of it in this bug report.