Bug 1463674 - check_http segfaults once Location header terminates with additional 0x0a and is last header line
check_http segfaults once Location header terminates with additional 0x0a and...
Status: CLOSED ERRATA
Product: Fedora EPEL
Classification: Fedora
Component: nagios-plugins (Show other bugs)
el6
Unspecified Unspecified
unspecified Severity medium
: ---
: ---
Assigned To: Stephen John Smoogen
Fedora Extras Quality Assurance
:
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2017-06-21 09:02 EDT by Peter Bieringer
Modified: 2017-08-10 02:22 EDT (History)
8 users (show)

See Also:
Fixed In Version: nagios-plugins-2.2.1-3git.fc24 nagios-plugins-2.2.1-4git.fc26 nagios-plugins-2.2.1-4git.el7 nagios-plugins-2.2.1-3git.fc25 nagios-plugins-2.2.1-4git.el6
Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2017-07-23 17:51:20 EDT
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)

  None (edit)
Description Peter Bieringer 2017-06-21 09:02:04 EDT
Description of problem:
Segfault

Version-Release number of selected component (if applicable):
rpm -qf /usr/lib64/nagios/plugins/check_http
nagios-plugins-http-2.1.4-3.el6.x86_64


How reproducible:
always

Steps to Reproduce:
1. /usr/lib64/nagios/plugins/check_http -H <hostname>

Actual results:
Segmentation fault


Expected results:
Working


Additional info:

strace:
sendto(3, "GET / HTTP/1.1\r\nUser-Agent: check_http/v2.1.4 (nagios-plugins 2.1.4)\r\nConnection: close\r\nHost: *****\r\nAccept: */*\r\n\r\n", 155, 0, NULL, 0) = 155
read(3, "HTTP/1.1 301 Moved Permanently\r\nDate: Wed, 21 Jun 2017 12:46:13 GMT\r\nServer: Mbedthis-Appweb/2.4.2\r\nContent-type: application/x-appweb-php\r\nContent-length: 0\r\nConnection: close\r\nLocation: https://*****/designs/imm/index.php\n\r\n\r\n", 8191) = 266
read(3, "", 8191)                       = 0
close(3)                                = 0
--- SIGSEGV {si_signo=SIGSEGV, si_code=SEGV_MAPERR, si_addr=0x22ce000} ---
+++ killed by SIGSEGV +++
Segmentation fault


ltrace:
strstr("HTTP/1.1 301 Moved Permanently\r\nDate: Wed, 21 Jun 2017 12:45:30 GMT\r\nServer: Mbedthis-Appweb/2.4.2\r\nContent-type: application/x-appweb-php\r\nContent-length: 0\r\nConnection: close\r\nLocation: https://*****/designs/imm/index.php\n\r\n\r\n", "\r\n") = "\r\nDate: Wed, 21 Jun 2017 12:45:30 GMT\r\nServer: Mbedthis-Appweb/2.4.2\r\nContent-type: application/x-appweb-php\r\nContent-length: 0\r\nConnection: close\r\nLocation: https://****/designs/imm/index.php\n\r\n\r\n"
strlen("HTTP/1.1 301 Moved Permanently")                                                                                          = 30
--- SIGSEGV (Segmentation fault) ---
+++ killed by SIGSEGV +++


gdb backtrace:

Program terminated with signal 11, Segmentation fault.
#0  0x0000000000405cbf in check_http () at check_http.c:1205
1205      while (page[0] != '\r' || page[1] != '\n') {
(gdb) bt
#0  0x0000000000405cbf in check_http () at check_http.c:1205
#1  0x0000000000407bec in main (argc=3, argv=<value optimized out>) at check_http.c:180
Comment 1 Stephen John Smoogen 2017-06-22 13:42:35 EDT
Could you try the nagios-plugins which is in epel-testing? It seems to work correctly against the IBM IMM cards I have access to.

# /usr/lib64/nagios/plugins/check_http -H data-analysis01.mgmt.fedoraproject.org
HTTP OK: HTTP/1.1 200 OK - 229 bytes in 0.005 second response time |time=0.005466s;;;0.000000 size=229B;;;0
Comment 2 Peter Bieringer 2017-06-23 04:20:27 EDT
2.2.1 did not crash dump anymore but behaves still strange:

# /usr/lib64/nagios/plugins/check_http -H ***
GET / HTTP/1.1
User-Agent: check_http/v2.2.1 (nagios-plugins 2.2.1)
Connection: close
Host: ***
Accept: */*


http://***:80/ is 262 characters
STATUS: HTTP/1.1 301 Moved Permanently
CRITICAL - Socket timeout



Using telnet it works fine:

# telnet **** 80
Trying ***...
Connected to ***.
Escape character is '^]'.
GET / HTTP/1.1
User-Agent: check_http/v2.2.1 (nagios-plugins 2.2.1)
Connection: close
Host: ***
Accept: */*

HTTP/1.1 301 Moved Permanently
Date: Fri, 23 Jun 2017 07:58:33 GMT
Server: Mbedthis-Appweb/2.4.2
Content-type: application/x-appweb-php
Content-length: 0
Connection: close
Location: https://***/designs/imm/index.php


Connection closed by foreign host.



strace of plugin:

read(3, "HTTP/1.1 301 Moved Permanently\r\nDate: Fri, 23 Jun 2017 08:17:04 GMT\r\nServer: Mbedthis-Appweb/2.4.2\r\nContent-type: application/x-appweb-php\r\nContent-length: 0\r\nConnection: close\r\nLocation: https://***/designs/imm/index.php\n\r\n\r\n", 8191) = 262
read(3, "", 8191)                       = 0
close(3)                                = 0
write(1, "http://***:80/ is 262 characters\n", 69http://***:80/ is 262 characters
) = 69
write(1, "STATUS: HTTP/1.1 301 Moved Permanently\n", 39STATUS: HTTP/1.1 301 Moved Permanently
) = 39
--- SIGALRM {si_signo=SIGALRM, si_code=SI_KERNEL, si_value={int=0, ptr=0x100000000}} ---
write(1, "CRITICAL", 8CRITICAL)                 = 8
write(1, " - Socket timeout\n", 18 - Socket timeout
)     = 18
exit_group(2)                           = ?
+++ exited with 2 +++


ltrace:

read(4, "HTTP/1.1 301 Moved Permanently\r\nDate: Fri, 23 Jun 2017 08:18:10 GMT\r\nServer: Mbedthis-Appweb/2.4.2\r\nContent-type: application/x-appweb-php\r\nContent-length: 0\r\nConnection: close\r\nLocation: https://***/designs/imm/index.php\n\r\n\r\n", 8191) = 262
gettimeofday(0x7ffcca9d0160, NULL)                                                                                                                                     = 0
__vasprintf_chk(0x7ffcca9d0248, 1, 0x41183d, 0x7ffcca9d00b0, 0)                                                                                                        = 262
free(0x1a2d280)                                                                                                                                                        = <void>
read(4, "", 8191)                                                                                                                                                      = 0
gettimeofday(0x7ffcca9d0160, NULL)                                                                                                                                     = 0
close(4)                                                                                                                                                               = 0
gettimeofday(0x7ffcca9d0160, NULL)                                                                                                                                     = 0
__printf_chk(1, 0x413850, 0x4119bb, 0x1a2d0e0, 80http://***:80/ is 262 characters
)                                                                                                                     = 69
__strdup(0x1a2d600, 69, 0x1a2d61e, 0x41386e, 12)                                                                                                                       = 0x1a2d710
strlen("HTTP/1.1 301 Moved Permanently")                                                                                                                               = 30
__printf_chk(1, 0x411842, 0x1a2d710, 1, 0x796c746e656e61STATUS: HTTP/1.1 301 Moved Permanently
)                                                                                                              = 39
--- SIGALRM (Alarm clock) ---
write(1, "CRITICAL", 8CRITICAL)                                                                                                                                                = 8
write(1, " - Socket timeout\n", 18 - Socket timeout
)                                                                                                                                    = 18
exit(2 <unfinished ...>
+++ exited (status 2) +++


I have no clue for what the Nagios plugin is waiting...
Comment 3 Stephen John Smoogen 2017-06-23 17:13:23 EDT
I can't duplicate with my IMM's but that may just mean they are different version. What type of IMM and version are the ones having problems?
Comment 4 Peter Bieringer 2017-06-27 10:51:36 EDT
Monitoring host is btw. CentOS6

Happen on various IMM firmware and also seen on

Dell DRAC 1.92

working on Dell DRAC 1.65

here the difference:


Dell DRAC 1.92
STATUS: HTTP/1.1 301 Moved Permanently


Dell DRAC 1.65
STATUS: HTTP/1.1 302 Found


Looks like all "301" results are handled not proper

ltrace of "301":

strlen("HTTP/1.1 301 Moved Permanently")                                                                        = 30
__printf_chk(1, 0x411842, 0x11ad480, 1, 0x796c746e656e61STATUS: HTTP/1.1 301 Moved Permanently
)                           
(nothing happen anymore)


ltrace of "302":

strlen("HTTP/1.1 302 Found")                                                                                    = 18
__printf_chk(1, 0x411842, 0x1af9290, 1, 0x646e756f462032STATUS: HTTP/1.1 302 Found
)                                                       = 27
__ctype_b_loc()                                                                                                 = 0x7f61b5319770
__ctype_tolower_loc()                                                                                           = 0x7f61b5319780
...
Comment 5 Fedora Update System 2017-07-03 16:54:29 EDT
nagios-plugins-2.2.1-2git.el7 has been submitted as an update to Fedora EPEL 7. https://bodhi.fedoraproject.org/updates/FEDORA-EPEL-2017-cc0aeaca30
Comment 6 Peter Bieringer 2017-07-04 03:45:35 EDT
rebuild nagios-plugins-2.2.1-2git.el7 on EL6 behaves even worse:

2.2.0:

write(1, "GET / HTTP/1.1\r\n", 16GET / HTTP/1.1
)      = 16
write(1, "User-Agent: check_http/v2.2.0 (nagios-plugins 2.2.0)\r\n", 54User-Agent: check_http/v2.2.0 (nagios-plugins 2.2.0)
) = 54
write(1, "Connection: close\r\n", 19Connection: close
)   = 19
write(1, "Host: ***\r\n", 44Host: ***
) = 44
write(1, "Accept: */*\r\n", 13Accept: */*
)         = 13
write(1, "\r\n", 2
)                     = 2
write(1, "\n", 1
)                       = 1
read(3, "HTTP/1.1 301 Moved Permanently\r\nDate: Tue, 04 Jul 2017 07:35:12 GMT\r\nServer: Mbedthis-Appweb/2.4.2\r\nContent-type: text/html\r\nETag: \"2f4cfd8-3e1-0\"\r\nContent-length: 0\r\nConnection: close\r\nLocation: https://***/start.html\n\r\n\r\n", 8191) = 256
read(3, "", 8191)                       = 0
write(1, "http://***:80/ is 256 characters\n", 66http://***:80/ is 256 characters
) = 66
write(1, "STATUS: HTTP/1.1 301 Moved Permanently\n", 39STATUS: HTTP/1.1 301 Moved Permanently
) = 39
--- SIGALRM {si_signo=SIGALRM, si_code=SI_KERNEL, si_value={int=0, ptr=0x100000000}} ---
write(1, "CRITICAL", 8CRITICAL)                 = 8
write(1, " - Socket timeout", 17 - Socket timeout)       = 17
+++ exited with 2 +++



2.2.1-2git.el7

write(1, "GET / HTTP/1.1\r\n", 16GET / HTTP/1.1
)      = 16
write(1, "User-Agent: check_http/v2.2.1 (nagios-plugins 2.2.1)\r\n", 54User-Agent: check_http/v2.2.1 (nagios-plugins 2.2.1)
) = 54
write(1, "Connection: close\r\n", 19Connection: close
)   = 19
write(1, "Host: ***\r\n", 44Host: ***
) = 44
write(1, "Accept: */*\r\n", 13Accept: */*
)         = 13
write(1, "\n", 1
)                       = 1
read(3, 0x623e40, 8191)                 = ? ERESTARTSYS (To be restarted if SA_RESTART is set)
--- SIGALRM {si_signo=SIGALRM, si_code=SI_KERNEL, si_value={int=0, ptr=0x100000000}} ---
write(1, "CRITICAL", 8CRITICAL)                 = 8
write(1, " - Socket timeout\n", 18 - Socket timeout
)     = 18
+++ exited with 2 +++



=> 2.2.1 misses after "Accept

write(1, "\r\n", 2


Strange is that single

write(1, "\n", 1

on both checks, is this covered by standards?
Comment 7 Peter Bieringer 2017-07-04 04:07:46 EDT
found in code that the last \n is not used in request, instead final request looks like:

2.2.0

"GET / HTTP/1.1\r\nUser-Agent: check_http/v2.2.0 (nagios-plugins 2.2.0)\r\nConnection: close\r\nHost: ***\r\nAccept: */*\r\n\r\n"


2.2.1

"GET / HTTP/1.1\r\nUser-Agent: check_http/v2.2.1 (nagios-plugins 2.2.1)\r\nConnection: close\r\nHost: ****\r\nAccept: */*\r\n"


and in code I found:

 // xasprintf (&buf, "%s%s", buf, CRLF);                              // grg: do not append extra CRLF, for HTTP 1.1 compliance


=> looks like one has removed this adding of additional \r\n because of "HTTP 1.1 compliance" 

Is this really valid?


BTW: here is the raw output of response using netcat:


$ echo -e "GET / HTTP/1.1\r\nUser-Agent: check_http/v2.2.0 (nagios-plugins 2.2.0)\r\nConnection: close\r\nHost: ***\r\nAccept: */*\r\n\r\n" | nc *** 80
HTTP/1.1 301 Moved Permanently
Date: Tue, 04 Jul 2017 08:05:57 GMT
Server: Mbedthis-Appweb/2.4.2
Content-type: text/html
ETag: "2f4cfd8-3e1-0"
Content-length: 0
Connection: close
Location: https://***/start.html


$

(just note the 2 new lines after Location are really sent by server)
Comment 8 Stephen John Smoogen 2017-07-04 14:06:10 EDT
Thanks for the feedback. Was this with the version I put in epel-testing yesterday ,2.2.1-2git, or the earlier 2.2.1-1 version as they both had major changes to check_http that keep fixing other bugs. If it was with the latest, I am going to open an upstream bug with all the data from this bug in it.
Comment 9 Peter Bieringer 2017-07-04 14:08:37 EDT
I've downloaded the el7 srpms from koji build with the "2git" suffix.
Comment 10 Fedora Update System 2017-07-05 22:48:18 EDT
nagios-plugins-2.2.1-2git.el6 has been pushed to the Fedora EPEL 6 testing repository. If problems still persist, please make note of it in this bug report.
See https://fedoraproject.org/wiki/QA:Updates_Testing for
instructions on how to install test updates.
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-EPEL-2017-62fe0218d0
Comment 11 Fedora Update System 2017-07-05 22:49:36 EDT
nagios-plugins-2.2.1-2git.el7 has been pushed to the Fedora EPEL 7 testing repository. If problems still persist, please make note of it in this bug report.
See https://fedoraproject.org/wiki/QA:Updates_Testing for
instructions on how to install test updates.
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-EPEL-2017-cc0aeaca30
Comment 12 Peter Bieringer 2017-07-06 03:36:06 EDT
issue found on Lenovo IMM and Dell DRAC controllers

examples for problematic responses:


IMM 4.55

  0d 0a 4c 6f 63 61 74 69    6f 6e 3a 20 68 74 74 70    ..Location: http
...
  73 2f 69 6d 6d 2f 69 6e    64 65 78 2e 70 68 70 0a    s/imm/index.php.
  0d 0a 0d 0a        


DRAC 1.92

  6e 3a 20 63 6c 6f 73 65    0d 0a 4c 6f 63 61 74 69    n: close..Locati
...
  63 2f 73 74 61 72 74 2e    68 74 6d 6c 0a 0d 0a 0d    c/start.html....
  0a      

Each header terminates with sequence of \n\r\n\r\n (5 chars)

I don't know whether this is according to RFCs, but at least real life results

nagios-plugins-2.2.1-2git is not solving the issue, but switching from segfault to timeout because of uncatched "end-of-header".
Comment 13 Bryan Heden 2017-07-06 07:45:11 EDT
From upstream: Testing against the latest maint patches, I can not confirm the behavior personally. But I can say that were attempting to match up the plugin with strict HTTP checking. The commented xasprintf comes from section 4.1 of RFC 2616 [https://tools.ietf.org/html/rfc2616] - 

>
In the interest of robustness, servers SHOULD ignore any empty
   line(s) received where a Request-Line is expected. In other words, if
   the server is reading the protocol stream at the beginning of a
   message and receives a CRLF first, it should ignore the CRLF.

   Certain buggy HTTP/1.0 client implementations generate extra CRLF's
   after a POST request. To restate what is explicitly forbidden by the
   BNF, an HTTP/1.1 client MUST NOT preface or follow a request with an
   extra CRLF.

It looks, based on your output, that the order of the carriage returns and line feeds may be causing an issue. (Or simply the extra.)

Hopefully we can get this resolved today..

What happens if you re-allow the xasprintf? Does it stop segfaulting?
Comment 14 Peter Bieringer 2017-07-06 08:27:36 EDT
without having "extra" \n\r the server simply won't respond, still waiting for next request header line I would assume.

./check_http-2.2.1-2git.el6 -v -H  ***
GET / HTTP/1.1
User-Agent: check_http/v2.2.1 (nagios-plugins 2.2.1)
Connection: close
Host: ***
Accept: */*


ngrep:

T 10.13.16.50:59146 -> 10.13.14.38:80 [AP]
  47 45 54 20 2f 20 48 54    54 50 2f 31 2e 31 0d 0a    GET / HTTP/1.1..
  55 73 65 72 2d 41 67 65    6e 74 3a 20 63 68 65 63    User-Agent: chec
  6b 5f 68 74 74 70 2f 76    32 2e 32 2e 31 20 28 6e    k_http/v2.2.1 (n
  61 67 69 6f 73 2d 70 6c    75 67 69 6e 73 20 32 2e    agios-plugins 2.
  32 2e 31 29 0d 0a 43 6f    6e 6e 65 63 74 69 6f 6e    2.1)..Connection
  3a 20 63 6c 6f 73 65 0d    0a 48 6f 73 74 3a 20 **    : close..Host: *
  **************************************************    ****************
  **************************************************    ****************
  ******** 0d 0a 41 63 63    65 70 74 3a 20 2a 2f 2a    ***..Accept: */*
  0d 0a                                                 ..

so 2nd 0d 0a sequence is missing that server detects that header has ended.
Comment 15 Fedora Update System 2017-07-12 16:30:37 EDT
nagios-plugins-2.2.1-3git.el6 has been submitted as an update to Fedora EPEL 6. https://bodhi.fedoraproject.org/updates/FEDORA-EPEL-2017-76229ef8c9
Comment 16 Fedora Update System 2017-07-13 15:49:10 EDT
nagios-plugins-2.2.1-3git.el6 has been pushed to the Fedora EPEL 6 testing repository. If problems still persist, please make note of it in this bug report.
See https://fedoraproject.org/wiki/QA:Updates_Testing for
instructions on how to install test updates.
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-EPEL-2017-76229ef8c9
Comment 17 Fedora Update System 2017-07-13 15:51:18 EDT
nagios-plugins-2.2.1-3git.el7 has been pushed to the Fedora EPEL 7 testing repository. If problems still persist, please make note of it in this bug report.
See https://fedoraproject.org/wiki/QA:Updates_Testing for
instructions on how to install test updates.
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-EPEL-2017-4b1c55c024
Comment 18 Fedora Update System 2017-07-13 17:21:45 EDT
nagios-plugins-2.2.1-3git.fc24 has been pushed to the Fedora 24 testing repository. If problems still persist, please make note of it in this bug report.
See https://fedoraproject.org/wiki/QA:Updates_Testing for
instructions on how to install test updates.
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2017-6401b28fc4
Comment 19 Fedora Update System 2017-07-13 17:24:03 EDT
nagios-plugins-2.2.1-3git.fc25 has been pushed to the Fedora 25 testing repository. If problems still persist, please make note of it in this bug report.
See https://fedoraproject.org/wiki/QA:Updates_Testing for
instructions on how to install test updates.
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2017-8d031793bf
Comment 20 Fedora Update System 2017-07-13 19:54:13 EDT
nagios-plugins-2.2.1-3git.fc26 has been pushed to the Fedora 26 testing repository. If problems still persist, please make note of it in this bug report.
See https://fedoraproject.org/wiki/QA:Updates_Testing for
instructions on how to install test updates.
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2017-a5f81422dc
Comment 21 Fedora Update System 2017-07-14 14:58:32 EDT
nagios-plugins-2.2.1-4git.fc26 has been submitted as an update to Fedora 26. https://bodhi.fedoraproject.org/updates/FEDORA-2017-c2e82de3b3
Comment 22 Fedora Update System 2017-07-16 17:21:25 EDT
nagios-plugins-2.2.1-4git.fc26 has been pushed to the Fedora 26 testing repository. If problems still persist, please make note of it in this bug report.
See https://fedoraproject.org/wiki/QA:Updates_Testing for
instructions on how to install test updates.
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2017-c2e82de3b3
Comment 23 Fedora Update System 2017-07-23 00:18:15 EDT
nagios-plugins-2.2.1-4git.el6 has been pushed to the Fedora EPEL 6 testing repository. If problems still persist, please make note of it in this bug report.
See https://fedoraproject.org/wiki/QA:Updates_Testing for
instructions on how to install test updates.
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-EPEL-2017-8973027f42
Comment 24 Fedora Update System 2017-07-23 00:23:16 EDT
nagios-plugins-2.2.1-4git.el7 has been pushed to the Fedora EPEL 7 testing repository. If problems still persist, please make note of it in this bug report.
See https://fedoraproject.org/wiki/QA:Updates_Testing for
instructions on how to install test updates.
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-EPEL-2017-87ebfdc686
Comment 25 Fedora Update System 2017-07-23 17:51:20 EDT
nagios-plugins-2.2.1-3git.fc24 has been pushed to the Fedora 24 stable repository. If problems still persist, please make note of it in this bug report.
Comment 26 Fedora Update System 2017-08-03 11:52:58 EDT
nagios-plugins-2.2.1-4git.fc26 has been pushed to the Fedora 26 stable repository. If problems still persist, please make note of it in this bug report.
Comment 27 Fedora Update System 2017-08-09 11:22:59 EDT
nagios-plugins-2.2.1-4git.el7 has been pushed to the Fedora EPEL 7 stable repository. If problems still persist, please make note of it in this bug report.
Comment 28 Fedora Update System 2017-08-09 15:57:13 EDT
nagios-plugins-2.2.1-3git.fc25 has been pushed to the Fedora 25 stable repository. If problems still persist, please make note of it in this bug report.
Comment 29 Fedora Update System 2017-08-10 02:19:43 EDT
nagios-plugins-2.2.1-4git.el6 has been pushed to the Fedora EPEL 6 stable repository. If problems still persist, please make note of it in this bug report.
Comment 30 Peter Bieringer 2017-08-10 02:22:24 EDT
be warned, there is next unexpected issue in check_http (content check is broken now)
https://bugzilla.redhat.com/show_bug.cgi?id=1480085

Note You need to log in before you can comment on or make changes to this bug.