Bug 1463674

Could you try the nagios-plugins which is in epel-testing? It seems to work correctly against the IBM IMM cards I have access to.

# /usr/lib64/nagios/plugins/check_http -H data-analysis01.mgmt.fedoraproject.org
HTTP OK: HTTP/1.1 200 OK - 229 bytes in 0.005 second response time |time=0.005466s;;;0.000000 size=229B;;;0

2.2.1 did not crash dump anymore but behaves still strange:

# /usr/lib64/nagios/plugins/check_http -H ***
GET / HTTP/1.1
User-Agent: check_http/v2.2.1 (nagios-plugins 2.2.1)
Connection: close
Host: ***
Accept: */*


http://***:80/ is 262 characters
STATUS: HTTP/1.1 301 Moved Permanently
CRITICAL - Socket timeout



Using telnet it works fine:

# telnet **** 80
Trying ***...
Connected to ***.
Escape character is '^]'.
GET / HTTP/1.1
User-Agent: check_http/v2.2.1 (nagios-plugins 2.2.1)
Connection: close
Host: ***
Accept: */*

HTTP/1.1 301 Moved Permanently
Date: Fri, 23 Jun 2017 07:58:33 GMT
Server: Mbedthis-Appweb/2.4.2
Content-type: application/x-appweb-php
Content-length: 0
Connection: close
Location: https://***/designs/imm/index.php


Connection closed by foreign host.



strace of plugin:

read(3, "HTTP/1.1 301 Moved Permanently\r\nDate: Fri, 23 Jun 2017 08:17:04 GMT\r\nServer: Mbedthis-Appweb/2.4.2\r\nContent-type: application/x-appweb-php\r\nContent-length: 0\r\nConnection: close\r\nLocation: https://***/designs/imm/index.php\n\r\n\r\n", 8191) = 262
read(3, "", 8191)                       = 0
close(3)                                = 0
write(1, "http://***:80/ is 262 characters\n", 69http://***:80/ is 262 characters
) = 69
write(1, "STATUS: HTTP/1.1 301 Moved Permanently\n", 39STATUS: HTTP/1.1 301 Moved Permanently
) = 39
--- SIGALRM {si_signo=SIGALRM, si_code=SI_KERNEL, si_value={int=0, ptr=0x100000000}} ---
write(1, "CRITICAL", 8CRITICAL)                 = 8
write(1, " - Socket timeout\n", 18 - Socket timeout
)     = 18
exit_group(2)                           = ?
+++ exited with 2 +++


ltrace:

read(4, "HTTP/1.1 301 Moved Permanently\r\nDate: Fri, 23 Jun 2017 08:18:10 GMT\r\nServer: Mbedthis-Appweb/2.4.2\r\nContent-type: application/x-appweb-php\r\nContent-length: 0\r\nConnection: close\r\nLocation: https://***/designs/imm/index.php\n\r\n\r\n", 8191) = 262
gettimeofday(0x7ffcca9d0160, NULL)                                                                                                                                     = 0
__vasprintf_chk(0x7ffcca9d0248, 1, 0x41183d, 0x7ffcca9d00b0, 0)                                                                                                        = 262
free(0x1a2d280)                                                                                                                                                        = <void>
read(4, "", 8191)                                                                                                                                                      = 0
gettimeofday(0x7ffcca9d0160, NULL)                                                                                                                                     = 0
close(4)                                                                                                                                                               = 0
gettimeofday(0x7ffcca9d0160, NULL)                                                                                                                                     = 0
__printf_chk(1, 0x413850, 0x4119bb, 0x1a2d0e0, 80http://***:80/ is 262 characters
)                                                                                                                     = 69
__strdup(0x1a2d600, 69, 0x1a2d61e, 0x41386e, 12)                                                                                                                       = 0x1a2d710
strlen("HTTP/1.1 301 Moved Permanently")                                                                                                                               = 30
__printf_chk(1, 0x411842, 0x1a2d710, 1, 0x796c746e656e61STATUS: HTTP/1.1 301 Moved Permanently
)                                                                                                              = 39
--- SIGALRM (Alarm clock) ---
write(1, "CRITICAL", 8CRITICAL)                                                                                                                                                = 8
write(1, " - Socket timeout\n", 18 - Socket timeout
)                                                                                                                                    = 18
exit(2 <unfinished ...>
+++ exited (status 2) +++


I have no clue for what the Nagios plugin is waiting...

I can't duplicate with my IMM's but that may just mean they are different version. What type of IMM and version are the ones having problems?

Monitoring host is btw. CentOS6

Happen on various IMM firmware and also seen on

Dell DRAC 1.92

working on Dell DRAC 1.65

here the difference:


Dell DRAC 1.92
STATUS: HTTP/1.1 301 Moved Permanently


Dell DRAC 1.65
STATUS: HTTP/1.1 302 Found


Looks like all "301" results are handled not proper

ltrace of "301":

strlen("HTTP/1.1 301 Moved Permanently")                                                                        = 30
__printf_chk(1, 0x411842, 0x11ad480, 1, 0x796c746e656e61STATUS: HTTP/1.1 301 Moved Permanently
)                           
(nothing happen anymore)


ltrace of "302":

strlen("HTTP/1.1 302 Found")                                                                                    = 18
__printf_chk(1, 0x411842, 0x1af9290, 1, 0x646e756f462032STATUS: HTTP/1.1 302 Found
)                                                       = 27
__ctype_b_loc()                                                                                                 = 0x7f61b5319770
__ctype_tolower_loc()                                                                                           = 0x7f61b5319780
...

nagios-plugins-2.2.1-2git.el7 has been submitted as an update to Fedora EPEL 7. https://bodhi.fedoraproject.org/updates/FEDORA-EPEL-2017-cc0aeaca30

rebuild nagios-plugins-2.2.1-2git.el7 on EL6 behaves even worse:

2.2.0:

write(1, "GET / HTTP/1.1\r\n", 16GET / HTTP/1.1
)      = 16
write(1, "User-Agent: check_http/v2.2.0 (nagios-plugins 2.2.0)\r\n", 54User-Agent: check_http/v2.2.0 (nagios-plugins 2.2.0)
) = 54
write(1, "Connection: close\r\n", 19Connection: close
)   = 19
write(1, "Host: ***\r\n", 44Host: ***
) = 44
write(1, "Accept: */*\r\n", 13Accept: */*
)         = 13
write(1, "\r\n", 2
)                     = 2
write(1, "\n", 1
)                       = 1
read(3, "HTTP/1.1 301 Moved Permanently\r\nDate: Tue, 04 Jul 2017 07:35:12 GMT\r\nServer: Mbedthis-Appweb/2.4.2\r\nContent-type: text/html\r\nETag: \"2f4cfd8-3e1-0\"\r\nContent-length: 0\r\nConnection: close\r\nLocation: https://***/start.html\n\r\n\r\n", 8191) = 256
read(3, "", 8191)                       = 0
write(1, "http://***:80/ is 256 characters\n", 66http://***:80/ is 256 characters
) = 66
write(1, "STATUS: HTTP/1.1 301 Moved Permanently\n", 39STATUS: HTTP/1.1 301 Moved Permanently
) = 39
--- SIGALRM {si_signo=SIGALRM, si_code=SI_KERNEL, si_value={int=0, ptr=0x100000000}} ---
write(1, "CRITICAL", 8CRITICAL)                 = 8
write(1, " - Socket timeout", 17 - Socket timeout)       = 17
+++ exited with 2 +++



2.2.1-2git.el7

write(1, "GET / HTTP/1.1\r\n", 16GET / HTTP/1.1
)      = 16
write(1, "User-Agent: check_http/v2.2.1 (nagios-plugins 2.2.1)\r\n", 54User-Agent: check_http/v2.2.1 (nagios-plugins 2.2.1)
) = 54
write(1, "Connection: close\r\n", 19Connection: close
)   = 19
write(1, "Host: ***\r\n", 44Host: ***
) = 44
write(1, "Accept: */*\r\n", 13Accept: */*
)         = 13
write(1, "\n", 1
)                       = 1
read(3, 0x623e40, 8191)                 = ? ERESTARTSYS (To be restarted if SA_RESTART is set)
--- SIGALRM {si_signo=SIGALRM, si_code=SI_KERNEL, si_value={int=0, ptr=0x100000000}} ---
write(1, "CRITICAL", 8CRITICAL)                 = 8
write(1, " - Socket timeout\n", 18 - Socket timeout
)     = 18
+++ exited with 2 +++



=> 2.2.1 misses after "Accept

write(1, "\r\n", 2


Strange is that single

write(1, "\n", 1

on both checks, is this covered by standards?

found in code that the last \n is not used in request, instead final request looks like:

2.2.0

"GET / HTTP/1.1\r\nUser-Agent: check_http/v2.2.0 (nagios-plugins 2.2.0)\r\nConnection: close\r\nHost: ***\r\nAccept: */*\r\n\r\n"


2.2.1

"GET / HTTP/1.1\r\nUser-Agent: check_http/v2.2.1 (nagios-plugins 2.2.1)\r\nConnection: close\r\nHost: ****\r\nAccept: */*\r\n"


and in code I found:

 // xasprintf (&buf, "%s%s", buf, CRLF);                              // grg: do not append extra CRLF, for HTTP 1.1 compliance


=> looks like one has removed this adding of additional \r\n because of "HTTP 1.1 compliance" 

Is this really valid?


BTW: here is the raw output of response using netcat:


$ echo -e "GET / HTTP/1.1\r\nUser-Agent: check_http/v2.2.0 (nagios-plugins 2.2.0)\r\nConnection: close\r\nHost: ***\r\nAccept: */*\r\n\r\n" | nc *** 80
HTTP/1.1 301 Moved Permanently
Date: Tue, 04 Jul 2017 08:05:57 GMT
Server: Mbedthis-Appweb/2.4.2
Content-type: text/html
ETag: "2f4cfd8-3e1-0"
Content-length: 0
Connection: close
Location: https://***/start.html


$

(just note the 2 new lines after Location are really sent by server)

Thanks for the feedback. Was this with the version I put in epel-testing yesterday ,2.2.1-2git, or the earlier 2.2.1-1 version as they both had major changes to check_http that keep fixing other bugs. If it was with the latest, I am going to open an upstream bug with all the data from this bug in it.

I've downloaded the el7 srpms from koji build with the "2git" suffix.

nagios-plugins-2.2.1-2git.el6 has been pushed to the Fedora EPEL 6 testing repository. If problems still persist, please make note of it in this bug report.
See https://fedoraproject.org/wiki/QA:Updates_Testing for
instructions on how to install test updates.
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-EPEL-2017-62fe0218d0

nagios-plugins-2.2.1-2git.el7 has been pushed to the Fedora EPEL 7 testing repository. If problems still persist, please make note of it in this bug report.
See https://fedoraproject.org/wiki/QA:Updates_Testing for
instructions on how to install test updates.
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-EPEL-2017-cc0aeaca30

issue found on Lenovo IMM and Dell DRAC controllers

examples for problematic responses:


IMM 4.55

  0d 0a 4c 6f 63 61 74 69    6f 6e 3a 20 68 74 74 70    ..Location: http
...
  73 2f 69 6d 6d 2f 69 6e    64 65 78 2e 70 68 70 0a    s/imm/index.php.
  0d 0a 0d 0a        


DRAC 1.92

  6e 3a 20 63 6c 6f 73 65    0d 0a 4c 6f 63 61 74 69    n: close..Locati
...
  63 2f 73 74 61 72 74 2e    68 74 6d 6c 0a 0d 0a 0d    c/start.html....
  0a      

Each header terminates with sequence of \n\r\n\r\n (5 chars)

I don't know whether this is according to RFCs, but at least real life results

nagios-plugins-2.2.1-2git is not solving the issue, but switching from segfault to timeout because of uncatched "end-of-header".

From upstream: Testing against the latest maint patches, I can not confirm the behavior personally. But I can say that were attempting to match up the plugin with strict HTTP checking. The commented xasprintf comes from section 4.1 of RFC 2616 [https://tools.ietf.org/html/rfc2616] - 

>
In the interest of robustness, servers SHOULD ignore any empty
   line(s) received where a Request-Line is expected. In other words, if
   the server is reading the protocol stream at the beginning of a
   message and receives a CRLF first, it should ignore the CRLF.

   Certain buggy HTTP/1.0 client implementations generate extra CRLF's
   after a POST request. To restate what is explicitly forbidden by the
   BNF, an HTTP/1.1 client MUST NOT preface or follow a request with an
   extra CRLF.

It looks, based on your output, that the order of the carriage returns and line feeds may be causing an issue. (Or simply the extra.)

Hopefully we can get this resolved today..

What happens if you re-allow the xasprintf? Does it stop segfaulting?

without having "extra" \n\r the server simply won't respond, still waiting for next request header line I would assume.

./check_http-2.2.1-2git.el6 -v -H  ***
GET / HTTP/1.1
User-Agent: check_http/v2.2.1 (nagios-plugins 2.2.1)
Connection: close
Host: ***
Accept: */*


ngrep:

T 10.13.16.50:59146 -> 10.13.14.38:80 [AP]
  47 45 54 20 2f 20 48 54    54 50 2f 31 2e 31 0d 0a    GET / HTTP/1.1..
  55 73 65 72 2d 41 67 65    6e 74 3a 20 63 68 65 63    User-Agent: chec
  6b 5f 68 74 74 70 2f 76    32 2e 32 2e 31 20 28 6e    k_http/v2.2.1 (n
  61 67 69 6f 73 2d 70 6c    75 67 69 6e 73 20 32 2e    agios-plugins 2.
  32 2e 31 29 0d 0a 43 6f    6e 6e 65 63 74 69 6f 6e    2.1)..Connection
  3a 20 63 6c 6f 73 65 0d    0a 48 6f 73 74 3a 20 **    : close..Host: *
  **************************************************    ****************
  **************************************************    ****************
  ******** 0d 0a 41 63 63    65 70 74 3a 20 2a 2f 2a    ***..Accept: */*
  0d 0a                                                 ..

so 2nd 0d 0a sequence is missing that server detects that header has ended.

nagios-plugins-2.2.1-3git.el6 has been submitted as an update to Fedora EPEL 6. https://bodhi.fedoraproject.org/updates/FEDORA-EPEL-2017-76229ef8c9

nagios-plugins-2.2.1-3git.el6 has been pushed to the Fedora EPEL 6 testing repository. If problems still persist, please make note of it in this bug report.
See https://fedoraproject.org/wiki/QA:Updates_Testing for
instructions on how to install test updates.
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-EPEL-2017-76229ef8c9

nagios-plugins-2.2.1-3git.el7 has been pushed to the Fedora EPEL 7 testing repository. If problems still persist, please make note of it in this bug report.
See https://fedoraproject.org/wiki/QA:Updates_Testing for
instructions on how to install test updates.
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-EPEL-2017-4b1c55c024

nagios-plugins-2.2.1-3git.fc24 has been pushed to the Fedora 24 testing repository. If problems still persist, please make note of it in this bug report.
See https://fedoraproject.org/wiki/QA:Updates_Testing for
instructions on how to install test updates.
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2017-6401b28fc4

nagios-plugins-2.2.1-3git.fc25 has been pushed to the Fedora 25 testing repository. If problems still persist, please make note of it in this bug report.
See https://fedoraproject.org/wiki/QA:Updates_Testing for
instructions on how to install test updates.
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2017-8d031793bf

nagios-plugins-2.2.1-3git.fc26 has been pushed to the Fedora 26 testing repository. If problems still persist, please make note of it in this bug report.
See https://fedoraproject.org/wiki/QA:Updates_Testing for
instructions on how to install test updates.
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2017-a5f81422dc

nagios-plugins-2.2.1-4git.fc26 has been submitted as an update to Fedora 26. https://bodhi.fedoraproject.org/updates/FEDORA-2017-c2e82de3b3

nagios-plugins-2.2.1-4git.fc26 has been pushed to the Fedora 26 testing repository. If problems still persist, please make note of it in this bug report.
See https://fedoraproject.org/wiki/QA:Updates_Testing for
instructions on how to install test updates.
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2017-c2e82de3b3

nagios-plugins-2.2.1-4git.el6 has been pushed to the Fedora EPEL 6 testing repository. If problems still persist, please make note of it in this bug report.
See https://fedoraproject.org/wiki/QA:Updates_Testing for
instructions on how to install test updates.
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-EPEL-2017-8973027f42

nagios-plugins-2.2.1-4git.el7 has been pushed to the Fedora EPEL 7 testing repository. If problems still persist, please make note of it in this bug report.
See https://fedoraproject.org/wiki/QA:Updates_Testing for
instructions on how to install test updates.
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-EPEL-2017-87ebfdc686

nagios-plugins-2.2.1-3git.fc24 has been pushed to the Fedora 24 stable repository. If problems still persist, please make note of it in this bug report.

nagios-plugins-2.2.1-4git.fc26 has been pushed to the Fedora 26 stable repository. If problems still persist, please make note of it in this bug report.

nagios-plugins-2.2.1-4git.el7 has been pushed to the Fedora EPEL 7 stable repository. If problems still persist, please make note of it in this bug report.

nagios-plugins-2.2.1-3git.fc25 has been pushed to the Fedora 25 stable repository. If problems still persist, please make note of it in this bug report.

nagios-plugins-2.2.1-4git.el6 has been pushed to the Fedora EPEL 6 stable repository. If problems still persist, please make note of it in this bug report.

be warned, there is next unexpected issue in check_http (content check is broken now)
https://bugzilla.redhat.com/show_bug.cgi?id=1480085