Bug 1072952 - lpstat/lpadmin occasionally fail (Broken pipe)
Summary: lpstat/lpadmin occasionally fail (Broken pipe)
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Fedora
Classification: Fedora
Component: cups
Version: 20
Hardware: x86_64
OS: Linux
unspecified
unspecified
Target Milestone: ---
Assignee: Tim Waugh
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2014-03-05 13:17 UTC by Jens Kleineheismann
Modified: 2014-06-06 11:23 UTC (History)
3 users (show)

Fixed In Version: cups-1.7.3-1.fc20
Clone Of:
Environment:
Last Closed: 2014-06-01 22:25:02 UTC
Type: Bug
Embargoed:


Attachments (Terms of Use)
Shell output (1.41 KB, text/plain)
2014-03-05 13:17 UTC, Jens Kleineheismann
no flags Details
cups error log (65.91 KB, text/plain)
2014-03-05 13:19 UTC, Jens Kleineheismann
no flags Details
cups access log (4.60 KB, text/plain)
2014-03-05 13:20 UTC, Jens Kleineheismann
no flags Details
/etc/cups/cupsd.conf (4.95 KB, text/plain)
2014-03-05 13:21 UTC, Jens Kleineheismann
no flags Details
/etc/cups/cups-files.conf (3.19 KB, text/plain)
2014-03-05 13:23 UTC, Jens Kleineheismann
no flags Details
/etc/cups/printers.conf (557 bytes, text/plain)
2014-03-05 13:24 UTC, Jens Kleineheismann
no flags Details
output of 'strace lpstat -p' (21.62 KB, text/plain)
2014-03-05 13:53 UTC, Jens Kleineheismann
no flags Details
cups-1.7.1-6.fc20: failed run of 'strace lpstat -p' (27.66 KB, text/plain)
2014-03-07 16:06 UTC, Jens Kleineheismann
no flags Details
cups-1.7.1-6.fc20: failed run of 'strace lpstat -p rp01' (26.68 KB, text/plain)
2014-03-07 16:07 UTC, Jens Kleineheismann
no flags Details
cups-1.7.1-6.fc20: successful run of 'strace lpstat -p rp01' (just for comparison) (32.95 KB, text/plain)
2014-03-07 16:08 UTC, Jens Kleineheismann
no flags Details
cups-str4386.patch (1.32 KB, patch)
2014-04-16 13:08 UTC, Tim Waugh
no flags Details | Diff


Links
System ID Private Priority Status Summary Last Updated
CUPS Bugs and Features 4386 0 None None None Never
CUPS Bugs and Features 4403 0 None None None Never

Description Jens Kleineheismann 2014-03-05 13:17:55 UTC
Created attachment 870955 [details]
Shell output

Description of problem:
Sometimes lpstat -p fail and report a broken pipe, while previous or subsequent calls succeed.
Same with lpadmin.


Version-Release number of selected component (if applicable):
Fedora 20
cups-1.7.0-9.fc20.x86_64


How reproducible:
Sometimes. About 25%. The fail raite seems to increase when adding more printers.


Steps to Reproduce:
1. lpstat -p
2. lpstat -p

Actual results:
1. printer rp01 is idle.  enabled since ...
2. lpstat: Broken pipe

Expected results:
1. printer rp01 is idle.  enabled since ...
2. printer rp01 is idle.  enabled since ...


Additional info:
At my site rp01 is a remote printer (IPP) on another cups server.
My fedora 19 hosts (cups-1.6.4-2) do not show the erroneous behaviour.

For debuging I set LogLevel to debug and after restarting cupsd calling lpstat -p every 30 seconds.
I will attach the shell output, the cups access/error log, and cups config.

Comment 1 Jens Kleineheismann 2014-03-05 13:19:06 UTC
Created attachment 870956 [details]
cups error log

Comment 2 Jens Kleineheismann 2014-03-05 13:20:03 UTC
Created attachment 870957 [details]
cups access log

Comment 3 Jens Kleineheismann 2014-03-05 13:21:54 UTC
Created attachment 870958 [details]
/etc/cups/cupsd.conf

Comment 4 Jens Kleineheismann 2014-03-05 13:23:26 UTC
Created attachment 870959 [details]
/etc/cups/cups-files.conf

Comment 5 Jens Kleineheismann 2014-03-05 13:24:00 UTC
Created attachment 870960 [details]
/etc/cups/printers.conf

Comment 6 Tim Waugh 2014-03-05 13:41:31 UTC
It looks like the error is happening on the client side. The reason I say that is that the error_log looks like it's just missing bits when the error occurs.

Here's the error_log that corresponds to the failure that occurred at 13:42:00:

D [05/Mar/2014:13:42:00 +0100] cupsdAcceptClient: skipping getpeercon()
D [05/Mar/2014:13:42:00 +0100] [Client 16] Accepted from localhost (Domain)
D [05/Mar/2014:13:42:00 +0100] [Client 16] Waiting for request.
D [05/Mar/2014:13:42:00 +0100] [Client 16] POST / HTTP/1.1
D [05/Mar/2014:13:42:00 +0100] cupsdSetBusyState: newbusy="Active clients", busy="Not busy"
D [05/Mar/2014:13:42:00 +0100] [Client 16] No authentication data provided.
D [05/Mar/2014:13:42:00 +0100] cupsdIsAuthorized: username=""
D [05/Mar/2014:13:42:00 +0100] [Client 16] WWW-Authenticate: Basic realm="CUPS"
D [05/Mar/2014:13:42:00 +0100] [Client 16] Closing connection.
D [05/Mar/2014:13:42:00 +0100] cupsdSetBusyState: newbusy="Not busy", busy="Active clients"
D [05/Mar/2014:13:42:00 +0100] cupsdAcceptClient: skipping getpeercon()
D [05/Mar/2014:13:42:00 +0100] [Client 16] Accepted from localhost (Domain)
D [05/Mar/2014:13:42:00 +0100] [Client 16] Waiting for request.
D [05/Mar/2014:13:42:00 +0100] [Client 16] POST / HTTP/1.1
D [05/Mar/2014:13:42:00 +0100] cupsdSetBusyState: newbusy="Active clients", busy="Not busy"
D [05/Mar/2014:13:42:00 +0100] [Client 16] No authentication data provided.
D [05/Mar/2014:13:42:00 +0100] cupsdIsAuthorized: username=""
D [05/Mar/2014:13:42:00 +0100] [Client 16] WWW-Authenticate: Basic realm="CUPS"
D [05/Mar/2014:13:42:00 +0100] [Client 16] Closing connection.

The next step should have been for the client to re-connect using PeerCred credentials, but for some reason it didn't.

Could you please try running 'strace' on the lpstat process, and saving the strace output for a failed run?

Comment 7 Jens Kleineheismann 2014-03-05 13:52:13 UTC
Will attach output of a failed 'strace lpstat -p 2>strace.log'

Comment 8 Jens Kleineheismann 2014-03-05 13:53:33 UTC
Created attachment 870978 [details]
output of 'strace lpstat -p'

Comment 9 Tim Waugh 2014-03-05 16:16:01 UTC
Thanks. Reported upstream.

Comment 10 Fedora Update System 2014-03-06 13:06:44 UTC
cups-1.7.1-6.fc20 has been submitted as an update for Fedora 20.
https://admin.fedoraproject.org/updates/FEDORA-2014-3451/cups-1.7.1-6.fc20

Comment 11 Fedora Update System 2014-03-07 06:32:33 UTC
Package cups-1.7.1-6.fc20:
* should fix your issue,
* was pushed to the Fedora 20 testing repository,
* should be available at your local mirror within two days.
Update it with:
# su -c 'yum update --enablerepo=updates-testing cups-1.7.1-6.fc20'
as soon as you are able to.
Please go to the following url:
https://admin.fedoraproject.org/updates/FEDORA-2014-3451/cups-1.7.1-6.fc20
then log in and leave karma (feedback).

Comment 13 Jens Kleineheismann 2014-03-07 16:04:06 UTC
Have tested cups-1.7.1-6.fc20. Error still comes up.

I will append a new strace output from 'lpstat -p'.

Also I will append another strace output from 'lpstat -p rp01' that also tend to fail with the error message 'Invalid destination name in list "rp01"'. I assume it's the result of the same bug. But maby the strace will help.

Unfortunately I will be offline for about two weeks from now on. So until then I can not provide any assistance. But I will be back :)

Thank you for your support so far.

Comment 14 Jens Kleineheismann 2014-03-07 16:06:27 UTC
Created attachment 871944 [details]
cups-1.7.1-6.fc20: failed run of 'strace lpstat -p'

Comment 15 Jens Kleineheismann 2014-03-07 16:07:03 UTC
Created attachment 871946 [details]
cups-1.7.1-6.fc20: failed run of 'strace lpstat -p rp01'

Comment 16 Jens Kleineheismann 2014-03-07 16:08:09 UTC
Created attachment 871948 [details]
cups-1.7.1-6.fc20: successful run of 'strace lpstat -p rp01' (just for comparison)

Comment 17 Fedora Update System 2014-03-09 04:33:13 UTC
cups-1.7.1-6.fc20 has been pushed to the Fedora 20 stable repository.  If problems still persist, please make note of it in this bug report.

Comment 18 Tim Waugh 2014-04-03 17:02:53 UTC
I've built a new release of cups pulling in a new upstream patch for this.

Comment 19 Tim Waugh 2014-04-03 17:03:59 UTC
Are you able to test the packages directly from koji?
  http://koji.fedoraproject.org/koji/buildinfo?buildID=508931

(don't worry if not... I can arrange for another test update)

Comment 20 Jens Kleineheismann 2014-04-04 13:27:11 UTC
I have downloaded and installed 
 - cups-1.7.1-9.fc20.x86_64.rpm 
 - cups-libs-1.7.1-9.fc20.x86_64.rpm
 - cups-libs-1.7.1-9.fc20.i686.rpm
 - cups-filesystem-1.7.1-9.fc20.noarch.rpm
from koji.
Rebooted after that.

But the error still occures. I would say the error comes up less often, but that may be illusion.

I am still testing, maybe the error only comes up, if multiple clients are running simultaneous or shortly after each other.

Comment 21 Tim Waugh 2014-04-11 16:49:22 UTC
OK, I see this too.

How to reproduce:

1. Edit /etc/cups/cupsd.conf and change the 'Location /' stanza like this:

<Location />
  Order allow,deny
  AuthType Default
  Require valid-user
</Location>

2. systemctl restart cups.service
3. Now run this:
  while :; do lpstat -p; done 2>&1 | grep roken
and wait. It might take, say, a minute of continuous looping before 'lpstat: Broken pipe' is seen.

I'm trying to run a slower test with CUPS debugging enabled on the server to find out what's going on.

Comment 22 Jens Kleineheismann 2014-04-14 07:43:26 UTC
Also for reproducing: the error occur more often if the system load is high.

Comment 23 Tim Waugh 2014-04-14 15:48:00 UTC
Finally got a clean log for this.

Here's the test script:

while :; do date; lpstat -p >/dev/null || break; sleep 5; done 2>&1

It ended like this:

Mon 14 Apr 16:36:26 BST 2014
Mon 14 Apr 16:36:31 BST 2014
lpstat: Broken pipe

Here's the error_log that corresponds to it:

D [14/Apr/2014:16:36:26 +0100] cupsdSetBusyState: newbusy="Not busy", busy="Not 
busy"
D [14/Apr/2014:16:36:31 +0100] cupsdAcceptClient: skipping getpeercon()
D [14/Apr/2014:16:36:31 +0100] [Client 14] Accepted from localhost (Domain)
D [14/Apr/2014:16:36:31 +0100] [Client 14] Waiting for request.
D [14/Apr/2014:16:36:31 +0100] [Client 14] POST / HTTP/1.1
D [14/Apr/2014:16:36:31 +0100] cupsdSetBusyState: newbusy="Active clients", busy
="Not busy"
D [14/Apr/2014:16:36:31 +0100] [Client 14] No authentication data provided.
D [14/Apr/2014:16:36:31 +0100] cupsdIsAuthorized: username=""
D [14/Apr/2014:16:36:31 +0100] [Client 14] WWW-Authenticate: Basic realm="CUPS"
D [14/Apr/2014:16:36:31 +0100] [Client 14] Closing connection.
D [14/Apr/2014:16:36:31 +0100] cupsdSetBusyState: newbusy="Not busy", busy="Active clients"
D [14/Apr/2014:16:36:31 +0100] cupsdAcceptClient: skipping getpeercon()
D [14/Apr/2014:16:36:31 +0100] [Client 14] Accepted from localhost (Domain)
D [14/Apr/2014:16:36:31 +0100] [Client 14] Waiting for request.
D [14/Apr/2014:16:36:31 +0100] [Client 14] POST / HTTP/1.1
D [14/Apr/2014:16:36:31 +0100] cupsdSetBusyState: newbusy="Active clients", busy="Not busy"
D [14/Apr/2014:16:36:31 +0100] [Client 14] No authentication data provided.
D [14/Apr/2014:16:36:31 +0100] cupsdIsAuthorized: username=""
D [14/Apr/2014:16:36:31 +0100] [Client 14] WWW-Authenticate: Basic realm="CUPS"
D [14/Apr/2014:16:36:31 +0100] [Client 14] Closing connection.
D [14/Apr/2014:16:36:31 +0100] cupsdSetBusyState: newbusy="Not busy", busy="Active clients"
D [14/Apr/2014:16:37:44 +0100] cupsdAcceptClient: skipping getpeercon()

Comment 24 Tim Waugh 2014-04-16 13:04:57 UTC
Here's how to reproduce it reliably:

gdb --args lpstat -p
b main
r
b request.c:747
c
c

i.e. put a breakpoint in cupsSendRequest() after httpPost() but before ippWrite().

Comment 25 Tim Waugh 2014-04-16 13:08:05 UTC
Created attachment 886893 [details]
cups-str4386.patch

This patch fixes it for me.

Comment 26 Tim Waugh 2014-04-16 13:15:39 UTC
I've added this new patch to STR #4386 for upstream comment.

Comment 27 Tim Waugh 2014-04-16 15:31:16 UTC
This is now STR #4403.

Comment 28 Fedora Update System 2014-05-28 08:53:25 UTC
cups-1.7.3-1.fc20 has been submitted as an update for Fedora 20.
https://admin.fedoraproject.org/updates/cups-1.7.3-1.fc20

Comment 29 Fedora Update System 2014-05-28 23:53:11 UTC
Package cups-1.7.3-1.fc20:
* should fix your issue,
* was pushed to the Fedora 20 testing repository,
* should be available at your local mirror within two days.
Update it with:
# su -c 'yum update --enablerepo=updates-testing cups-1.7.3-1.fc20'
as soon as you are able to.
Please go to the following url:
https://admin.fedoraproject.org/updates/FEDORA-2014-6798/cups-1.7.3-1.fc20
then log in and leave karma (feedback).

Comment 30 Fedora Update System 2014-06-01 22:25:02 UTC
cups-1.7.3-1.fc20 has been pushed to the Fedora 20 stable repository.  If problems still persist, please make note of it in this bug report.

Comment 31 Jens Kleineheismann 2014-06-06 10:44:51 UTC
Updates installed -> Bug fixed. Great. Thank you very, very much.


Note You need to log in before you can comment on or make changes to this bug.