Bug 453610 - Add printer in CUPS admin stalls
Summary: Add printer in CUPS admin stalls
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Enterprise Linux 5
Classification: Red Hat
Component: cups
Version: 5.2
Hardware: i386
OS: Linux
low
low
Target Milestone: rc
: ---
Assignee: Tim Waugh
QA Contact:
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2008-07-01 15:12 UTC by Matthew Mosesohn
Modified: 2018-10-20 02:59 UTC (History)
10 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2009-01-20 21:59:34 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)
cups debug output (8.03 KB, text/plain)
2008-07-18 23:59 UTC, Kai Engert (:kaie) (inactive account)
no flags Details
firefox-http-request.txt (2.21 KB, text/plain)
2008-07-21 18:45 UTC, Tim Waugh
no flags Details
firefox-https.log (7.67 KB, text/plain)
2008-07-21 18:45 UTC, Tim Waugh
no flags Details
cups-debug.patch (431 bytes, patch)
2008-07-21 18:46 UTC, Tim Waugh
no flags Details | Diff


Links
System ID Private Priority Status Summary Last Updated
CUPS Bugs and Features 2892 0 None None None Never
Red Hat Product Errata RHEA-2009:0201 0 normal SHIPPED_LIVE cups enhancement update 2009-01-20 16:06:07 UTC

Description Matthew Mosesohn 2008-07-01 15:12:02 UTC
Description of problem:

Adding a printer in CUPS admin fails when trying to select Make of printer.

Version-Release number of selected component (if applicable):

RHEL 5.2
firefox-3.0-0.beta5.6.el5
cups-1.2.4-11.14.el5


How reproducible:

Always

Steps to Reproduce:
1. Go to host:631/admin
2. Click Add Printer
3. Fill out name, description, etc
4. Choose printer type
5. Chose Make (for example HP)
6. Hit Continue
  
Actual results:

Page loads and loads, eventually drops to a blank screen.

Expected results:

Proceed to the Model selection page.

Additional info:

Not an issue in Firefox 1.5 or 2.
Reproduceable across all Firefox 3 beta 5 versions in RHEL 5.1, 5.2, Fedora 9,
and on a Mac.

No issues reported in Error Console.

Comment 1 Matthew Mosesohn 2008-07-01 15:29:00 UTC
Apparently this is part of a known CGI issue in CUPS 1.2.4:
http://cups.org/newsgroups.php?gcups.general+v:32688

From the link, the latest 1.2 stable release (1.2.12) has this fixed.


Comment 2 Christopher Aillon 2008-07-01 15:38:24 UTC
Moving to cups.

Comment 3 Tim Waugh 2008-07-02 08:50:36 UTC
I can't reproduce this here.

If this is reproducible on Fedora 9, the problem is probably not related to the
issue you cite in comment #1, as Fedora 9 ships with CUPS 1.3.7.

See the thread "webinterface with firefox 3", which starts here:
http://cups.org/newsgroups.php?s15516+gcups.general+v16496+T1

The most recent message in the thread, from Michael Sweet:

==>
John Holmstadt wrote:
> ...
> I am using Firefox 3.0 on Windows XP, attempting to add a printer
 > to Cups 1.3.7 on Fedora 8 and I am seeing the same problem.  I hit
 > continue after specifying the Make/Manufacturer and Firefox hangs
 > saying "Waiting for <mydomain.com>". I confirmed that trying the
 > same operations on IE 7 from the same machine worked fine.

Hmm, that sounds like a FF3 bug then.  Can you download Wireshark
(if you don't already have it) and run it to record the session
between FF3 and your FC8 system?
<==

Perhaps you could try that?

Comment 4 Tim Waugh 2008-07-04 16:42:05 UTC
Changing component to Firefox, since it seems to be a problem with its handling
of encrypted connections (according to upstream discussion referenced above).

The correct URL for the cups.general thread is this:
  http://cups.org/newsgroups.php?gcups.general+v:33070

and the specific message that causes me to think it is a Firefox bug is this:
  http://cups.org/newsgroups.php?gcups.general+v:33140

Comment 5 Kai Engert (:kaie) (inactive account) 2008-07-08 21:17:29 UTC
After reading this bug and the quoted links on cups.org, I suspect/hope there is
some confusion going on.

A server that expects https can not be accessed using plain http.
A server that expects plain http can not be accessed using https.

Attempts to access a server using the wrong protocol will either result in an
error message or the stalling you're experiencing.

It appears to me that cups can be configured to expect either https or http, but
always uses the same port number 631?

I think that's confusing and unfortunate.

Any other protocols I'm aware of are using different port numbers for
always-plain and always-encrypted.

(With the exception of protocols that use the start-tls variant, which always
start with a plaintext phase and switch to ssl/tls as part of the application
protocol).


Having said that, I would like to ask for more tests.

- disable encryption in cups (port 631 expects http)
- use firefox 2 and http (should work)
- use firefox 3 and http (should work)
- use firefox 2 and https (should stall or fail)
- use firefox 3 and https (should stall or fail)

- enable encryption in cups (port 631 expects https)
- use firefox 2 and http (should stall or fail)
- use firefox 3 and http (should stall or fail)
- use firefox 2 and https (should work)
- use firefox 3 and https (should work)


Comment 6 Tim Waugh 2008-07-09 11:09:51 UTC
(In reply to comment #5)
> A server that expects https can not be accessed using plain http.
> A server that expects plain http can not be accessed using https.

This is not how it works at all.  For example, take a look at how 'lpstat -h
localhost -E -s' works.

The initial connection is made using plain HTTP.  The client then requests a
secure connection with:

Connection: Upgrade
Upgrade: TLS/1.0,SSL/2.0/SSL/3.0

Then the server responds with:

Connection: Upgrade
Upgrade: TLS/1.0,HTTP/1.1 (or whatever)

and further communications are encrypted.

Additionally, CUPS can automatically check new connections for an SSL/TLS
handshake, and encrypt the connection from the start.  This is how Firefox,
links etc should be treated when they use 'https://localhost:631'.

> It appears to me that cups can be configured to expect either https or http, but
> always uses the same port number 631?

CUPS can accept both HTTPS and HTTP at the same time from different clients,
depending on the 'Encryption' keyword in /etc/cups/cupsd.conf.

Comment 7 Kai Engert (:kaie) (inactive account) 2008-07-14 16:21:14 UTC
Thanks for the explanation, but I think I still need more info.

You're saying that the CUPS server tries to be smart in multiple ways.

You say, if the conversation on the CUPS protocol port starts in plaintext, it
can switch to encrypted, which sounds similar to the STARTTLS mechanism used in
email protocols. That's smart.

But you also say, the single CUPS server port tries to detect whether the client
is sending encrypted or not. This is a smart idea, but I think it's tricky to
implement and might be where some software is getting confused and end up
stalling. I don't know.

I would like to understand the exact configuration that is failing.

Are you saying the failing scenario is:
- server with encryption enabled
- firefox 3 connecting with https

And are you saying that with the same server configuration:
- firefox 2 connecting with https works
?

I would like to reproduce your setup locally.


Comment 8 Kai Engert (:kaie) (inactive account) 2008-07-18 20:11:44 UTC
from cupsd.conf manpage:
       Encryption IfRequested

       Encryption Never

       Encryption Required
            Specifies the level of encryption that is required 
            for a particular location.

Which setting did you use?

Comment 9 Kai Engert (:kaie) (inactive account) 2008-07-18 21:14:40 UTC
I am able to reproduce on both RHEL 5.2 and Fedora 9.

I use the default cupsd.conf, and it's necessary to use https.
https://localhost.localdomain:631/admin

I added an exception for the self-signed certificate, and while the first few
pages of the process work just fine, it stalls when I press "continue" on the
page with the heading "Make/Manufacturer".

All pages use https, but only one page stalls.


Comment 10 Kai Engert (:kaie) (inactive account) 2008-07-18 21:16:08 UTC
On RHEL I see some messages in the logfiles:

I [18/Jul/2008:20:54:41 +0200] cupsdCloseClient: SSL shutdown successful!

==> access_log <==
localhost - - [18/Jul/2008:20:58:15 +0200] "POST /admin HTTP/1.1" 200 1742 - -

==> error_log <==
I [18/Jul/2008:20:58:20 +0200] Started "/usr/lib/cups/cgi-bin/admin.cgi" (pid=7753)
I [18/Jul/2008:20:58:20 +0200] Started "/usr/lib/cups/daemon/cups-driverd"
(pid=7754)

==> access_log <==
localhost - - [18/Jul/2008:20:58:20 +0200] "POST / HTTP/1.1" 200 30002
CUPS-Get-PPDs -

==> error_log <==
E [18/Jul/2008:20:58:27 +0200] cupsdCloseClient: Error in the push function.
E [18/Jul/2008:20:58:27 +0200] PID 7753 (/usr/lib/cups/cgi-bin/admin.cgi)
crashed on signal 9!
I [18/Jul/2008:20:58:27 +0200] Hint: Try setting the LogLevel to "debug" to find
out more.

Comment 11 Kai Engert (:kaie) (inactive account) 2008-07-18 21:17:19 UTC
I'm going to move this over to the cups component, which seems the most likely
component having a problem, but I'll try to trace the SSL connection.


Comment 12 Kai Engert (:kaie) (inactive account) 2008-07-18 22:59:06 UTC
After lots of output from ssl tracing tool "ssltap" and a long delay (5
minutes), when Firefox switches to the blank page, the ssltap output concludes
with the following piece:

<-- [
(37 bytes of 32)
SSLRecord { [Fri Jul 18 23:41:14 2008]
   type    = 21 (alert)
   version = { 3,0 }
   length  = 32 (0x20)
            < encrypted >
}
]
Read EOF on Server socket. [Fri Jul 18 23:41:14 2008]
Read EOF on Client socket. [Fri Jul 18 23:41:14 2008]
Connection 11 Complete [Fri Jul 18 23:41:14 2008]


The cups server sends an alert record and closes the socket.

We don't know yet whether the server is incorrectly waiting for more data to
arrive, or whether Firefox fails to send required data.


Comment 13 Kai Engert (:kaie) (inactive account) 2008-07-18 23:59:52 UTC
Created attachment 312181 [details]
cups debug output

I changed cupsd.conf to contain:
  LogLevel debug

I'm attaching the output that was produced when I clicked "continue" which
results in "nothing but stalls".

Comment 14 Tim Waugh 2008-07-21 18:45:10 UTC
Comparing what Firefox sends over the 'http://localhost:631' connection when
submitting that form with what cupsd receives over the equivalent
'https://localhost:631' form submission from Firefox, there is definitely data
missing from the SSL connection.

However, when I click 'Stop' in Firefox, the remaining data does appear at the
cupsd end.

Further investigation required. (I'm away the rest of this week.. anyone else
able to look at this?)

Attached:
1. firefox-http-request.txt, obtained from wireshark, showing the form
submission from Firefox when using the 'http://localhost:631' (non-SSL) URL
2. firefox-http.log, obtained from 'cupsd -f', showing the form submission from
Firefox when using the 'https://localhost:631' (SSL) URL.  This shows the
incomplete request, and the remaining data when the 'stop' button is clicked.
3. cups-debug.patch, used to obtain firefox-http.log


Comment 15 Tim Waugh 2008-07-21 18:45:38 UTC
Created attachment 312288 [details]
firefox-http-request.txt

Comment 16 Tim Waugh 2008-07-21 18:45:59 UTC
Created attachment 312289 [details]
firefox-https.log

Comment 17 Tim Waugh 2008-07-21 18:46:17 UTC
Created attachment 312290 [details]
cups-debug.patch

Comment 18 Kai Engert (:kaie) (inactive account) 2008-07-21 18:59:01 UTC
I suspect the CUPS/SSL server might be unable to correctly handle certain data
packets.

I compared the data sent by FF2 vs. FF3.

The data wrapped inside SSL is identical, if we ignore the "Accept:" and
"User-Agent:" HTML headers. The data I see is equivalent to Tim's attachment 312288 [details].

I noticed that Firefox 2 sends the request using 4 separate write calls to the
operating system:
-1244062832[8a677a8]: [87cb5b8] wrote 449 bytes (out of 449)
-1244062832[8a677a8]: [87cb5b8] wrote 102 bytes (out of 102)
-1244062832[8a677a8]: [87cb5b8] wrote 24 bytes (out of 24)
-1244062832[8a677a8]: [87cb5b8] wrote 1736 bytes (out of 1736)

But Firefox 3 is smarter, it queues the bytes for the request and I see only a
single request being sent out:
-1240466544[b4a0e8a0]: [b14cd120] wrote 2275 bytes (out of 2275)


(Although those numbers don't sum up to the same total number, I verified the
contents are equivalent. The differences are caused by different user-agent and
accept headers.)


I was able to reproduce the hang with a local debug build of Firefox 3.

Then I changed FF 3 to use a maximum of 1000 bytes for write calls on SSL
connections, and I got:
-1240466544[b4a118a0]: [b14c5680] wrote 1000 bytes (out of 2296)
-1240466544[b4a118a0]: [b14c5680] wrote 1000 bytes (out of 1296)
-1240466544[b4a118a0]: [b14c5680] wrote 296 bytes (out of 296)


This made it work for me.

I seem to remember a similar bug in an SSL library, that has been fixed in newer
versions. I'm not sure if it's related and I should try to look up the details.


However, I'd like to invite the CUPS developers to look into this issue now.
Is it possible that there is a bug in the CUPS receiving side?

In comment 14 Tim said he receives the remaining data in the debug log after he
presses the stop button in Firefox.
According to my tracing, the data has already been sent by Firefox.
I suspect that pressing the stop button causes another tcp/ip event to go out
that causes a wakeup on the CUPS/SSL server side and causes it to process the
remaining data.


Comment 19 Tim Waugh 2008-08-01 14:48:28 UTC
(In reply to comment #18)
> I seem to remember a similar bug in an SSL library, that has been fixed in newer
> versions. I'm not sure if it's related and I should try to look up the details.

That would be extremely useful.

Comment 20 Tim Waugh 2008-08-01 16:06:32 UTC
I've filed this upstream with a work-around patch that seems to show it really
is a CUPS problem:
  http://cups.org/str.php?L2892

In this case, gnutls has read the data and holds it in a buffer, but cupsd is
using poll() to await more data instead of trying gnutls_record_check_pending().

Comment 22 RHEL Program Management 2008-08-01 16:11:27 UTC
This request was evaluated by Red Hat Product Management for inclusion in a Red
Hat Enterprise Linux maintenance release.  Product Management has requested
further review of this request by Red Hat Engineering, for potential
inclusion in a Red Hat Enterprise Linux Update release for currently deployed
products.  This request is not yet committed for inclusion in an Update
release.

Comment 23 Fedora Update System 2008-08-03 14:00:51 UTC
cups-1.3.8-1.fc9 has been submitted as an update for Fedora 9

Comment 25 Fedora Update System 2008-08-07 23:56:45 UTC
cups-1.3.8-2.fc9 has been pushed to the Fedora 9 testing repository.  If problems still persist, please make note of it in this bug report.
 If you want to test the update, you can install it with 
 su -c 'yum --enablerepo=updates-testing update cups'.  You can provide feedback for this update here: http://admin.fedoraproject.org/updates/F9/FEDORA-2008-7093

Comment 26 Tim Waugh 2008-08-11 11:30:15 UTC
F-9 update works for me.

Comment 27 Fedora Update System 2008-09-25 00:18:07 UTC
cups-1.3.8-2.fc9 has been pushed to the Fedora 9 stable repository.  If problems still persist, please make note of it in this bug report.

Comment 29 Nick Strugnell 2008-11-19 16:23:10 UTC
We just reproduced with FF3.0.2. 

However, if you are hitting this on customer sites, IE7 works just fine.

Nick

Comment 31 errata-xmlrpc 2009-01-20 21:59:34 UTC
An advisory has been issued which should help the problem
described in this bug report. This report is therefore being
closed with a resolution of ERRATA. For more information
on therefore solution and/or where to find the updated files,
please follow the link below. You may reopen this bug report
if the solution does not work for you.

http://rhn.redhat.com/errata/RHEA-2009-0201.html


Note You need to log in before you can comment on or make changes to this bug.