Cups on my Fedora 41 desktop system was recently upgraded to 1:2.4.11-8. When trying to print, printer pauses. Try to resume printer, but it pauses again without printing. Rebooted system. Did not fix problem. Updated all packages in Fedora updates as of 12/14/21 and rebooted. Did not fix problem. Did a "dnf downgrade cups" which replaced the 1:2.4.11-8 version with the previous 1:2.4.11-2 version. Printing now works. Seems to indicate an issue with the -8 version. Error in system logs each time printer failed to resume was "Backend ipps returned status 4". Reproducible: Always Steps to Reproduce: 1. Upgrade to cups 1:2.4.11-8 2. Attempt to print. Printer will pause without printing 3. Try to resume printer. Printer will throw the error mentioned above and pause again. Actual Results: Printer pauses and will not resume Expected Results: Printer should not pause As mentioned above, downgrading to the previous cups version fixes the issue. Edit to correct typos.
More Info: Printer is a Brother DCP-L2640DW, wireless laser printer/scanner. It uses the IPP everywhere driver. That's what the configuration process picks when adding the printer. From the CUPS web interface (localhost:631): Driver: DCP-L2640DW - IPP Everywhere (grayscale, 2-sided printing) Connection: ipps://Brother2640.local:443/ipp/print The printer has worked with no issues until the latest update. Also tried reinstalling latest cups packages. Problem returned so I downgraded them to the previous version once again. Works fine on the older version, not on the newest.
Changed the driver so now the CUPS web interface shows: Driver: DCP-L2640DW - IPP Everywhere (grayscale, 2-sided printing) Connection: dnssd://Brother%20DCP-L2640DW._ipp._tcp.local/?<some long hex string> This now works. Don't understand the difference between ipps and dnssd but since I originally used ipps and it stopped working with the latest update, I think it should still be considered a bug. Unless it never should have worked in the first place and I just got lucky.
Hi Charles, thank you for reporting the issue! In the latest update CUPS started to comply with system crypto policies, but set of enabled TLS versions and ciphers should not change between the releases - I've scanned cupsd port with nmap before the fix and after the fix and the set of available ciphers was the same, so it is curious such issue showed up. The workaround you've found confirms it is a TLS issue - the backend ipps does send IPP over TLS, dnssd (in your case - dnssd switches internally to IPP if the device supports it) sends plain IPP. But if the printer works over TLS only by insecure/weak algorithms, the behavior is expected. So we have several options: 1. since I see your device work with IPP Everywhere and you use MDNS, you can remove the printer you've installed from CUPS - the printer should work just fine as temporary queue, which should automatically choose the working connection, and common applications support it, 2. change your crypto-policy settings via update-crypto-policies, 3. debug TLS for your printer and your machine - for that I will need: - first cancel any hanging jobs by 'cancel -a' and resume the printer by 'cupsenable <printer-name>', install the latest cups and install the printer with URI 'ipps://Brother2640.local:443/ipp/print' - follow first five points from https://docs.fedoraproject.org/en-US/quick-docs/cups-debug-printing-issues/#_cups_generic_issue , - attach then your /etc/cups/cupsd.conf, /etc/cups/client.conf and ~/.cups/client.conf if any, - attach output of 'update-crypto-policies --show' - capture network traffic to the printer when you print by: $ sudo tcpdump -n -i any -U -s0 -w ipp.pcap port ipp For point 3 I will have to find out about your SSL settings in for CUPS (it is done by SSLOptions directive in client.conf and cupsd.conf files), your current crypto policy (via update-crypto-policies) and what TLS version IPP backend tries to connect to the printer. It is possible the failure happens not due disabled cipher or TLS version, but due disabled key exchange algorithm, which is affected by crypto-policy too but not handled in CUPS, since it is too low-level - in that case you will have to adjust crypto-policies as well too, if you want to use IPPS. Do let me know which option you would like pursue.
Thanks for all that info. Not sure if I really understand it all other than it involves encryption. For the options you mentioned: 1. (Added some extra steps that hopefully explore the issue further.) - I deleted the printer from CUPS by going to localhost:631 in my browser and deleting the printer. - Verified printer was not listed. - Ran "lpinfo -v". Got the following: lpinfo -v network beh network socket network ipps file cups-brf:/ network lpd network ipp network http network https network smb network ipps://Brother%20DCP-L2640DW._ipps._tcp.local/ network dnssd://Brother%20DCP-L2640DW._ipp._tcp.local/?uuid=long-hex-string -- Verified printer did not return to CUPS due to last command. -- Tried to print something by going to thunderbird, selecting the mail message I received from bugzilla and clicking on print. -- The printer was listed as a destination. -- Before clicking the actual print button on the preview, I checked CUPS. The printer was back with the connection type that started with ipps:// -- Clicked the print button. Same problem as before. -- Deleted the printer from CUPS once again. -- Logged into the printers web interface. Was able to find a place where it listed all the network protocols it used. -- Found ipps (and beneath that both Airprint and Mopria) was selected. -- Also selected was mDNS -- Unselected ipps (which also unselected Airprint and Mopria) -- Printer reset itself as expected. -- reran lpinfo -v and got: lpinfo -v network lpd network ipp network socket network ipps network beh file cups-brf:/ network http network https network smb network dnssd://Brother%20DCP-L2640DW._pdl-datastream._tcp.local/?uuid=e3248000-80ce-11db-8000-94ddf812d40f -- Note ipps line is now absent. -- Verified printer still not listed in CUPS. -- Attempted to print email message again. No printer is listed as a destination. -- Checked for printer firmware update. Found there was one available so I updated the firmware. -- Attempted to print email message again. No printer is listed as a destination. -- Issued "systemctl restart avahi-daemon.service" as I though that had something to do with mDNS. Did not fix and issue. At this point I decided to go back and add it to CUPS manually. Since I had disabled the ipp protocol in the printer, the only choice as the dnssd connection so I selected that. However, when I reached the final page of adding a printer and clicked the "Add Printer" button nothing happened. At this point I'm going to go back to my original setting that I got to work and use those.
I removed all printes from CUPS and added my one printer with the DNSSD connection. Works fine. However, when I go to print something (using thunderbird as a test), a second entry for the same printer is added to CUPS using the IPPS connection which does not work. I've named the DNSSD connected printer to include the word DNSSD in it so I know which is which from the dropdown list of printers. If I have time, I may try out what you suggest in option 3 and report back.
Created attachment 2062701 [details] /etc/cups/client.conf
Created attachment 2062703 [details] /etc/cups/cupsd.conf
I've attached /etc/cups/client.conf and /etc/cups/cupsd.conf. You also requested the following: update-crypto-policies --show DEFAULT
Something I've noticed. I set up the printer to use the DNSSD connection and it works. If CUPS is restarted (systemctl restart cups.service), the priver reverts back to the ipps connection which does not work. I then have to modify the printer to use the DNSSD connection.
Found something on the printer using its web interface. There are two choices related to certificates, One is labeled CA Certificate and the other is labeled Certificate. Both have a place to list any certificate but both lists are empty. Since you mentioned a TLS issue, wouldn't that require certificate? In the CA Certificate selection it will allow me to import one but I have no idea where to get one. (yes, I know CA stands for certificate authority.) In the Certificate selection it will allow me to create a self signed certificate, create a CSR and (hopefully, it's greyed out now) install the certificate. Just wondering if I should try generating and installing a self signed cert.
(In reply to Charles Dennett from comment #9) > Something I've noticed. I set up the printer to use the DNSSD connection > and it works. If CUPS is restarted (systemctl restart cups.service), the > priver reverts back to the ipps connection which does not work. I then have > to modify the printer to use the DNSSD connection. This sounds like you use cups-browsed - the daemon prioritizes IPPS over IPP and there is no way how to switch to plain IPP in that daemon atm. Can you disable+stop cups-browsed and do the tests with temporary queue once more (ofc with enabled IPPS+AirPrint on the device)? If you would like to use cups-browsed still, you probably have to change the crypto policy to LEGACY. Otherwise you have default SSL configuration for CUPS, so probably the certificate from the printer is signed by weak algorithm, which gets disabled by crypto policy. If we would pursue TLS debug, I would need you to switch to LEGACY crypto policy at least temporarily so we can find out from GNUTLS debug log what algorithm is used there for connection - hopefully this command will do the trick (otherwise we would have to get output of filters to pass there): $ GNUTLS_DEBUG_LEVEL=99 SSLKEYLOGFILE=/tmp/keylog DEVICE_URI="ipps://Brother%20DCP-L2640DW._ipps._tcp.local/" /usr/lib/cups/backend/ipps 1 root '' 1 '' - &> /tmp/gnutls_ipp_log If the temporary queue does not work or you don't want to use temporary queue, run the command once under DEFAULT crypto policy and attach the created files, then change policy to LEGACY, restart the machine, run the command again and attach the newly created files - I will use it to compare the behavior. And just to be sure, try printing via ipps under LEGACY policy - my guess is it will work, but just to be sure.
Just to be sure I've tested TLS connection between two Fedora machines where one shares IPP queue, and the other prints to temporary queue which connects to the first machine over TLS - this works, so TLS is not completely broken, but some models might have problem with TLS - either due insecure key exchange algorithms or certificate signature algorithms. For any other users coming here, workarounds are: - use plain IPP - IIUC temporary queue can switch to IPP automatically, or you can reinstall the printer with IPP connection, - change crypto policy to LEGACY.
I checked cups-browsed. I do not use it. It was already disabled: sudo systemctl status cups-browsed.service ○ cups-browsed.service - Make remote CUPS printers available locally Loaded: loaded (/usr/lib/systemd/system/cups-browsed.service; disabled; preset: disabled) Drop-In: /usr/lib/systemd/system/service.d └─10-timeout-abort.conf, 50-keep-warm.conf Active: inactive (dead) I changed crypto policy to LEGACY (and rebooted): update-crypto-policies --show LEGACY I ran the tests from option 3 mentioned in comment 3. I will attach the journalctl and tcpdump outputs after finishing this comment. Since the crypto policy was still set to LEGACY, I ran the test you suggested in comment 11 and will attach the output log. You mentioned a temporary queue. Not exactly sure what you mean by that.
Created attachment 2062842 [details] output from tcpdump
Created attachment 2062843 [details] Output from journalctl taken same time as tcpdump
Created attachment 2062845 [details] Output from GNUTLSDEBUG with crypto policy LEGACY
Created attachment 2062848 [details] Output from GNUTLSDEBUG with crypro policy DEFAULT
I changed the crypto policy back to DEFAULT, rebooted, and reran the GNUTLSDEBUG command. The output is attached.
I changed the crypto policy to LEGACY, rebooted and tried to print via ipps. Same problem. It will not print. I think I've dome the tests you requested. If I've missed something, please let me know.
(In reply to Charles Dennett from comment #9) > Something I've noticed. I set up the printer to use the DNSSD connection > and it works. If CUPS is restarted (systemctl restart cups.service), the > priver reverts back to the ipps connection which does not work. I then have > to modify the printer to use the DNSSD connection. I want to make a correction here. I just did a dnf upgrade (unrelated to this issue) and rebooted the system. After the reboot the connection changed from dnssd:// to ipp://, not ipps:// as I stated orignally. I tried printing and it worked.
(In reply to Charles Dennett from comment #13) > You mentioned a temporary queue. Not exactly sure what you mean by that. Temporary queue is a UIX feature available in CUPS since 2016 - the core of the feature is user does not have to install AirPrint/IPP Everywhere printer in common desktop environments (USB printer conntected to the PC, printer on local network by ethernet or wifi) - the printer shows up in printer dialog (is created automatically when it is required) and removed after inactivity. (In reply to Charles Dennett from comment #20) > I want to make a correction here. I just did a dnf upgrade (unrelated to > this issue) and rebooted the system. After the reboot the connection > changed from dnssd:// to ipp://, not ipps:// as I stated orignally. I tried > printing and it worked. Damn, I'm sorry I thought I mentioned already in the previous comments, but it looks I didn't - switching to ipp:// works this around, since it does not use encryption. Those automatic changes are strange however - cupsd/libcups probably updates the queue since its name matches with the name of temporary queue, so it takes as it is temporary, which is IMHO incorrect, but let's focus on get IPPS working atm. Ad logs: DEBUG: Credentials are invalid (New credentials are older than stored credentials.) DEBUG: Printer credentials: Brother2640.local (issued by unknown) / Wed, 31 Dec 2110 23:59:59 GMT / ECDSA-SHA256 / gnutls[3]: ASSERT: ../../../lib/x509/dn.c[_gnutls_x509_parse_dn_oid]:344 gnutls[3]: ASSERT: ../../../lib/x509/dn.c[_gnutls_x509_parse_dn_oid]:442 gnutls[3]: ASSERT: ../../../lib/x509/x509.c[gnutls_x509_crt_get_issuer_dn_by_oid]:998 DEBUG: Stored credentials: Brother2640.local (issued by unknown) / Wed, 31 Dec 2110 23:59:59 GMT / RSA-SHA256 / you've mentioned there was a firmware update, right? This can be caused by firmware update and TLS issue is of different kind we see in this log, or the change in TLS support in CUPS caused the printer to provide better certificate, which does not match with the stored one sent by printer in the past. If certificate changed, desktop environment should offer you a pop-up window mentioning the cert has changed and offer you to choose whether update the cert (you know it is your printer and no some impostor on the network), or do not update which stops the printing. If you accept the new cert, a desktop application should save the new cert and resume printing - but I do not know if there is such desktop pop-up (I don't maintain desktop, so I don't know for sure, but that's what we are counting on in CUPS - we send dbus message stating there was cert change). Can you try moving files from /etc/cups/ssl somewhere else and try printing with IPPS under DEFAULT policy? If it works, then the only problem was the changed certificate.
I think you may have fixed it. I did as you suggested and moved the files in /etc/cups/ssl out of the way, set crypto policy to DEFAULT, deleted the existing printer in CUPS and added it back as a new one and selected the ipps:// connection. Printing works. Even after restarting cups.service and also after a reboot printing still works. Must have been the old certificate. All this started before the firmware update to the printer, by the way, so I can't see how that would have cause the issue. I looked at both old and new cert files and they were very different in terms of their size. I have not yet decoded them. I did notice one thing about the connection. When I first added the printer, it was shown as : Connection: ipps://Brother%20DCP-L2640DW._ipps._tcp.local/ After restarting cups.service it changed to: Connection: ipps://Brother2640.local:443/ipp/print Both worked as expected. I also looked at /etc/cups/printer.conf both before and after restarting cups.service. It did not change. It was: DeviceURI ipps://Brother2640.local:443/ipp/print As for the temporary queue, I see what you mean. I tried printing from both thunderbird and chrome. The printer shows up in the print dialog along with the printer I configured. In chrome, either one works. In thunderbird, if I select the temporary one, nothing happens. No output at the printer and nothing in the journal. Probably a thunderbird issue, then. I'm not going to worry about it since the configured printer now seems to work without issue. If you need anything else, let me know. Thanks for all the help. I really appreciate it.
(In reply to Charles Dennett from comment #22) > All this started before the firmware update to the printer, by > the way, so I can't see how that would have cause the issue. I've meant it like the cert issue could hide the original in the logs you've provided, but since the printing works after removing the old certs, I suspect GNUTLS (crypto provider) required better signing algorithm from the printer, so the printer generated a new one signed with ECDSA-SHA256, which of course did not match with the old one, causing the printing stop because there was suspicion there is a different device you are connecting to. > As for the temporary queue, I see what you mean. I tried printing from both > thunderbird and chrome. The printer shows up in the print dialog along with > the printer I configured. In chrome, either one works. In thunderbird, if > I select the temporary one, nothing happens. No output at the printer and > nothing in the journal. Probably a thunderbird issue, then. I'm not going > to worry about it since the configured printer now seems to work without > issue. Interesting - thunderbird works fine for in F40 and temporary queue. It would be great if I could look into it, if you have time. In general you can test whether it is thunderbird issue or CUPS/printer issue by printing by lp, however your permanent and temporary queue names are the same - so you will have to rename the permanent one, so you can be sure you print to the temporary. So if you want to check, remove the installed print queue and install it again this way (it is doable by CUPS web ui, but this requires less clicks :) ) - f.e.: $ lpadmin -p brother-permanent -v ipps://Brother%20DCP-L2640DW._ipps._tcp.local/ -m everywhere -E then check the output of `lpstat -l -e` - the one with "network" on the line is the temporary one. And then try printing to it by, f.e.: $ lp -d <name of temp queue> /etc/fstab If this works, then the issue is in thunderbird (but more likely in GTK3 which implements the dialog), if it does not work, I would like to know more if you have time to file another ticket and help with debug. > Thanks for all the help. I really appreciate it. No problem, I'm sorry for inconvenience. IMO there is nothing I can do to automate this (without possible security implications if the "printer" really was an attacker and I would remove certs during update to fix this), so each user has to be check certificates in /etc/cups/ssl if they hit this issue and desktop does not notify them.
I can't automate the fix (removal old cert) due security implications in case there was really an attacker impersonating as a known printer - cupsd generates dbus event about this situation, but it is up to desktop if it processes and ask user to confirm the action ("This printer changed certificate, do you want to proceed") and saves the new certificate as the correct one. The way how to fix the situation is to remove cert and keyfile related to the printer (their filename is printer's hostname) from /etc/cups/ssl, clean and resume the print queue by: $ cancel -a $ cupsenable <print queue name> and then the print queue works.
I learned a few days ago to give a unique name to my permanent printer queue. I named it THE_REAL_Brother_DCP-L2640DW. The temporary one shows up as . Brother_DCP-L2640DW. I was able to print to both of them successfully from the command line as you suggested. So it's apparently not a CUPS/printer issue. When I printed to the temporary one, I quickly looked in the CUPS web interface and it used the name of the permanent one even though I used the name of the temporary one. But, as long as it works, I'm happy. BTW, here's what lpstat told me: lpstat -l -e Brother_DCP_L2640DW network none ipps://Brother%20DCP-L2640DW._ipps._tcp.local/ THE_REAL_Brother_DCP-L2640DW permanent ipp://localhost/printers/THE_REAL_Brother_DCP-L2640DW ipps://Brother2640.local:443/ipp/print Thanks again!