Bug 770682

Summary: nss update breaks pidgin-sipe connectivity
Product: [Fedora] Fedora Reporter: peter
Component: pidgin-sipeAssignee: Stefan Becker <chemobejk>
Status: CLOSED ERRATA QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: high Docs Contact:
Priority: unspecified    
Version: 20CC: birger, caillon, cfeller, chemobejk, chkr, csimpson, dpal, dwmw2, emaldona, harry.sutton, jhorak, jyundt, kdudka, kengert, ktdreyer, matt, rrelyea, tmraz
Target Milestone: ---Keywords: Reopened
Target Release: ---   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: nss-3.14.3-13.0.fc19 Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2014-01-27 06:02:49 EST Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---
Attachments:
Description Flags
NSS_SSL_CBC_RANDOM_IV defaults to 0, is changed to 1 when asked for rrelyea: review+

Description peter 2011-12-28 05:30:21 EST
Description of problem:

Latest nss update breaks pidgin-sipe connectivity.


Version-Release number of selected component (if applicable):
nss-3.13.1-9.fc16


How reproducible:
Every time.

Steps to Reproduce:
1. Update of firefox to version firefox-9.0-3.fc16.x86_64 (+ other requisites, such as xulrunner, gecko-libs, nss, etc..)
2. Start pidgin, connect to Microsoft OCS / Lync server using sipe plugin
  
Actual results:
Pidgin-sipe connection terminates with error: 'Read error'

Expected results:
Pidgin-sipe connects to OSC normally.

Additional info:
Similar bug report on Debian:
bugs.debian.org/cgi-bin/bugreport.cgi?bug=649456

Workaround:
Downgrade nss and all dependent packages.

yum downgrade nss-sysinit nss xulrunner gecko-libs firefox
Loaded plugins: langpacks, presto, protectbase, refresh-packagekit, versionlock
Setting up Downgrade Process
5 packages excluded due to repository protections
Resolving Dependencies
--> Running transaction check
---> Package firefox.x86_64 0:7.0.1-1.fc16 will be a downgrade
---> Package firefox.x86_64 0:9.0-3.fc16 will be erased
---> Package nss.i686 0:3.12.10-7.fc16 will be a downgrade
---> Package nss.x86_64 0:3.12.10-7.fc16 will be a downgrade
---> Package nss.i686 0:3.13.1-9.fc16 will be erased
---> Package nss.x86_64 0:3.13.1-9.fc16 will be erased
---> Package nss-sysinit.x86_64 0:3.12.10-7.fc16 will be a downgrade
---> Package nss-sysinit.x86_64 0:3.13.1-9.fc16 will be erased
---> Package xulrunner.x86_64 0:7.0.1-1.fc16 will be a downgrade
---> Package xulrunner.x86_64 0:9.0-2.fc16 will be erased
--> Finished Dependency Resolution

Dependencies Resolved

================================================================================
 Package             Arch           Version                Repository      Size
================================================================================
Downgrading:
 firefox             x86_64         7.0.1-1.fc16           fedora          17 M
 nss                 i686           3.12.10-7.fc16         fedora         778 k
 nss                 x86_64         3.12.10-7.fc16         fedora         764 k
 nss-sysinit         x86_64         3.12.10-7.fc16         fedora          32 k
 xulrunner           x86_64         7.0.1-1.fc16           fedora         9.2 M

Transaction Summary
================================================================================
Downgrade     5 Packages

Total download size: 28 M
Is this ok [y/N]: y
Downloading Packages:
(1/5): firefox-7.0.1-1.fc16.x86_64.rpm                   |  17 MB     00:03     
(2/5): nss-3.12.10-7.fc16.i686.rpm                       | 778 kB     00:00     
(3/5): nss-3.12.10-7.fc16.x86_64.rpm                     | 764 kB     00:00     
(4/5): nss-sysinit-3.12.10-7.fc16.x86_64.rpm             |  32 kB     00:00     
(5/5): xulrunner-7.0.1-1.fc16.x86_64.rpm                 | 9.2 MB     00:01     
--------------------------------------------------------------------------------
Total                                           4.8 MB/s |  28 MB     00:05     
Running Transaction Check
Running Transaction Test
Transaction Test Succeeded
Running Transaction
  Installing : nss-3.12.10-7.fc16.x86_64                                   1/10 
  Installing : nss-sysinit-3.12.10-7.fc16.x86_64                           2/10 
  Installing : xulrunner-7.0.1-1.fc16.x86_64                               3/10 
  Installing : firefox-7.0.1-1.fc16.x86_64                                 4/10 
  Installing : nss-3.12.10-7.fc16.i686                                     5/10 
  Cleanup    : nss-3.13.1-9.fc16                                           6/10 
  Cleanup    : firefox-9.0-3.fc16.x86_64                                   7/10 
  Cleanup    : xulrunner-9.0-2.fc16.x86_64                                 8/10 
  Cleanup    : nss-3.13.1-9.fc16                                           9/10 
  Cleanup    : nss-sysinit-3.13.1-9.fc16.x86_64                           10/10 

Removed:
  firefox.x86_64 0:9.0-3.fc16          nss.i686 0:3.13.1-9.fc16                 
  nss.x86_64 0:3.13.1-9.fc16           nss-sysinit.x86_64 0:3.13.1-9.fc16       
  xulrunner.x86_64 0:9.0-2.fc16       

Installed:
  firefox.x86_64 0:7.0.1-1.fc16        nss.i686 0:3.12.10-7.fc16               
  nss.x86_64 0:3.12.10-7.fc16          nss-sysinit.x86_64 0:3.12.10-7.fc16     
  xulrunner.x86_64 0:7.0.1-1.fc16
Comment 1 Adam Hough 2011-12-28 11:39:23 EST
http://sourceforge.net/apps/mediawiki/sipe/index.php?title=FAQ

Q: After upgrading to NSS 3.13.1 I only get "SSL Read Error"
A: There is a bug in NSS 3.13.1, reportedly caused by a fix for CV2-2011-3640, which causes SSL connections to abort. Unfortunately there is nothing that can be done in the pidgin-sipe code, as it simply uses the SSL connection API offered by libpurple. Confirmed workarounds are:
Downgrade to NSS 3.12.x. If prelink is activated on your system, please make sure to do a forced prelink run to ensure that the new "old" libraries are taken into use by libpurple's ssl-nss.so plugin.
Compile pidgin with the options "--enable-gnutls=yes --enable-nss=no" to replace NSS with GnuTLS.
Comment 2 Kamil Dudka 2011-12-28 12:38:54 EST
(In reply to comment #0)
> Workaround:
> Downgrade nss and all dependent packages.

There is a better workaround suggested in the Debian bug:

http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=649456#27

export NSS_SSL_CBC_RANDOM_IV=0

... and this is a duplicate of bug 770419 I guess.
Comment 3 Adam Hough 2011-12-28 13:57:01 EST
Thanks Kamil,

That environment variable worked at getting pidgin to connect to the Microsoft Communicator server.

- Adam
Comment 4 peter 2011-12-29 04:31:40 EST
The proposed workaround works.
Is it possible to expect this to be fixed permanently in the future versions?
Comment 5 Kamil Dudka 2011-12-29 09:14:10 EST
(In reply to comment #4)
> Is it possible to expect this to be fixed permanently in the future versions?

Unfortunately, it is the other way around.  This breakage is the result of a security fix.  As far as I understand it, by setting the environment variable, you paralyze the security fix.  NSS upstream expects this to be fixed on the server's side.  The best place to discuss this is the upstream bugzilla:

https://bugzilla.mozilla.org/show_bug.cgi?id=665814#c129
Comment 6 Stefan Becker 2011-12-30 07:43:15 EST
(In reply to comment #4)
> The proposed workaround works.
> Is it possible to expect this to be fixed permanently in the future versions?

I've raised this issue with a Microsoft Manager responsible for the Lync platform. Let's see how they react.
Comment 7 Elio Maldonado Batiz 2012-01-05 18:09:10 EST
The fix in https://bugzilla.mozilla.org/show_bug.cgi?id=665814#c129 sets the NSS_SSL_CBC_RANDOM_IV to 1 by default - following the principle of setting the default to the most secure setting as opposed to than the most compatible one I presume. Commendable as it is from a security point of view, this has caused quite a bit of problems as we are uncovering flaws in existing products and have broken several applications or libraries on unsuspecting users. Some have been patched, (e.g. libcurl and a pki tool). For others (e.g OpenSSL) the fedora package maintainers had submitted patches upstream which are taking quite while to be reviewed and approved. The change review and approval machinery often grinds slowly. And they are others that we don't know yet. 

Rather that asking the user to set the environment variable to zero when they run into problems we should instead turn things around. I propose that NSS set NSS_SSL_CBC_RANDOM_IV to 0 by default and if client applications or users prefer more security they can always set to 1 themselves. I'll attach a patch for this next.
Comment 8 Elio Maldonado Batiz 2012-01-05 18:13:35 EST
Created attachment 551038 [details]
NSS_SSL_CBC_RANDOM_IV defaults to 0, is changed to 1 when asked for
Comment 9 Elio Maldonado Batiz 2012-01-05 18:16:18 EST
scratch build at http://koji.fedoraproject.org/koji/taskinfo?taskID=3623116
Comment 10 Elio Maldonado Batiz 2012-01-05 18:19:34 EST
(In reply to comment #9)
Correct URL is http://koji.fedoraproject.org/koji/taskinfo?taskID=3623429
Comment 11 Bob Relyea 2012-01-06 18:08:22 EST
Comment on attachment 551038 [details]
NSS_SSL_CBC_RANDOM_IV defaults to 0, is changed to 1 when asked for

r+ rrelyea
Comment 12 Bob Relyea 2012-01-06 18:24:06 EST
This patch fixes the regression that the security fix introduces. It fixes it by reverting to the vunerable behavior by default.

I believe that this is the right action because F16 and F15 are stable branches, and because the security vunerability is really low. Users who are worried about this attack and turn the code back on by flipping the environment variable to 1.

That said, the issues that are occuring are actually latent bugs in the underlying applications (or the servers they connect to). I suggest that we start looking at the applications that are failing and make sure we fix the underlying issues for rawhide. (NSS 3.13 has been in rawhide since Oct).

NOTE: Explicitly turning off this functionality in the application should not be the solution, but if doing so solves the problem, the the underlying issue is probably with the server the application connects to rather than the application itself.


bob
Comment 13 Fedora Update System 2012-01-06 20:42:29 EST
nss-softokn-3.13.1-15.fc16, nss-3.13.1-10.fc16 has been submitted as an update for Fedora 16.
https://admin.fedoraproject.org/updates/FEDORA-2012-0004/nss-3.13.1-10.fc16,nss-softokn-3.13.1-15.fc16
Comment 14 Stefan Becker 2012-01-07 01:52:27 EST
(In reply to comment #12)
> That said, the issues that are occuring are actually latent bugs in the
> underlying applications (or the servers they connect to). I suggest that we
> start looking at the applications that are failing and make sure we fix the
> underlying issues for rawhide. (NSS 3.13 has been in rawhide since Oct).

In the pidgin case this would mean to add this option disable to the ssl-nss.so plugin, as the protocol plugins handle SSL via libpurple APIs and don't see NSS directly. That would mean the security fix would be disabled for all SSL connections, although in reality maybe only the Microsoft Lync Server is the only one left who can't handle it.

But it might better than nothing...
Comment 15 Fedora Update System 2012-01-07 18:07:56 EST
Package nss-softokn-3.13.1-15.fc16, nss-3.13.1-10.fc16:
* should fix your issue,
* was pushed to the Fedora 16 testing repository,
* should be available at your local mirror within two days.
Update it with:
# su -c 'yum update --enablerepo=updates-testing nss-softokn-3.13.1-15.fc16 nss-3.13.1-10.fc16'
as soon as you are able to.
Please go to the following url:
https://admin.fedoraproject.org/updates/FEDORA-2012-0004/nss-3.13.1-10.fc16,nss-softokn-3.13.1-15.fc16
then log in and leave karma (feedback).
Comment 16 Fedora Update System 2012-01-11 01:18:47 EST
nss-softokn-3.13.1-15.fc16, nss-3.13.1-10.fc16 has been pushed to the Fedora 16 stable repository.  If problems still persist, please make note of it in this bug report.
Comment 17 Chad Feller 2012-01-11 20:36:54 EST
Is this going to be pushed to the F15 testing repo?  Koji still shows the updates-candidate tag for the F15 build.
Comment 18 Elio Maldonado Batiz 2012-01-11 21:13:18 EST
(In reply to comment #17)
> Is this going to be pushed to the F15 testing repo?  Koji still shows the
> updates-candidate tag for the F15 build.

Definitely, this must be pushed for F-15. Must coordinate with FF/TB folks.

Jan, Are you planning to release that FF/TB 9 bundle for F-15?
If so, please replace the versions of nss and nss-softoken that you have with nss-softokn-3.13.1-15.fc15 and nss-3.13.1-10.fc15. If not, I'll then release nspr/nss-util/nss-sotokn/nss by myself.
Comment 19 Stefan Becker 2012-12-18 14:06:37 EST
*** Bug 888074 has been marked as a duplicate of this bug. ***
Comment 20 Stefan Becker 2012-12-31 04:03:42 EST
*** Bug 890931 has been marked as a duplicate of this bug. ***
Comment 21 Matt Moldvan 2013-05-13 16:02:41 EDT
Just upgraded to Fedora 19 alpha and I'm getting the read error, unless I set the environment variable above... is this a regression in newer versions of the package(s)?

[mmoldva@sysl001t Downloads] rpm -qa | grep ^nss.*x86_64 | sort
nss-3.14.3-12.0.fc19.x86_64
nss_compat_ossl-0.9.6-5.fc19.x86_64
nss-mdns-0.10-12.fc19.x86_64
nss-pam-ldapd-0.8.12-4.fc19.x86_64
nss-softokn-3.14.3-1.fc19.x86_64
nss-softokn-freebl-3.14.3-1.fc19.x86_64
nss-sysinit-3.14.3-12.0.fc19.x86_64
nss-tools-3.14.3-12.0.fc19.x86_64
nss-util-3.14.3-1.fc19.x86_64
Comment 22 Matt Moldvan 2013-05-13 16:15:05 EDT
Just saw the comments by Stefan on the other bug reports, so not sure if my post above is still relevant...

FWIW, we're using MS Lync 2010, not sure of the patch level though.
Comment 23 Elio Maldonado Batiz 2013-05-13 20:25:20 EDT
I keep it more secure on Rawhide and more compatible on stables branches. Now that F-19 has branched and more than that its a beta candidate I have to tread it treat it like a stable branch. New build coming soon.
Comment 24 Fedora Update System 2013-05-13 21:58:49 EDT
nss-3.14.3-13.0.fc19 has been submitted as an update for Fedora 19.
https://admin.fedoraproject.org/updates/nss-3.14.3-13.0.fc19
Comment 25 Fedora Update System 2013-05-24 16:27:03 EDT
nss-3.14.3-13.0.fc19 has been pushed to the Fedora 19 stable repository.  If problems still persist, please make note of it in this bug report.
Comment 26 Harry Sutton 2013-08-12 12:35:39 EDT
I'm still seeing this problem on fully patched Fedora 19 (nss-3.15.1-3.fc19) with no manual change to NSS_SSL_CBC_RANDOM_IV; forcing that parameter to 0 results in the same behavior (error message in the pidgin debug log reads "GLib: g_has_table_destroy: assertion `hash_table != NULL' failed"

Same behavior is seen on fully patched RHEL 6.4 workstation (nss-3.14.3-4.el6_4), with and without changing the NSS environmental variable.
Comment 27 David Woodhouse 2013-12-19 18:12:08 EST
Pidgin in F20 broke again, unless I export NSS_SSL_CBC_RANDOM_IV=0.
Comment 28 Stefan Becker 2013-12-19 22:40:10 EST
(In reply to David Woodhouse from comment #27)
> Pidgin in F20 broke again, unless I export NSS_SSL_CBC_RANDOM_IV=0.

This is a FESCo decision for F20, see bug 1020420. Closing.
Comment 29 Kai Engert (:kaie) 2014-01-07 11:59:59 EST
*** Bug 1020424 has been marked as a duplicate of this bug. ***
Comment 30 David Woodhouse 2014-01-27 05:14:01 EST
(In reply to Stefan Becker from comment #28)
> (In reply to David Woodhouse from comment #27)
> > Pidgin in F20 broke again, unless I export NSS_SSL_CBC_RANDOM_IV=0.
> 
> This is a FESCo decision for F20, see bug 1020420. Closing.

Why close it? Why not leave it open and refile it against pidgin-sipe?

My pidgin-sipe users are now left with a non-functional piece of software. It does not connect to the Microsoft Lync server, which was really the only point of pidgin-sipe from their point of view.

What do I tell them? That they need to wait for Microsoft to fix the server? Or is there a fix already, that we need to chase the IT staff to install?

Working around this in pidgin or pidgin-sipe would certainly seem to be advisable in the short term.

Reopening the bug for now, at least that should help people find this instead of filing duplicates when they find it's broken for them in F20.
Comment 31 Stefan Becker 2014-01-27 06:02:49 EST
(In reply to David Woodhouse from comment #30)
> What do I tell them? That they need to wait for Microsoft to fix the server?

Yes.


> Or is there a fix already, that we need to chase the IT staff to install?

I don't know. I tried several times to find a M$ KB article, but was unable to do so. M$ might simply tell you to update to Lync 2013 or Office365 instead of providing a fix for Lync 2010, though.


> Working around this in pidgin or pidgin-sipe would certainly seem to be
> advisable in the short term.

This problem can't be fixed in pidgin-sipe code, because it does handle SSL. It just tells the backend to open a SSL connection to host:port.

AFAIR NSS does not offer an API to disable SSL BEAST mitigation per connection, like CDSA does on Mac OS X 10.9. Therefore the Adium solution won't work for pidgin, even if we ignore that pidgin 2.x is dead and nobody knows when pidgin 3.x will move to production status...


> Reopening the bug for now, at least that should help people find this
> instead of filing duplicates when they find it's broken for them in F20.

Closing this bug again, as F20 change is already mentioned on SIPE FAQ.
Comment 32 David Woodhouse 2014-01-27 06:54:09 EST
(In reply to Stefan Becker from comment #31)
> AFAIR NSS does not offer an API to disable SSL BEAST mitigation per
> connection, like CDSA does on Mac OS X 10.9.

Hm, if it's necessary for interworking with certain known-broken servers (yay Microsoft!) then that seems like a fairly serious omission.
Comment 33 Stefan Becker 2014-04-11 06:09:17 EDT
*** Bug 1086668 has been marked as a duplicate of this bug. ***