Bug 2982 - Oplock problem with Samba and glibc2.1
Summary: Oplock problem with Samba and glibc2.1
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: Red Hat Linux
Classification: Retired
Component: samba
Version: 6.0
Hardware: i386
OS: Linux
high
high
Target Milestone: ---
Assignee: Trond Eivind Glomsrxd
QA Contact:
URL:
Whiteboard:
: 3715 (view as bug list)
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 1999-05-23 13:48 UTC by sean
Modified: 2015-09-29 12:03 UTC (History)
1 user (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 1999-05-25 18:25:55 UTC
Embargoed:


Attachments (Terms of Use)

Description sean 1999-05-23 13:48:56 UTC
I recently installed my first production RH 6.0 system for a
client.  It's use was primarily a file/print server
replacement for an old Netware 3.12 box we had installed
years earlier.

Upon installing/testing their primary (network) application
I encountered problems.  Only one user could enter and use
the system at a time.  After many hours of hunting I have
traced it to some problem between glibc2.1 and Samba.

Symptoms include:
problems accessing files in a shared/locked manner
Log entries such as:
[1999/05/22 14:42:07, 3] lib/util.c:fcntl_lock(2658)
  fcntl lock gave errno 11 (Resource temporarily
unavailable)

Temporary fixes:

Fix #1:
in the [global] section set oplocks = false
This is the easiest but you might experience a severe
performance impact (I know my client wasn't happy with this
which is partly why I kept digging)

Fix #2:
install the compat-glibc-5.2-2.0.7.1 RPM
then LD_LIBRARY_PATH=/usr/i386-glibc20-linux/lib
Probably in the /etc/rc.d/init.d/smb script.
This seems to fix the problem and preserve the oplock
ability (and thus, the speed)

I marked this as high priority/high severity since Samba is,
at least from what I see, a major reason people use Linux.
You could theoretically get by without ever noticing this
problem if your uses of the server were minimal and two
people never tried accessing the same file at once but in
any serious server environment this could cause massive
failures and headaches (at the least) and maybe even severe
data loss (I'm not smart enough about oplocks, windows
filesharing/record locking, and samba to know if that is
true or not).

I suggest as a temporary solution releasing an initscripts
upgrade for all your users.  Of course, you don't need me to
tell you how to do your job *laugh*.

Congrats on an otherwise great product! :)

If you need more info please feel free to contact me.

Comment 1 sean 1999-05-23 13:59:59 UTC
I meant releasing an update to the samba scripts, not initscripts.
Duh.  Serves me right for trying to start work at 7:30 AM on a Sunday
:)

Comment 2 Bill Nottingham 1999-05-24 18:59:59 UTC
Actually, I'm suprised it would run with the compat libs; that's
not really a solution. The correct solution would be to find
out what the conflict is with glibc2.1, and fix that.

Comment 3 Bill Nottingham 1999-05-24 19:03:59 UTC
In fact, testing reveals that samba shouldn't even *run* with
the glibc2.0 compat libs. Are you sure you aren't running an
older/rebuilt samba version?  What happens if you shut down the
current version and run (from the command line)

LD_LIBRARY_PATH=/usr/i386-glibc20-linux/lib /usr/sbin/smbd

Comment 4 Bill Nottingham 1999-05-24 19:05:59 UTC
Also, are you sharing local files, or files that are mounted
from another server?

Comment 5 Bill Nottingham 1999-05-24 19:08:59 UTC
What file is it that's having locking problems?

Comment 6 sean 1999-05-24 19:11:59 UTC
Well... even while doing it I knew that using the compat libs was
definily not the "correct" solution, not any more then globally
setting oplocks=false.  But Samba ran, my application was happy so I
went with it.  I certainly don't have the knowledge of Samba or the
programming experience to have fixed or further diagnosed such a
problem (tho I hope to change that some day).  After having two weeks
with egg on my face I was desperate to give the client anything to
make things chug along.

All the files were on a standalone box, no network drive mounts.
There were a number of files that were having problems.  Part of the
Business Works accounting package from State of the Art software
(www.sota.com).  Business Works has been running flawlessly on my
office's various versions of Red Hat (currently 5.2) since we phased
Netware out about 3 years ago.  I privately received email from
another Samba on RH 6.0 user who was experiencing similar locking
problems with Word/Excel documents.  I can give you his email addy if
that would help.

I'm afraid that the testbed I was using has been disconnected so it
would be very hard at this point for me to try any additional
conditions at this time.

Comment 7 sean 1999-05-24 19:41:59 UTC
Well... even while doing it I knew that using the compat libs was
definily not the "correct" solution, not any more then globally
setting oplocks=false.  But Samba ran, my application was happy so I
went with it.  I certainly don't have the knowledge of Samba or the
programming experience to have fixed or further diagnosed such a
problem (tho I hope to change that some day).  After having two weeks
with egg on my face I was desperate to give the client anything to
make things chug along.

All the files were on a standalone box, no network drive mounts.
There were a number of files that were having problems.  Part of the
Business Works accounting package from State of the Art software
(www.sota.com).  Business Works has been running flawlessly on my
office's various versions of Red Hat (currently 5.2) since we phased
Netware out about 3 years ago.  I privately received email from
another Samba on RH 6.0 user who was experiencing similar locking
problems with Word/Excel documents.  I can give you his email addy if
that would help.

I'm afraid that the testbed I was using has been disconnected so it
would be very hard at this point for me to try any additional
conditions at this time.

Comment 8 Bill Nottingham 1999-05-24 20:45:59 UTC
That might help. So far, I'm noticing identical locking
behavior between 5.2 & 6.0 here (I can't reproduce it.)

Comment 9 sean 1999-05-24 20:56:59 UTC
I have the Samba logs at both level 3 and level 12 from the failed
tests on Saturday.  I can provide those if you would like (warning:
very large).

I could also maybe have the test network re-setup but it would take me
a day or two.  I am currently working out of home so moving hardware
around in the office would be difficult :)

Also, I have the email from the other person who was having problems.
I can forward that to you if you tell me where you would like it sent.

On the samba mailing list Jeremy Allison said he thinks there is a
problem with how the Samba autoconf picks a 32 vs 64 bit locking
compatibility option with glibc2.1.  I had neglected to mention it was
Windows 95 and not NT on that post and his reply specifically
mentioned NT having the bug.  Whether it would apply to 95/98 too or
not I don't know.

Comment 10 Bill Nottingham 1999-05-24 21:15:59 UTC
Well, there is this in the Samba 2.0.4 changelog:
(on new parameters)

mangle locks
------------

This parameter was added to get around a bug in Windows NT
when dealing with Samba running on 32-bit systems (such
as Linux x86). This bug causes NT to send 64 bit locking
requests to 32-bit systems even though Samba correctly
tells the NT client not to do so. This option causes Samba
to map the lock requests from 64 bits to 32 bits on these
systems.

If you could send the large stuff & the forwarded
e-mail to notting (not bugzilla), I'll
look at it.

Comment 11 Bill Nottingham 1999-05-25 18:25:59 UTC
fixed in samba-2.0.4b-2, which will be in the next
Raw Hide release.

Comment 12 Bill Nottingham 1999-06-24 22:05:59 UTC
*** Bug 3715 has been marked as a duplicate of this bug. ***

> To any one who is using Red Hat Linux 6.0 (or, presumably,
any glibc2.1
> system)
>
> I recently installed a new RH 6.0 system on Intel.  After
installing my
> samba and my network application I noticed that I was
experiencing oplock
> problems that were not present with the same version of
Samba on my RH 5.2
> systems and caused my application to not function.  Since
the problem
> seems to be with oplocks I don't know if this could cause
data
> corruption/inconsistency or not but I would assume so.  I
have tested this
> both with Samba 2.0.3 (that ships with RH) and also Samba
2.0.4b.  I have
> also tried compiling Samba fresh (as opposed to using the
binary RPMs).
>
> Evidence of the problem are log entries like the following
(gotten at
> debug level=3 and higher)
>
> [1999/05/22 14:42:06, 3] lib/util.c:fcntl_lock(2658)
>   fcntl lock gave errno 11 (Resource temporarily
unavailable)
>
> It seems that I only see "fcntl" when this error occurs so
that may be a
> safe expression to grep for if you want to look for it.  I
had quite a few
> of these lines.  Presumably my application trying to get
to it's files
> before it died a horrible death.
>
> There are two fixes I've found for this so far:
>
> 1 - set oplocks = false for the system.  The downside of
this is that with
> oplocks disabled you will have anywhere from a minor to a
massive
> performance hit.
>
> 2 - The better fix, IMHO, is to do the following
>
> rpm -Uvh
/path-to-redhat-6.0/compat-glibc-5.2-2.0.7.1.i386.rpm
>
> edit /etc/rc.d/init.d/smb
>
> In the "start)" case add
> LD_LIBRARY_PATH=/usr/i386-glibc20-linux/lib
> before the "daemon smbd -D" line
>
> and then add
> LD_LIBRARY_PATH=
> after the "daemon nmbd -D" line
>
> (note: I haven't tried the above in a script yet, just
manually with
> manually executing smbd/nmbd but I see no reason why it
would be any
> different in a script)
>
> As best I understand this, this points the dynamic linker
to look in the
> old glibc2.0 directory first to resolve objects.  In this
case it makes
> Samba run as if it were still on a glibc2.0 (ie: RH 5.2)
system and things
> work fine again.  It seems to run just fine like that and
you can leave
> oplocks enabled so your users/clients/self stop yelling at
you :)
>
> I don't know if this is a Samba bug for not following some
new glibc2.1
> API change or just a glibc2.1 bug.  But Samba is the only
application I've
> had problems with so far so I'm starting here.
>
> samba-bugs has been cc'd and I am also reporting this as a
bug to Red Hat.
> I consider this a pretty severe problem.
>
> If anyone has any questions please feel free to ask me.

Ok, I'm pretty sure what's going on here. I'm also CC:ingThe
problem is that WinNT has a bug in which it sends 64
bit lock range requests to systems that only have 32 bit
lock ranges available, despite being told via the protocol
not to do so (just another area they never tested I guess).

Samba is fully 64-bit clean on a 64 bit system and will
work fine on these systems. Unfortunately x86 linux is only
a 32 bit system. Samba 2.0.4 has a fix in it in that if it
detects it is running on a 32-bit system it will 'fold' the
64 bit lock range into 32 bits as best it can. It needs to
do this as applications like Word and Excel need byte range
locks to conflict correctly in order to detect multiple
opens
on a file. This works all well and good on glibc2.0 systems
(which Samba detects as 32 bit lock aware).

But glibc2.1 lies on a Linux x86 system :-). It tells Samba
that it can do 64 bit lock reqests. It can't. What glibc2.1
does is just drop the top 32 bits of any range request. This
causes the bottom half of the lock ranges to collide with
earlier lock ranges - hence the messages you are getting.

Now Samba has the code to work correctly with glibc2.1, it
just needs to stop believing that glibc2.1 is a 64 bit lock
aware library on a 32 bit system. Personally I think this is
a glibc2.1 bug on x86 Linux. As Samba already contains the
correct code (as evidenced by the fact it works with the
glibc2.0 libraries) we need to find a way for the Samba
autoconf scripts to detect lying libc libraries and turn
on the 32-bit workaround automatically on Linux x86
glibc2.1.

Regards,

        Jeremy Allison,
        Samba Team.

this to Ulich Drepper who is the maintainer of the glibc
library at Cygnus & also Erik Troan at RedHat.

Comment 13 openshift-github-bot 2015-09-29 12:03:43 UTC
Commit pushed to master at https://github.com/openshift/origin

https://github.com/openshift/origin/commit/0f4e2493764ad0a2930b6accb05dc75707d6abec
Merge pull request #4780 from rhcarvalho/issue2982-pushspec-in-build-status

Merged by openshift-bot


Note You need to log in before you can comment on or make changes to this bug.