Bug 480903

Summary: "IOError: [Errno 22] Invalid argument" when syncing from export on NFS
Product: Red Hat Satellite 5 Reporter: Jan Hutař <jhutar>
Component: Satellite SynchronizationAssignee: Pradeep Kilambi <pkilambi>
Status: CLOSED CURRENTRELEASE QA Contact: Jan Hutař <jhutar>
Severity: high Docs Contact:
Priority: high    
Version: 530CC: bperkins, cperry, dherrman, gbock, gkhachik, nicolas.raspail, tao, xdmoon
Target Milestone: ---   
Target Release: ---   
Hardware: All   
OS: Linux   
Whiteboard:
Fixed In Version: sat530 Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
: 501060 502665 (view as bug list) Environment:
Last Closed: 2009-09-10 18:14:12 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 456985, 486216, 501060, 502665    
Attachments:
Description Flags
email by satellite: RHN TRACEBACK from dell-pe1950-01.rhts.englab.brq.redhat.com none

Description Jan Hutař 2009-01-21 09:33:24 UTC
Description of problem:
When I try to sync from an RO export on NFS, I get the traceback (bellow).


Version-Release number of selected component (if applicable):
spacewalk-backend-tools-0.4.9-1.el5
Satellite-5.3.0-RHEL5-re20081223.1/x86_64


How reproducible:
always


Steps to Reproduce:
1. # satellite-sync --mount-point=/mnt/exports  -c rhel-i386-server-5


Actual results:
...
03:50:12 Retrieving short package metadata (used for indexing)
03:50:12    Retrieving / parsing short package metadata: rhel-i386-server-5 (4462)
03:50:20 Diffing package metadata (what's missing locally?): rhel-i386-server-5
            ________________________________________
Diffing:    +++ sending log as an email +++
 
SYNC ERROR: unhandled exception occurred:
 
(Check logs/email for potentially more detail)
 
[Errno 22] Invalid argument


Expected results:
sync works


Additional info:
# mount | grep /mnt/exports
hostel.englab.brq.redhat.com:/mnt/hostel/export/sputnik-prod on /mnt/exports type nfs (rw,noatime,nodiratime,intr,soft,addr=10.34.32.71)
# service nfslock status
rpc.statd (pid 2617) is running...

Content of the email I obtain:
03:44:36 
SYNC ERROR: unhandled exception occurred:
 
Exception reported from dell-pe1950-01.rhts.englab.brq.redhat.com
Time: Wed Jan 21 03:44:36 2009
Exception type exceptions.IOError
 
Exception Handler Information
Traceback (most recent call last):
  File "/usr/bin/satellite-sync", line 142, in main
    return satsync.Runner().main()
  File "/usr/share/rhn/satellite_tools/satsync.py", line 206, in main
    ret = method()
  File "/usr/share/rhn/satellite_tools/satsync.py", line 293, in _step_short
    return self.syncer.processShortPackages()
  File "/usr/share/rhn/satellite_tools/satsync.py", line 825, in processShortPackages
    self._diff_packages()
  File "/usr/share/rhn/satellite_tools/satsync.py", line 861, in _diff_packages
    package = package_collection.get_package(pid, p_timestamp)
  File "/usr/share/rhn/satellite_tools/sync_handlers.py", line 245, in get_package
    return self._cache.cache_get(package_id, timestamp=timestamp)
  File "/usr/share/rhn/satellite_tools/syncCache.py", line 41, in cache_get
    compressed=self._compressed)
  File "/usr/share/rhn/common/rhnCache.py", line 71, in get
    return cache.get(name, modified)
  File "/usr/share/rhn/common/rhnCache.py", line 363, in get
    return self.cache.get(name, modified)
  File "/usr/share/rhn/common/rhnCache.py", line 334, in get
    pickled = self.cache.get(name, modified)
  File "/usr/share/rhn/common/rhnCache.py", line 232, in get
    fd.close()
  File "/usr/share/rhn/common/rhnCache.py", line 175, in close
    _unlock(self.fd.fileno())
  File "/usr/share/rhn/common/rhnCache.py", line 59, in _unlock
    fcntl.fcntl(fd, rhn_fcntl.F_SETLKW, UNLOCK)
IOError: [Errno 22] Invalid argument

Comment 1 Jan Hutař 2009-01-21 10:02:09 UTC
Created attachment 329576 [details]
email by satellite: RHN TRACEBACK from dell-pe1950-01.rhts.englab.brq.redhat.com

Comment 2 Clifford Perry 2009-01-22 20:48:29 UTC
https://bugzilla.redhat.com/show_bug.cgi?id=480047

Not quite certain if I agree it is NOT A BUG ...

Comment 8 Pradeep Kilambi 2009-02-19 15:33:44 UTC
fixed!

very bizarre issue. flock is more forgiving in such cases looks like. So I decided to try fcntl and if it raises an IOError switch to flock.

commit 6e026c1a371ad809046f1720b684cc6ef81100dc

Comment 9 Jan Hutař 2009-02-24 21:54:20 UTC
Thanks, just updated satellite packages to Satellite-5.3.0-RHEL5-re20090220.1 embedded DB, restarted satellite and sync runs nicely!

Comment 10 Miroslav Suchý 2009-03-13 08:29:48 UTC
*** Bug 490054 has been marked as a duplicate of this bug. ***

Comment 18 Nicolas Raspail 2009-05-21 17:14:35 UTC
Hi

I'am a running the following configuration :

* two RHEL 5.3 x86_64 machines
* Satellite 5.2
* a shared disk on two EMC Cx4 managed by Veritas Volume Manager with a vxfs volume on top
* Veritas Cluster Suite controlling the fail over of the Satellite

We have also two Proxy running on RHEL 5.3 x86_64 with Satellite Proxy 5.2

I have tried with two clients, one running RHEL 4U6 and one running RHEL5.1, and I'm facing the same problem of the infamous IOError: [Errno 22] Invalid argument.

If I make a up2date -i or a yum install or yum check-update, I have the error on my proxy

Extra information about this error:
Unhandled exception type

Exception Handler Information
Traceback (most recent call last):
  File "/usr/share/rhn/proxy/apacheServer.py", line 55, in __call__
    ret = f(req)
  File "/usr/share/rhn/proxy/apacheHandler.py", line 337, in handler
    ret = handlerObj.handler()
  File "/usr/share/rhn/proxy/broker/rhnBroker.py", line 156, in handler
    getResult = self.__local_GET_handler(self.req)
  File "/usr/share/rhn/proxy/broker/rhnBroker.py", line 296, in __local_GET_handler
    self.__checkAuthSessionTokenCache(token, reqchannel)
  File "/usr/share/rhn/proxy/broker/rhnBroker.py", line 445, in __checkAuthSessionTokenCache
    self.cachedClientInfo = UserDictCase(shelf[self.clientServerId])
  File "/usr/share/rhn/proxy/broker/rhnProxyAuth.py", line 375, in __getitem__
    val = rhnCache.get(rkey, missing_is_null=0)
  File "/usr/share/rhn/common/rhnCache.py", line 62, in get
    return cache.get(name, modified)
  File "/usr/share/rhn/common/rhnCache.py", line 325, in get
    pickled = self.cache.get(name, modified)
  File "/usr/share/rhn/common/rhnCache.py", line 223, in get
    fd.close()
  File "/usr/share/rhn/common/rhnCache.py", line 166, in close
    _unlock(self.fd.fileno())
  File "/usr/share/rhn/common/rhnCache.py", line 50, in _unlock
    fcntl.fcntl(fd, rhn_fcntl.F_SETLKW, UNLOCK)
IOError: [Errno 22] Invalid argument

and on the clients

up2date -i firefox

Fetching Obsoletes list for channel: rhel-x86_64-as-4...

Fetching Obsoletes list for channel: bnp-rhel-x86_64-as-4...

Fetching Obsoletes list for channel: rhn-tools-rhel-4-as-x86_64...

Fetching rpm headers...
########################################

Name                                    Version        Rel     
----------------------------------------------------------
firefox                                 3.0.10         1.el4             x86_64
No advisory information available



Testing package set / solving RPM inter-dependencies...
There was a fatal error communicating with the server. The message was:
Connection timed out on readline


yum check-update
Loaded plugins: rhnplugin, security
bnp-rhel-x86_64-server-5  100% |=========================| 1.2 kB    00:00     
primary.xml.gz            100% |=========================|  38 kB    00:00     
bnp-rhel-x86_64-server-5                                       138/138
rhel-x86_64-server-5      100% |=========================| 1.4 kB    00:00     
Error: failed to retrieve repodata/primary.xml.gz from rhel-x86_64-server-5
error was [Errno 14] HTTP Error 500: Internal Server Error

I wrote this to let you know that this error occurs on other situation, not just when syncing from NFS.

If you want, I can open an another bug

Regards

Nicolas

Comment 19 Pradeep Kilambi 2009-05-21 17:57:08 UTC
Hello Nicolas:

This issue will be resolved in satellite-5.3. We do however have HotFix packages available if needed for 5.2 satellites addressing this very issue. Please contact Red Hat Support and file a Hotfix request if you need this for 5.2.

Regards,
~ Prad

Comment 20 Nicolas Raspail 2009-05-22 05:45:10 UTC
Hi Pradeep

thanks for the tip about the hotfix. As our proxy are unusable in their current state, I will contact our TAM to get the hotfix

Regards

Nicolas

Comment 24 Brandon Perkins 2009-09-10 18:14:12 UTC
An advisory has been issued which should help the problem
described in this bug report. This report is therefore being
closed with a resolution of ERRATA. For more information
on therefore solution and/or where to find the updated files,
please follow the link below. You may reopen this bug report
if the solution does not work for you.

http://rhn.redhat.com/errata/RHEA-2009-1434.html