Bug 234409 - rhnreg_ks fails on Xeon x86_64 system running RHEL-RT (2.6.20) kernel
Summary: rhnreg_ks fails on Xeon x86_64 system running RHEL-RT (2.6.20) kernel
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Enterprise Linux 5
Classification: Red Hat
Component: rhn-client-tools
Version: 5.0
Hardware: x86_64
OS: Linux
medium
medium
Target Milestone: ---
: ---
Assignee: Pradeep Kilambi
QA Contact: Preethi Thomas
URL:
Whiteboard:
Depends On:
Blocks: 230752
TreeView+ depends on / blocked
 
Reported: 2007-03-28 22:17 UTC by Clark Williams
Modified: 2007-11-30 22:07 UTC (History)
2 users (show)

Fixed In Version: RHBA-2007-0591
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2007-11-07 17:57:25 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)
emailed traceback from Satellite for failed registration (78.15 KB, text/plain)
2007-06-06 20:51 UTC, Clark Williams
no flags Details


Links
System ID Private Priority Status Summary Last Updated
Red Hat Product Errata RHBA-2007:0591 0 normal SHIPPED_LIVE rhn-client-tools bug fix update 2007-10-30 18:58:35 UTC

Description Clark Williams 2007-03-28 22:17:37 UTC
Description of problem:

Registering using rhnreg_ks while running a RHEL-RT kernel (2.6.20) on an Intel
Xeon x86_64 system fails (probably due to new processor flags in /proc/cpuinfo;
Flags ssse3 and dca have been added). 

RHN Satellite log message:
SQL Error generated: (1401, 'ORA-01401: inserted value too large for column\n',
'insert into rhnCPU ( id, cpu_arch_id, server_id, nrcpu, vendor, family, cache,
bogomips, version, mhz, stepping, flags, model ) values ( :p0, :p_cpu_arch_id, :
p_server_id, :p_nrcpu, :p_vendor, :p_family, :p_cache, :p_bogomips, :p_version,
:p_mhz, :p_stepping, :p_flags, :p_model )')

Version-Release number of selected component (if applicable):

$ /usr/sbin/rhnreg_ks --version
rhnreg_ks (Red Hat Network Client Tools) 0.4.13-1.el5

How reproducible:
Always

Steps to Reproduce:
1. Boot RHEL-RT kernel on Xeon x86_64 system
2. Attempt to register with rhn (with valid activationkey)
3. Watch it fail
  
Actual results:
output in /var/log/up2date:
[Wed Mar 28 12:35:28 2007] up2date A protocol error occurred: Internal Server
Error , attempt #1,
[Wed Mar 28 12:35:33 2007] up2date A protocol error occurred: Internal Server
Error , attempt #2,
[Wed Mar 28 12:35:38 2007] up2date A protocol error occurred: Internal Server
Error , attempt #3,
[Wed Mar 28 12:35:44 2007] up2date A protocol error occurred: Internal Server
Error , attempt #4,
[Wed Mar 28 12:35:49 2007] up2date A protocol error occurred: Internal Server
Error , attempt #5,
[Wed Mar 28 12:35:49 2007] up2date Error communicating with server. The message was:
Internal Server Error
[Wed Mar 28 12:35:49 2007] up2date
Traceback (most recent call last):
  File "/usr/sbin/rhnreg_ks", line 254, in ?
    cli.run()
  File "/usr/share/rhn/up2date_client/rhncli.py", line 65, in run
    sys.exit(self.main() or 0)
  File "/usr/sbin/rhnreg_ks", line 139, in main
    rhnreg.sendHardware(systemId, hardwareList)
  File "/usr/share/rhn/up2date_client/rhnreg.py", line 560, in sendHardware
    s.registration.add_hw_profile(systemId, hardwareList)
  File "/usr/share/rhn/up2date_client/rhnserver.py", line 50, in __call__
    return rpcServer.doCall(method, *args, **kwargs)
  File "/usr/share/rhn/up2date_client/rpcServer.py", line 263, in doCall
    raise up2dateErrors.CommunicationError(e.errmsg)
up2date_client.up2dateErrors.CommunicationError: Error communicating with
server. The message was:
Internal Server Error


Expected results:
Successful registration

Additional info:
Rebooted into stock RHEL5 kernel and successfully registered system. 

I'd expect this to happen on any system where the kernel enables ssse3 or dca
for the processors.

Comment 2 Pradeep Kilambi 2007-06-05 14:23:47 UTC
As i request earlier, Could you please attach the server side traceback to this
bug. That would be very helpful. Also could you guys give us access to specific
hardware to test this bug on with relevant kernel?

Thanks
Prad

Comment 3 Pradeep Kilambi 2007-06-05 15:27:12 UTC
reading through the issue:

We can fix this on the client itself by truncating the data to the appropriate
length so that we don't get a "inserted value too large for column" error or we
can  fix the column length in the db. This will involve server side change. But
to determine this we would need the server side traceback/logs to see on which
column it is breaking while inserting into the db.

- Prad.

Comment 4 Clark Williams 2007-06-05 21:24:46 UTC
I think I'll have to try and recreate this, since I can't seem to find the "SQL
Error generated" message in any of the /var/log/rhn files on the satellite. 


Comment 5 Clark Williams 2007-06-05 22:04:22 UTC
Ok, I recreated it. I would guess that the :p_flags column is the culprit.

From /var/log/httpd/error.log:

Exception reported from rhn.farm.hsv.redhat.com
Time: Tue Jun  5 14:18:00 2007
Exception type server.rhnSQL.sql_base.SQLError
Exception while handling function registration.add_hw_profile
Request object information:
URI: /XMLRPC
Remote Host: 192.168.40.87
Server Name: rhn.farm.hsv.redhat.com:80
Headers passed in:
        Accept-Encoding: identity
        Content-Length: 31130
        Host: rhn.farm.hsv.redhat.com
        content-type: text/xml
        user-agent: rhn.rpclib.py/$Revision: 102540 $
        x-client-version: 1
        x-info: RPC Processor (C) Red Hat, Inc (version 102540)
        x-rhn-client-capability:
packages.verifyAll(1)=1,caneatCheese(1)=1,packages.extended_profile(1)=1,reboot.reboot(1)=1,packages.verify(1)=1,packages.runTransaction(1)=1,packages.rollBack(1)=1
        x-rhn-transport-capability: follow-redirects=2
        x-transport-info: Extended Capabilities Transport (C) Red Hat, Inc
(version 102540)
        x-up2date-version: 0.4.13-1.el5
Extra information about this error:
SQL Error generated: (1401, 'ORA-01401: inserted value too large for column\n',
'insert into rhnCPU ( id, cpu_arch_id, server_id, nrcpu, vendor, family, cache,
bogomips, version, mhz, stepping, flags, model ) values ( :p0, :p_cpu_arch_id,
:p_server_id, :p_nrcpu, :p_vendor, :p_family, :p_cache, :p_bogomips, :p_version,
:p_mhz, :p_stepping, :p_flags, :p_model )')

Exception Handler Information
Traceback (most recent call last):
  File "/usr/share/rhn/server/apacheRequest.py", line 108, in call_function
    response = apply(func, params)
  File "/usr/share/rhn/server/handlers/xmlrpc/registration.py", line 471, in
add_hw_profile
    server.save_hardware()
  File "/usr/share/rhn/server/rhnServer/server_wrapper.py", line 81, in
save_hardware
    ret = self.save_hardware_byid(self.server["id"])
  File "/usr/share/rhn/server/rhnServer/server_hardware.py", line 662, in
save_hardware_byid
    hw.save(sysid)
  File "/usr/share/rhn/server/rhnServer/server_hardware.py", line 192, in save
    t[devid] = self.data
  File "/usr/share/rhn/server/rhnSQL/sql_table.py", line 179, in __setitem__
    apply(h.execute, (), pdict)
  File "/usr/share/rhn/server/rhnSQL/int_oracle.py", line 100, in execute
    return apply(self._execute_wrapper, (self._execute, ) + p, kw)
  File "/usr/share/rhn/server/rhnSQL/int_oracle.py", line 132, in _execute_wrapper
    raise apply(sql_base.SQLError, ret)
SQLError: (1401, 'ORA-01401: inserted value too large for column\n', 'insert
into rhnCPU ( id, cpu_arch_id, server_id, nrcpu, vendor, family, cache,
bogomips, version, mhz, stepping, flags, model ) values ( :p0, :p_cpu_arch_id,
:p_server_id, :p_nrcpu, :p_vendor, :p_family, :p_cache, :p_bogomips, :p_version,
:p_mhz, :p_stepping, :p_flags, :p_model )')


Comment 6 Pradeep Kilambi 2007-06-06 14:43:14 UTC
as per the sat db, columns have the following lengths:
 
        bogomips        varchar(16),
        cache           varchar(16),
        family          varchar(32),
        MHz             varchar(16),
        stepping        varchar(16),
        flags           varchar(512),
        model           varchar(64),
        version         varchar(32),
        vendor          varchar(32),
        nrcpu           number default 1,
        acpiVersion     varchar(64),
        apic            varchar(32),
        apmVersion      varchar(32),
        chipset         varchar(64),

could you put some debug statements to see which one of these fields is
exceeding this length(or confirm if :p_flags is the culprit). Unfortunately as I
mentioned earlier we don't have relevant hardware/kernel to reproduce this. If
you could give me access to one of your boxes where you reproduced this that
would work for me too or If you can get me this info we can move ahead with
appropriate fix. Thanks much..

Regards
Prad.



Comment 7 Pradeep Kilambi 2007-06-06 15:10:24 UTC
to give you little more info on the client..if you look at /usr/bin/rhnreg_ks

there is a call (line 139 on mine)

if not self.options.nohardware:
    rhnreg.sendHardware(systemId, hardwareList)

in /usr/share/rhn/up2date_client/rhnreg.py , line 583:

def sendHardware(systemId, hardwareList):
    s = rhnserver.RhnServer()
    s.registration.add_hw_profile(systemId, hardwareList)

thats the one that sends in the hardware profile back to the server. So we could
just put a debug statement before that to see what info its sending to the
server and which field in that is exceeding the limit. 

Comment 8 Clark Williams 2007-06-06 20:51:26 UTC
Created attachment 156390 [details]
emailed traceback from Satellite for failed registration

Found this traceback email from the Satellite from when I duplicated the
registration failure yesterday. Looks like it might have enough information
about the failure but if it doesn't I'll put some debug statements in the
python code on the satellite.

Comment 9 Pradeep Kilambi 2007-06-08 14:02:06 UTC
 Could you jus try the following for me :(feel free to back up your db if you
want, but just a minor change).

on your sat, open up your satdb in sqlplus or something and just run:

ALTER TABLE RHNCPU MODIFY(FLAGS VARCHAR2(2048 BYTE));

then commit;

then run your rhnreg_ks from client. See if it still errors out. If this works
probably we could align this to sat-500 release and increate the flags column in
our sat db. As per hosted I see that we already raised flags to 2048. So
probably we might jus want to do the same.

Comment 10 Pradeep Kilambi 2007-06-08 14:37:13 UTC
fixed on sat...
Transmitting file data ..
Committed revision 117288.

Comment 12 Preethi Thomas 2007-08-08 15:46:09 UTC
verified.
flags has been increased to  FLAGS                                        
VARCHAR2(2048)


desc RHNCPU;                                                                   
            
 Name                                      Null?    Type
 ----------------------------------------- -------- ----------------------------
 ID                                        NOT NULL NUMBER
 SERVER_ID                                 NOT NULL NUMBER
 CPU_ARCH_ID                               NOT NULL NUMBER
 BOGOMIPS                                           VARCHAR2(16)
 CACHE                                              VARCHAR2(16)
 FAMILY                                             VARCHAR2(32)
 MHZ                                                VARCHAR2(16)
 STEPPING                                           VARCHAR2(16)
 FLAGS                                              VARCHAR2(2048)
 MODEL                                              VARCHAR2(64)
 VERSION                                            VARCHAR2(32)
 VENDOR                                             VARCHAR2(32)
 NRCPU                                              NUMBER
 ACPIVERSION                                        VARCHAR2(64)
 APIC                                               VARCHAR2(32)
 APMVERSION                                         VARCHAR2(32)
 CHIPSET                                            VARCHAR2(64)
 CREATED                                   NOT NULL DATE
 MODIFIED                                  NOT NULL DATE

SQL> select ID, server_id, flags, model, vendor, created from RHNCPU;            

        ID  SERVER_ID
---------- ----------
FLAGS
--------------------------------------------------------------------------------
MODEL
----------------------------------------------------------------
VENDOR                           CREATED
-------------------------------- ---------
        12 1000010011
fpu tsc msr pae mce cx8 apic mtrr mca cmov pat pse36 clflush dts acpi mmx fxsr s
se sse2 ss ht tm pbe cid xtpr
Intel(R) Xeon(TM) CPU 2.40GHz
GenuineIntel      

Comment 14 errata-xmlrpc 2007-11-07 17:57:25 UTC
An advisory has been issued which should help the problem
described in this bug report. This report is therefore being
closed with a resolution of ERRATA. For more information
on the solution and/or where to find the updated files,
please follow the link below. You may reopen this bug report
if the solution does not work for you.

http://rhn.redhat.com/errata/RHBA-2007-0591.html



Note You need to log in before you can comment on or make changes to this bug.