Bug 873747 - Spacewalk 1.8 | Error after "spacewalk-schema-upgrade"
Summary: Spacewalk 1.8 | Error after "spacewalk-schema-upgrade"
Keywords:
Status: CLOSED NOTABUG
Alias: None
Product: Spacewalk
Classification: Community
Component: Installation
Version: 1.8
Hardware: x86_64
OS: Linux
unspecified
unspecified
Target Milestone: ---
Assignee: Jan Pazdziora
QA Contact: Red Hat Satellite QA List
URL:
Whiteboard:
Depends On:
Blocks: space27
TreeView+ depends on / blocked
 
Reported: 2012-11-06 15:51 UTC by Holger Schmitt
Modified: 2017-09-28 17:56 UTC (History)
2 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2012-11-14 11:24:38 UTC
Embargoed:


Attachments (Terms of Use)
Tracebacks (23.48 KB, text/plain)
2012-11-06 15:51 UTC, Holger Schmitt
no flags Details

Description Holger Schmitt 2012-11-06 15:51:10 UTC
Created attachment 639420 [details]
Tracebacks

Description of problem:
After spacewalk update and schema upgrade I get "Internal Server Error".


Version-Release number of selected component (if applicable):


How reproducible:
always


Steps to Reproduce:
Clicking on detail pages:
- https://myserver/rhn/systems/details/Overview.do?sid=1000010052
- https://myserver/rhn/channels/ChannelDetail.do?cid=108)
- https://myserver/rhn/channels/manage/Edit.do?cid=101
- ...

  
Actual results:
"Internal Server Error"


Expected results:
Show detail pages


Additional info:
For tracebacks see attachment.


Software-Versions:
[root@myserver schema-upgrade]# uname -a
Linux myserver.adm-group.com 2.6.32-279.9.1.el6.x86_64 #1 SMP Tue Sep 25 14:55:44 CDT 2012 x86_64 x86_64 x86_64 GNU/Linux


[root@myserver schema-upgrade]# rpm -qa|grep spacewalk|sort
spacewalk-admin-1.8.6-1.el6.noarch
spacewalk-backend-1.8.85-1.el6.noarch
spacewalk-backend-app-1.8.85-1.el6.noarch
spacewalk-backend-applet-1.8.85-1.el6.noarch
spacewalk-backend-config-files-1.8.85-1.el6.noarch
spacewalk-backend-config-files-common-1.8.85-1.el6.noarch
spacewalk-backend-config-files-tool-1.8.85-1.el6.noarch
spacewalk-backend-iss-1.8.85-1.el6.noarch
spacewalk-backend-iss-export-1.8.85-1.el6.noarch
spacewalk-backend-libs-1.9.2-1.el6.noarch
spacewalk-backend-package-push-server-1.8.85-1.el6.noarch
spacewalk-backend-server-1.8.85-1.el6.noarch
spacewalk-backend-sql-1.8.85-1.el6.noarch
spacewalk-backend-sql-postgresql-1.8.85-1.el6.noarch
spacewalk-backend-tools-1.8.85-1.el6.noarch
spacewalk-backend-xml-export-libs-1.8.85-1.el6.noarch
spacewalk-backend-xmlrpc-1.8.85-1.el6.noarch
spacewalk-base-1.8.49-1.el6.noarch
spacewalk-base-minimal-1.8.49-1.el6.noarch
spacewalk-branding-1.8.7-1.el6.noarch
spacewalk-certs-tools-1.9.1-1.el6.noarch
spacewalk-client-repo-1.9-1.el6.noarch
spacewalk-common-1.8.6-1.el6.noarch
spacewalk-config-1.8.6-1.el6.noarch
spacewalk-doc-indexes-1.1.1-1.el6.noarch
spacewalk-grail-1.8.49-1.el6.noarch
spacewalk-html-1.8.49-1.el6.noarch
spacewalk-java-1.8.180-1.el6.noarch
spacewalk-java-config-1.8.180-1.el6.noarch
spacewalk-java-lib-1.8.180-1.el6.noarch
spacewalk-java-postgresql-1.8.180-1.el6.noarch
spacewalk-jpp-workaround-1.0.4-1.el6.noarch
spacewalk-monitoring-1.4.1-1.el6.noarch
spacewalk-monitoring-selinux-1.8.4-1.el6.noarch
spacewalk-postgresql-1.8.6-1.el6.noarch
spacewalk-pxt-1.8.49-1.el6.noarch
spacewalk-repo-1.8-4.el6.noarch
spacewalk-schema-1.8.88-1.el6.noarch
spacewalk-search-1.8.6-1.el6.noarch
spacewalk-selinux-1.8.2-1.el6.noarch
spacewalk-setup-1.8.24-1.el6.noarch
spacewalk-setup-jabberd-1.8.7-1.el6.noarch
spacewalk-slf4j-1.6.1-1.el6.noarch
spacewalk-sniglets-1.8.49-1.el6.noarch
spacewalk-taskomatic-1.8.180-1.el6.noarch


root@myserver schema-upgrade]# spacewalk-schema-upgrade
Schema upgrade: [spacewalk-schema-1.8.88-1.el6] -> [spacewalk-schema-1.8.88-1.el6]
Your database schema already matches the schema package version [spacewalk-schema-1.8.88-1.el6].


[root@myserver schema-upgrade]# rpm -qa|grep postgres|sort
postgresql-8.4.13-1.el6_3.x86_64
postgresql-contrib-8.4.13-1.el6_3.x86_64
postgresql-jdbc-8.4.701-8.el6.noarch
postgresql-libs-8.4.13-1.el6_3.x86_64
postgresql-server-8.4.13-1.el6_3.x86_64
spacewalk-backend-sql-postgresql-1.8.85-1.el6.noarch
spacewalk-java-postgresql-1.8.180-1.el6.noarch
spacewalk-postgresql-1.8.6-1.el6.noarch


[root@myserver schema-upgrade]# rpm -qa|grep tomcat
apache-tomcat-apis-0.1-1.el6.noarch
tomcat5-jsp-2.0-api-5.5.27-7.jpp5.noarch
tomcat6-el-2.1-api-6.0.24-45.el6.noarch
tomcat6-servlet-2.5-api-6.0.24-45.el6.noarch
tomcat6-lib-6.0.24-45.el6.noarch
tomcat6-6.0.24-45.el6.noarch
tomcat6-jsp-2.1-api-6.0.24-45.el6.noarch
tomcat5-servlet-2.4-api-5.5.27-7.jpp5.noarch

Comment 1 Jan Pazdziora 2012-11-06 16:33:32 UTC
What Spacewalk version was this installation upgraded from?

Comment 2 Holger Schmitt 2012-11-06 16:43:04 UTC
(In reply to comment #1)
> What Spacewalk version was this installation upgraded from?

Always from the latest (http://spacewalk.redhat.com/yum/latest/RHEL/6/x86_64/)

Maybe there is a problem with the db schema. Is it possible to repair it, without losing data?

Comment 3 Jan Pazdziora 2012-11-06 17:20:42 UTC
(In reply to comment #2)
> (In reply to comment #1)
> > What Spacewalk version was this installation upgraded from?
> 
> Always from the latest
> (http://spacewalk.redhat.com/yum/latest/RHEL/6/x86_64/)
> 
> Maybe there is a problem with the db schema. Is it possible to repair it,
> without losing data?

Restoring your 1.7 schema from backup and re-runing the schema upgrade would be the correct way.

But we would also like to know what caused this -- what is in your schema upgrade logs, etc. The missing rhnabrtinfo table sounds like something non-trivially wrong happened there.

Comment 4 Holger Schmitt 2012-11-06 18:00:14 UTC
Here the recent files in "/var/log/spacewalk/schema-upgrade/"

Do you need any more files?


--------------------------------------------------------

[root@myserver ~]# cat /var/log/spacewalk/schema-upgrade/20121106-161051-script.sql 
update rhnVersionInfo
       set label = 'schema-from-' || to_char(created, 'YYYYMMDD-HH24MISS'),
           modified = current_timestamp
       where label = 'schema';
commit;
-- oracle equivalent source sha1 cb07e6926fb5f79d7645be270e632f9168631964
--
-- Copyright (c) 2010--2012 Red Hat, Inc.
--
-- This software is licensed to you under the GNU General Public License,
-- version 2 (GPLv2). There is NO WARRANTY for this software, express or
-- implied, including the implied warranties of MERCHANTABILITY or FITNESS
-- FOR A PARTICULAR PURPOSE. You should have received a copy of GPLv2
-- along with this software; if not, see
-- http://www.gnu.org/licenses/old-licenses/gpl-2.0.txt.
--
-- Red Hat trademarks are not licensed under GPLv2. No permission is
-- granted to use or replicate Red Hat trademarks that are incorporated
-- in this software or its documentation.
--

--
-- empty varchars are not allowed for the oracle-postgres compatibility
-- create constraints on all varchar columns (all tables for the current user)
-- and returns number of errors during processing (if not 0 then
-- check the pgsql log -- usually /var/lib/pgsql/data/pg_log -- for errors)
--
create or replace function create_varnull_constriants() returns integer as $$
declare
    tabs record;
    total integer default 0;
begin

    for tabs in select 
        c.relname as "tab",
        a.attname as "col"
    from
        pg_catalog.pg_attribute a
        left outer join pg_catalog.pg_class c on a.attrelid = c.oid 
    where
        -- skip system columns
        a.attnum > 0
        -- skip dropped columns
        and not a.attisdropped
        -- filter only varchars
        and a.atttypid = 1043
        -- skip cols that already has this constraint
        and not exists (
            select 1 from pg_catalog.pg_constraint 
            where conname = 'vn_' || c.relname || '_' || a.attname
        )
        -- filter only tables owned by current user
        and a.attrelid in (
            select c.oid from pg_catalog.pg_class c
            where relkind = 'r' and pg_catalog.pg_table_is_visible(c.oid) and relowner = (
                select oid from pg_catalog.pg_authid where rolname = current_user
            )
        ) loop
        begin
            -- create constraint
            execute 'alter table ' || tabs.tab || ' add constraint vn_' ||
                tabs.tab || '_' || tabs.col || ' check (' || tabs.col || ' <> '''')';
            -- count them
        exception when others then
            total = total + 1;
            raise warning '% unable to create constraint for %.%', now(), tabs.tab, tabs.col;
        end;
    end loop;

    return total;
end;
$$ language plpgsql;

select create_varnull_constriants();


insert into rhnVersionInfo
       ( label, name_id, evr_id, created, modified )
       values ('schema', lookup_package_name('spacewalk-schema'),
               lookup_evr(null, '1.8.88' , '1.el6' ),
               current_timestamp, current_timestamp );
commit;


--------------------------------------------------------


[root@myserver ~]# cat /var/log/spacewalk/schema-upgrade/20121106-161051-to-spacewalk-schema-1.8.log 
UPDATE 1
COMMIT
CREATE FUNCTION
 create_varnull_constriants 
----------------------------
                          0
(1 row)

INSERT 0 1
COMMIT

Comment 5 Jan Pazdziora 2012-11-06 18:46:36 UTC
(In reply to comment #4)
> Here the recent files in "/var/log/spacewalk/schema-upgrade/"
> 
> Do you need any more files?
> 
> 
> --------------------------------------------------------
> 
> [root@myserver ~]# cat
> /var/log/spacewalk/schema-upgrade/20121106-161051-script.sql 

Yes, the previous ones. This ones are from the run when the schema version was already 1.8.88-1.el6. Which is however strange.

Did you by any chance migrate your Spacewalk server installation from (say) EL5 to EL6?

Comment 6 Jan Pazdziora 2012-11-06 18:48:57 UTC
Could you please paste output of

select label, created, ( select evr from rhnpackageevr where evr_id = id ) as evr from rhnVersionInfo order by created ;

when run in your psql?

Comment 7 Jan Pazdziora 2012-11-10 16:23:19 UTC
Also, could you show a list of databases in your PostgreSQL?

Command

\l

in psql.

Comment 8 Holger Schmitt 2012-11-13 16:34:27 UTC
Meanwhile I set up a new 1.8 installation, because we need a working spacewalk server.

Here are the informations from the database, I backed up before. 



spaceschema=# \l
                                   List of databases
    Name     |  Owner   | Encoding |   Collate   |    Ctype    |   Access privileges   
-------------+----------+----------+-------------+-------------+-----------------------
 postgres    | postgres | UTF8     | en_US.UTF-8 | en_US.UTF-8 | 
 spaceschema | postgres | UTF8     | en_US.UTF-8 | en_US.UTF-8 | 
 template0   | postgres | UTF8     | en_US.UTF-8 | en_US.UTF-8 | =c/postgres          +
             |          |          |             |             | postgres=CTc/postgres
 template1   | postgres | UTF8     | en_US.UTF-8 | en_US.UTF-8 | =c/postgres          +
             |          |          |             |             | postgres=CTc/postgres
(4 rows)


spaceschema=# select label, created, ( select evr from rhnpackageevr where evr_id = id ) as evr from rhnVersionInfo order by created ;
            label            |            created            |       evr       
-----------------------------+-------------------------------+-----------------
 schema-from-20120524-175710 | 2012-05-24 17:57:10.916031+02 | (,1.8.39,1.el6)
 schema-from-20120525-095231 | 2012-05-25 09:52:31.403195+02 | (,1.8.40,1.el6)
 schema                      | 2012-11-06 16:11:18.546448+01 | (,1.8.88,1.el6)
(3 rows)

Comment 9 Jan Pazdziora 2012-11-14 11:24:38 UTC
(In reply to comment #8)
> Meanwhile I set up a new 1.8 installation, because we need a working
> spacewalk server.

And I assume it works fine.

> spaceschema=# select label, created, ( select evr from rhnpackageevr where
> evr_id = id ) as evr from rhnVersionInfo order by created ;
>             label            |            created            |       evr    
> 
> -----------------------------+-------------------------------+---------------
> --
>  schema-from-20120524-175710 | 2012-05-24 17:57:10.916031+02 |
> (,1.8.39,1.el6)
>  schema-from-20120525-095231 | 2012-05-25 09:52:31.403195+02 |
> (,1.8.40,1.el6)
>  schema                      | 2012-11-06 16:11:18.546448+01 |
> (,1.8.88,1.el6)
> (3 rows)

This means the original installation was a Spacewalk nightly.

We don't support upgrades from Spacewalk nightly to next released release.

Comment 10 Eric Herget 2017-09-28 17:56:14 UTC
This BZ closed some time during 2.5, 2.6 or 2.7.  Adding to 2.7 tracking bug.


Note You need to log in before you can comment on or make changes to this bug.