Bug 1325497

Summary: hammer import content-host fails with ISE - Content View and Environment not set for registration.
Product: Red Hat Satellite Reporter: Lukas Pramuk <lpramuk>
Component: UncategorizedAssignee: Tom McKay <tomckay>
Status: CLOSED ERRATA QA Contact: Lukas Pramuk <lpramuk>
Severity: high Docs Contact:
Priority: unspecified    
Version: 6.2.0CC: bbuckingham, bkearney, ehelms, tomckay
Target Milestone: UnspecifiedKeywords: Regression, Triaged
Target Release: Unused   
Hardware: Unspecified   
OS: Unspecified   
URL: http://projects.theforeman.org/issues/14615
Whiteboard:
Fixed In Version: rubygem-hammer_cli_import-0.11.2-1 Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2016-07-27 11:34:38 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Lukas Pramuk 2016-04-09 09:31:30 UTC
Description of problem:
hammer import content-host fails with ISE - Content View and Environment not set for registration. No content-host are being created.

Version-Release number of selected component (if applicable):
@Sat6.2.0-Beta-Snap6.2
tfm-rubygem-hammer_cli_import-0.10.23-1.el7sat.noarch

How reproducible:
always

Steps to Reproduce:
1. # cat users.csv 
organization_id,organization,user_id,username,last_name,first_name,position,email,role,creation_time,last_login_time,active
1,SatelliteQE,1,admin,Admin,Admin,,root@localhost,Satellite Administrator,2014-09-10 19:39:59,2016-04-07 01:00:14,enabled

2. # hammer import organization --verbose --csv-file users.csv 
Importing from users.csv
Creating new organization: SatelliteQE
Summary
  Created 1 organization.

3. # cat system-profiles.csv 
server_id,profile_name,hostname,description,organization_id,organization_name,base_channel_id,base_channel_label,child_channel_id,child_channel_label,system_group_id,system_group,release,architecture,virtual_host,virtual_guest,is_virtualized,virt_type
1000010009,profile1,tyan-gt24-01.example.com,,1,Red Hat SatTeam QA,,,,,,,7Server,x86_64,,,No,

4. # hammer -d import content-host --verbose --csv-file system-profiles.csv --export-directory .
...
Importing from system-profiles.csv
Creating new system: profile1
[ INFO 2016-04-09 11:25:42 API] POST /katello/api/systems
[DEBUG 2016-04-09 11:25:42 API] Params: {
                   :name => "profile1",
            :description => "\nsat5_system_id: 1000010009",
                  :facts => {
             "release" => "7Server",
        "architecture" => "x86_64"
    },
                   :type => "system",
        :organization_id => 8,
        :content_view_id => nil,
    :host_collection_ids => []
}
[DEBUG 2016-04-09 11:25:42 API] Headers: {}
[DEBUG 2016-04-09 11:25:43 API] 500 Internal Server Error
{
    "displayMessage" => "Content View and Environment not set for registration.",
            "errors" => [
        [0] "Content View and Environment not set for registration."
    ]
}
Caught RestClient::InternalServerError:500 Internal Server Error while processing CSV line:
{"server_id"=>"1000010009", "profile_name"=>"profile1", "hostname"=>"tyan-gt24-01.example.com", "description"=>nil, "organization_id"=>"1", "base_channel_id"=>nil, "base_channel_label"=>nil, "child_channel_id"=>nil, "system_group_id"=>nil, "release"=>"7Server", "architecture"=>"x86_64", "virtual_host"=>nil, "virtual_guest"=>nil}
Summary
  No action taken.


Actual results:
ISE, no content-host is created

Expected results:
content-hosts are successfully created

Comment 3 Tom McKay 2016-04-13 14:48:15 UTC
Created redmine issue http://projects.theforeman.org/issues/14615 from this bug

Comment 4 Bryan Kearney 2016-04-13 16:12:54 UTC
Upstream bug component is Uncategorized

Comment 5 Bryan Kearney 2016-04-13 20:13:08 UTC
Upstream bug component is Uncategorized

Comment 7 Tom McKay 2016-06-06 14:06:32 UTC
*** Bug 1325337 has been marked as a duplicate of this bug. ***

Comment 8 Bryan Kearney 2016-07-08 14:13:17 UTC
Moving to POST since upstream bug http://projects.theforeman.org/issues/14615 has been closed

Comment 10 Lukas Pramuk 2016-07-14 16:25:49 UTC
With upstream changes host is created anyway but if it doesn't belong to systemgroup(=>hostcollection) or is unentitled then this ugly error is shown:
Caught ApipieBindings::MissingArgumentsError:ApipieBindings::MissingArgumentsError: id while processing CSV line:


Two issues we have: 

1) Caught ApipieBindings::MissingArgumentsError:ApipieBindings::MissingArgumentsError: id 
for a system(sat5) system-group (=>Host Collection) doesnt have to be defined, the same it is for host(sat6)

2) Caught ApipieBindings::MissingArgumentsError:ApipieBindings::MissingArgumentsError: content_view_id 
for a system(sat5) channels don't have to be defined (unentitled machine) but for host (sat6) Content View has to be defined => Default Organization View should be selected

Comment 11 Lukas Pramuk 2016-07-18 13:52:08 UTC
FailedQA.

wanted to mark as verified as issues of comment #10 are fixed but I just revealed that mapping file packed in transition rpm is completely broken = empty

@satellite-6.2.0-20.1.el7sat.noarch
tfm-rubygem-hammer_cli_import-0.11.0-1.el7sat.noarch

# cat /tmp/exports/system-profiles.csv
server_id,profile_name,hostname,description,organization_id,organization_name,base_channel_id,base_channel_label,child_channel_id,child_channel_label,system_group_id,system_group,release,architecture,virtual_host,virtual_guest,is_virtualized,virt_type
1000010009,profile1,<FQDN1>,,1,Red Hat SatTeam QA,,,,,,,7Server,x86_64,,,No,
1000010011,nec-em29,<FQDN2>,,1,Red Hat SatTeam QA,101,rhel-x86_64-server-6,None;105,None;rhn-tools-rhel-x86_64-server-6,,,6Server,x86_64,,,No,
1000010010,profile2,<FQDN3>,,2,Organization #2,101,rhel-x86_64-server-6,None;105,None;rhn-tools-rhel-x86_64-server-6,19,sysgrp-2,6Server,x86_64,,,No,

# hammer import content-host --verbose --csv-file /tmp/exports/system-profiles.csv --export-directory /root/content-hosts
Importing from /tmp/exports/system-profiles.csv
Creating new system: profile1
  Content view rhel-x86_64-server-6_15_21 already created, reusing.
Creating new system: nec-em29
  Content view rhel-x86_64-server-6_16_22 already created, reusing.
Creating new system: profile2

To build the system-profile-transition rpm, run:

        cd /root/content-hosts/SPECS && 
          rpmbuild -ba --define "_topdir /root/content-hosts" system-profile-transition-<SAT_FQDN>-1468848422-0.0.1.spec

Then find your system-profile-transition-hp-<SAT_FQDN>-1468848422 package
        in /root/content-hosts/RPMS/noarch/ directory.
Summary
  Created 3 systems.

>>> import succeeded even if a sytem was un-entitled or wasn't a member of a system-group (issues of comment #10 are fixed)

Comment 12 Lukas Pramuk 2016-07-18 13:59:38 UTC
But, 

1) Mapping file packed in transition rpm is completely broken = empty

tar -Oxvf content-hosts/SOURCES/system-profile-transition-<SAT_FQDN>-1468848422-0.0.1.tar.gz *.map
system-profile-transition-<SAT_FQDN>-1468848422-0.0.1/system-id_to_uuid-1468848422.map
system_id,uuid,org_id
,,1
,,1
,,2


2) the latter symptom of broken mapping is that --delete option says no systems were imported

# hammer import content-host --verbose --csv-file /tmp/exports/system-profiles.csv --delete
Deleting from /tmp/exports/system-profiles.csv
System with id 1000010009 wasn't imported. Skipping deletion.
System with id 1000010011 wasn't imported. Skipping deletion.
System with id 1000010010 wasn't imported. Skipping deletion.
Summary
  No action taken.


>>> thus failing

Comment 15 Lukas Pramuk 2016-07-21 11:15:53 UTC
VERIFIED.

@satellite-6.2.0-21.0.el7sat.noarch
tfm-rubygem-hammer_cli_import-0.11.2-1.el7sat.noarch

# cat > system-profiles.csv << EOF
server_id,profile_name,hostname,description,organization_id,organization_name,base_channel_id,base_channel_label,child_channel_id,child_channel_label,system_group_id,system_group,release,architecture,virtual_host,virtual_guest,is_virtualized,virt_type
1000010009,profile1,<FQDN1>,,1,Red Hat SatTeam QA,,,,,,,7Server,x86_64,,,No,
1000010011,nec-em29,<FQDN2>,,1,Red Hat SatTeam QA,101,rhel-x86_64-server-6,None;105,None;rhn-tools-rhel-x86_64-server-6,,,6Server,x86_64,,,No,
1000010010,profile2,<FQDN3>,,2,Organization #2,101,rhel-x86_64-server-6,None;105,None;rhn-tools-rhel-x86_64-server-6,19,sysgrp-2,6Server,x86_64,,,No,
EOF

# hammer import content-host --verbose --csv-file system-profiles.csv --export-directory /root/content-hosts
Importing from /tmp/exports/system-profiles.csv
Creating new system: profile1
  Content view rhel-x86_64-server-6_15_21 already created, reusing.
Creating new system: nec-em29
  Content view rhel-x86_64-server-6_16_22 already created, reusing.
Creating new system: profile2

To build the system-profile-transition rpm, run:

        cd /root/content-hosts/SPECS && 
          rpmbuild -ba --define "_topdir /root/content-hosts" system-profile-transition-<SAT_FQDN>-1469098884-0.0.1.spec

Then find your system-profile-transition-hp-<SAT_FQDN>-1469098884 package
        in /root/content-hosts/RPMS/noarch/ directory.
Summary
  Created 3 systems.

# tar -Oxf content-hosts/SOURCES/system-profile-transition-`hostname`-1469098884-0.0.1.tar.gz *.map
system_id,uuid,org_id
1000010009,f94dd82c-9656-4611-9a4e-1ed9e8d7ff7f,1
1000010010,5b883a01-90a7-4e19-a189-527438b79e9e,1
1000010011,67ca6222-263f-448c-8b4e-97338aaef3b1,1

>>> content-host import succeeded and created mapping file for sat5 to consume contains correct mappings for all imported systems/hosts

Comment 16 Lukas Pramuk 2016-07-21 11:19:02 UTC
oops small typo, system_id 1000010010 is in org 2
...
1000010010,5b883a01-90a7-4e19-a189-527438b79e9e,2 <
...

Comment 17 Bryan Kearney 2016-07-27 11:34:38 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2016:1501