Bug 1351483

Summary: Upgrade failed at step: update_subscription_facet_backend_data with error: Validation failed: Identifier Can't add or remove `.` from identifier
Product: Red Hat Satellite Reporter: Sachin Ghai <sghai>
Component: Subscription ManagementAssignee: Ivan Necas <inecas>
Status: CLOSED ERRATA QA Contact: Sachin Ghai <sghai>
Severity: high Docs Contact:
Priority: unspecified    
Version: 6.2.0CC: bbuckingham, inecas, sghai, zhunting
Target Milestone: UnspecifiedKeywords: Triaged
Target Release: Unused   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: rubygem-katello-3.0.0.57-1 Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2016-07-27 11:23:39 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
foreman-debug logs none

Description Sachin Ghai 2016-06-30 07:35:53 UTC
Description of problem:

While performing upgrade from sat6.1.9 -> sat6.2 GA snap18.1 on rhel7, installer failed at step: "update_subscription_facet_backend_data" with error:


==> /var/log/foreman-installer/satellite.log <==
[ERROR 2016-06-30 03:23:04 main] rake aborted!
ActiveRecord::RecordInvalid: Validation failed: Identifier Can't add or remove `.` from identifier
/opt/rh/rh-ror41/root/usr/share/gems/gems/activerecord-4.1.5/lib/active_record/validations.rb:57:in `save!'
/opt/rh/rh-ror41/root/usr/share/gems/gems/activerecord-4.1.5/lib/active_record/attribute_methods/dirty.rb:29:in `save!'
/opt/rh/rh-ror41/root/usr/share/gems/gems/activerecord-4.1.5/lib/active_record/transactions.rb:273:in `block in save!'
/opt/rh/rh-ror41/root/usr/share/gems/gems/activerecord-4.1.5/lib/active_record/transactions.rb:329:in `block in with_transaction_returning_status'
/opt/rh/rh-ror41/root/usr/share/gems/gems/activerecord-4.1.5/lib/active_record/connection_adapters/abstract/database_statements.rb:201:in `block in transaction'


Version-Release number of selected component (if applicable):
upgrade from sat6.1.9 -> sat6.2 GA snap18.1 on rhel7

How reproducible:


Steps to Reproduce:
1. set 62 snap18.1 repo
2. katello-service stop
3. yum update -y
4. satellite-installer --scenario satellite --upgrade

Actual results:
upgrade failed:

[ERROR 2016-06-30 03:23:04 main] rake aborted!
ActiveRecord::RecordInvalid: Validation failed: Identifier Can't add or remove `.` from identifier


Expected results:
upgrade should complete successfully

Additional info:==> 

/var/log/foreman-installer/satellite.log <==
[ERROR 2016-06-30 03:23:04 main] rake aborted!
ActiveRecord::RecordInvalid: Validation failed: Identifier Can't add or remove `.` from identifier
/opt/rh/rh-ror41/root/usr/share/gems/gems/activerecord-4.1.5/lib/active_record/validations.rb:57:in `save!'
/opt/rh/rh-ror41/root/usr/share/gems/gems/activerecord-4.1.5/lib/active_record/attribute_methods/dirty.rb:29:in `save!'
/opt/rh/rh-ror41/root/usr/share/gems/gems/activerecord-4.1.5/lib/active_record/transactions.rb:273:in `block in save!'
/opt/rh/rh-ror41/root/usr/share/gems/gems/activerecord-4.1.5/lib/active_record/transactions.rb:329:in `block in with_transaction_returning_status'
/opt/rh/rh-ror41/root/usr/share/gems/gems/activerecord-4.1.5/lib/active_record/connection_adapters/abstract/database_statements.rb:201:in `block in transaction'
/opt/rh/rh-ror41/root/usr/share/gems/gems/activerecord-4.1.5/lib/active_record/connection_adapters/abstract/database_statements.rb:209:in `within_new_transaction'
/opt/rh/rh-ror41/root/usr/share/gems/gems/activerecord-4.1.5/lib/active_record/connection_adapters/abstract/database_statements.rb:201:in `transaction'
/opt/rh/rh-ror41/root/usr/share/gems/gems/activerecord-4.1.5/lib/active_record/transactions.rb:208:in `transaction'
/opt/rh/rh-ror41/root/usr/share/gems/gems/activerecord-4.1.5/lib/active_record/transactions.rb:326:in `with_transaction_returning_status'
/opt/rh/rh-ror41/root/usr/share/gems/gems/activerecord-4.1.5/lib/active_record/transactions.rb:273:in `save!'
/usr/share/foreman/app/models/host/base.rb:415:in `block in update_virtuals'
/opt/rh/rh-ror41/root/usr/share/gems/gems/activerecord-4.1.5/lib/active_record/relation/delegation.rb:46:in `each'
/opt/rh/rh-ror41/root/usr/share/gems/gems/activerecord-4.1.5/lib/active_record/relation/delegation.rb:46:in `each'
/usr/share/foreman/app/models/host/base.rb:410:in `update_virtuals'
/usr/share/foreman/app/models/host/base.rb:395:in `set_interface'
/usr/share/foreman/app/models/host/base.rb:210:in `block in set_interfaces'
/usr/share/foreman/app/models/host/base.rb:207:in `each'
/usr/share/foreman/app/models/host/base.rb:207:in `set_interfaces'
/usr/share/foreman/app/models/host/base.rb:184:in `populate_fields_from_facts'
/usr/share/foreman/app/models/host/managed.rb:474:in `populate_fields_from_facts'
/usr/share/foreman/app/models/host/base.rb:158:in `import_facts'
/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.0.0.54/app/models/katello/host/subscription_facet.rb:82:in `update_facts'
/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.0.0.54/lib/katello/tasks/upgrades/3.0/update_subscription_facet_backend_data.rake:16:in `block (5 levels) in <top (required)>'
/opt/rh/rh-ror41/root/usr/share/gems/gems/activerecord-4.1.5/lib/active_record/relation/batches.rb:52:in `block (2 levels) in find_each'
/opt/rh/rh-ror41/root/usr/share/gems/gems/activerecord-4.1.5/lib/active_record/relation/batches.rb:52:in `each'
/opt/rh/rh-ror41/root/usr/share/gems/gems/activerecord-4.1.5/lib/active_record/relation/batches.rb:52:in `block in find_each'
/opt/rh/rh-ror41/root/usr/share/gems/gems/activerecord-4.1.5/lib/active_record/relation/batches.rb:125:in `find_in_batches'
/opt/rh/rh-ror41/root/usr/share/gems/gems/activerecord-4.1.5/lib/active_record/relation/batches.rb:51:in `find_each'
/opt/rh/rh-ror41/root/usr/share/gems/gems/activerecord-4.1.5/lib/active_record/querying.rb:9:in `find_each'
/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.0.0.54/lib/katello/tasks/upgrades/3.0/update_subscription_facet_backend_data.rake:8:in `block (4 levels) in <top (required)>'
Tasks: TOP => katello:upgrades:3.0:update_subscription_facet_backend_data
(See full trace by running task with --trace)
Updating backend data for subscription facets

[ERROR 2016-06-30 03:23:04 main] Upgrade step update_subscription_facet_backend_data failed. Check logs for more information.

Comment 1 Sachin Ghai 2016-06-30 07:41:31 UTC
Created attachment 1174364 [details]
foreman-debug logs

Comment 2 Sachin Ghai 2016-06-30 07:47:42 UTC
could it be because we have configured a host with vlan interface identifier eth0.1 on sat6.1.9 before performing upgrade ?

Comment 4 Ivan Necas 2016-06-30 12:59:57 UTC
This is seems to be introduced by https://bugzilla.redhat.com/show_bug.cgi?id=1344449

Not sure if the problem is in the fix of https://bugzilla.redhat.com/show_bug.cgi?id=1344449 or it's just showing us other error that was already there.
 
It might be pretty likely be connected with the vlan interface identifier 'eth0.1' on sat 6.1.9

Comment 5 Ivan Necas 2016-06-30 13:13:39 UTC
Seems like related to this issue: investigating how we could fix this http://projects.theforeman.org/issues/11247

Comment 6 Ivan Necas 2016-06-30 14:04:45 UTC
Forgot to put the question: how was the original created?

Comment 7 Ivan Necas 2016-06-30 14:05:25 UTC
And I mean host "mac001a4a3ea613.satqe.lab.eng.rdu2.redhat.com", that has the eth0.1 and eth0:1 interfaces

Comment 8 Sachin Ghai 2016-06-30 14:09:49 UTC
I created this host before upgrade on 6.1.9 with alias and vlan interfaces and attached to eth0 interface on static subnet.

Comment 11 Ivan Necas 2016-07-01 10:30:04 UTC
Created redmine issue http://projects.theforeman.org/issues/15559 from this bug

Comment 12 Ivan Necas 2016-07-01 10:37:16 UTC
The real cause described in http://projects.theforeman.org/issues/15559, Fix proposed in https://github.com/Katello/katello/pull/6160

Comment 13 Ivan Necas 2016-07-01 10:50:13 UTC
After applying the patch, I was able to proceed with upgrade.

Comment 14 Bryan Kearney 2016-07-01 12:17:25 UTC
Upstream bug component is Subscription Management

Comment 15 Brad Buckingham 2016-07-01 12:58:12 UTC
Upstream bug merged, moving to POST

Comment 16 Sachin Ghai 2016-07-07 08:08:51 UTC
Verified with sat62 GA snap19. The reported issue is no longer reproducible. Installer completed successfully without any issue.

[root@qe-sat6-upgrade-rhel7 ~]# satellite-installer --scenario satellite --upgrade
Upgrading...
Upgrade Step: stop_services...
Upgrade Step: start_databases...
Upgrade Step: update_http_conf...
Upgrade Step: migrate_pulp...
Upgrade Step: fix_pulp_httpd_conf...
Upgrade Step: start_httpd...
Upgrade Step: start_qpidd...
Upgrade Step: start_pulp...
Upgrade Step: migrate_candlepin...
Upgrade Step: start_tomcat...
Upgrade Step: fix_katello_settings_file...
Upgrade Step: migrate_foreman...
Upgrade Step: remove_nodes_distributors...
Upgrade Step: Running installer...
Installing             Done                                               [100%] [..................................................................]
  The full log is at /var/log/foreman-installer/satellite.log
Upgrade Step: restart_services...
Upgrade Step: db_seed...
Upgrade Step: import_package_groups (this may take a while) ...
Upgrade Step: import_rpms (this may take a while) ...
Upgrade Step: import_distributions (this may take a while) ...
Upgrade Step: import_puppet_modules (this may take a while) ...
Upgrade Step: import_subscriptions (this may take a while) ...
Upgrade Step: elasticsearch_message...
Elasticsearch has been removed as a dependency.  The database files can be removed manually with #rm -rf /var/lib/elasticsearch.  Some packages are no longer needed and can be removed:  #rpm -e ruby193-rubygem-tire tfm-rubygem-tire elasticsearch sigar-java sigar snappy-java lucene4-contrib lucene4
Upgrade Step: add_export_distributor (this may take a while) ...
Upgrade Step: remove_docker_v1_content (this may take a while) ...
Upgrade Step: update_puppet_repository_distributors (this may take a while) ...
Upgrade Step: update_subscription_facet_backend_data (this may take a while) ...
Upgrade Step: remove_gutterball...
Satellite upgrade completed!

Comment 17 Bryan Kearney 2016-07-27 11:23:39 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2016:1501