Bug 1356447

Summary: upgrade failed at migrate_pulp step: /usr/share/ruby/erb.rb:849:in `eval': (erb):45: syntax error, unexpected tGVAR, expecting ')' (SyntaxError)
Product: Red Hat Satellite Reporter: Sachin Ghai <sghai>
Component: UpgradesAssignee: Ivan Necas <inecas>
Status: CLOSED ERRATA QA Contact: Sachin Ghai <sghai>
Severity: high Docs Contact:
Priority: unspecified    
Version: 6.2.0CC: ahumbe, bbuckingham, inecas, mtaru, xdmoon
Target Milestone: UnspecifiedKeywords: Triaged
Target Release: Unused   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: katello-installer-base-3.0.0.55-1 Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2016-07-27 11:27:12 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
foreman-debug
none
upgrade successfully completed none

Description Sachin Ghai 2016-07-14 07:07:27 UTC
Description of problem:
I got following errors on console while performing upgrade from sat6.1.9 -> sat62 snap20 on rhel7

[root@satupgrade ~]# satellite-installer --scenario satellite --upgrade
Upgrading...
Upgrade Step: stop_services...
Redirecting to /bin/systemctl stop  httpd.service

Redirecting to /bin/systemctl stop  pulp_workers.service

Redirecting to /bin/systemctl stop  pulp_resource_manager.service

Redirecting to /bin/systemctl stop  pulp_celerybeat.service

Redirecting to /bin/systemctl stop  foreman-proxy.service

Redirecting to /bin/systemctl stop  tomcat.service

Redirecting to /bin/systemctl stop  qdrouterd.service

Redirecting to /bin/systemctl stop  qpidd.service

Success!

Upgrade Step: start_databases...
Redirecting to /bin/systemctl start  mongod.service

Redirecting to /bin/systemctl start  postgresql.service

Success!

Upgrade Step: update_http_conf...

Upgrade Step: migrate_pulp...
3772

/usr/share/ruby/erb.rb:849:in `eval': (erb):45: syntax error, unexpected tGVAR, expecting ')' (SyntaxError)
  {'$set': {constants.PULP_USER_METADATA_FIELDNAME: {}}}, multi=True)
        ^
(erb):81: syntax error, unexpected tGVAR, expecting ')'
... collection.update({}, {'$unset': {'owner_type': "", 'owner_...
...                               ^
(erb):81: syntax error, unexpected tIDENTIFIER, expecting ')'
...ate({}, {'$unset': {'owner_type': "", 'owner_id': ''}}, mult...
...                               ^
(erb):81: syntax error, unexpected tIDENTIFIER, expecting ')'
...': {'owner_type': "", 'owner_id': ''}}, multi=True)
...                               ^
(erb):162: syntax error, unexpected tGVAR, expecting ')'
...{"translated_name": ""}, {'$set': {'translated_name': {}}}, ...
...                               ^
(erb):162: syntax error, unexpected tIDENTIFIER, expecting ')'
...""}, {'$set': {'translated_name': {}}}, multi=True)
...                               ^
(erb):203: syntax error, unexpected ')', expecting end-of-input
', :yellow) ).to_s); _erbout.force_encoding(__ENCODING__)
                   ^
	from /usr/share/ruby/erb.rb:849:in `result'
	from /usr/share/gems/gems/highline-1.6.21/lib/highline.rb:719:in `format_statement'
	from /usr/share/gems/gems/highline-1.6.21/lib/highline.rb:617:in `say'
	from /usr/share/katello-installer-base/hooks/boot/01-helpers.rb:19:in `log_and_say'
	from /usr/share/katello-installer-base/hooks/boot/01-helpers.rb:35:in `block in execute'
	from /usr/share/katello-installer-base/hooks/boot/01-helpers.rb:31:in `each'
	from /usr/share/katello-installer-base/hooks/boot/01-helpers.rb:31:in `execute'
	from /usr/share/katello-installer-base/hooks/pre/30-upgrade.rb:41:in `migrate_pulp'
	from /usr/share/katello-installer-base/hooks/pre/30-upgrade.rb:81:in `upgrade_step'
	from /usr/share/katello-installer-base/hooks/pre/30-upgrade.rb:101:in `block (4 levels) in load'
	from /usr/share/gems/gems/kafo-0.7.5/lib/kafo/hooking.rb:34:in `instance_eval'
	from /usr/share/gems/gems/kafo-0.7.5/lib/kafo/hooking.rb:34:in `block (4 levels) in load'
	from /usr/share/gems/gems/kafo-0.7.5/lib/kafo/hook_context.rb:13:in `instance_exec'
	from /usr/share/gems/gems/kafo-0.7.5/lib/kafo/hook_context.rb:13:in `execute'
	from /usr/share/gems/gems/kafo-0.7.5/lib/kafo/hooking.rb:51:in `block in execute'
	from /usr/share/gems/gems/kafo-0.7.5/lib/kafo/hooking.rb:49:in `each'
	from /usr/share/gems/gems/kafo-0.7.5/lib/kafo/hooking.rb:49:in `execute'
	from /usr/share/gems/gems/kafo-0.7.5/lib/kafo/kafo_configure.rb:377:in `run_installation'
	from /usr/share/gems/gems/kafo-0.7.5/lib/kafo/kafo_configure.rb:143:in `execute'
	from /usr/share/gems/gems/clamp-0.6.2/lib/clamp/command.rb:67:in `run'
	from /usr/share/gems/gems/clamp-0.6.2/lib/clamp/command.rb:125:in `run'
	from /usr/share/gems/gems/kafo-0.7.5/lib/kafo/kafo_configure.rb:150:in `run'
	from /usr/sbin/satellite-installer:12:in `<main>'


Version-Release number of selected component (if applicable):
sat6.1.9 -> sat62 snap20 on rhel7


How reproducible:

Steps to Reproduce:
1. installed 6.1.9 on rhel7
2. synced rhel6 repos from CDN, 
3. publish/promote to cv
4. disable 6.1 repo
5. enable 6.2 repo
6. katello-service stop
7. yum update -y
8. satellite-installer --scenario satellite --upgrade

Actual results:


Expected results:
installer should successfully complete

Additional info:
in /varlog/messages I see following:

 Jul 13 22:53:48 satupgrade pulp: pulp.server.db.manage:INFO: Database migrations complete.
Jul 13 22:55:57 satupgrade puppet-agent[3883]: Unable to fetch my node definition, but the agent run will continue:
Jul 13 22:55:57 satupgrade puppet-agent[3883]: Connection refused - connect(2)
Jul 13 22:55:58 satupgrade puppet-agent[3883]: (/File[/var/lib/puppet/facts.d]) Failed to generate additional resources using 'eval_generate': Connection refused - connect(2)
Jul 13 22:55:58 satupgrade puppet-agent[3883]: (/File[/var/lib/puppet/facts.d]) Could not evaluate: Could not retrieve file metadata for puppet://satupgrade.example.com/pluginfacts: Connection refused - connect(2)
Jul 13 22:55:58 satupgrade puppet-agent[3883]: (/File[/var/lib/puppet/lib]) Failed to generate additional resources using 'eval_generate': Connection refused - connect(2)
Jul 13 22:55:58 satupgrade puppet-agent[3883]: (/File[/var/lib/puppet/lib]) Could not evaluate: Could not retrieve file metadata for puppet://satupgrade.example.com/plugins: Connection refused - connect(2)

Comment 1 Sachin Ghai 2016-07-14 07:26:38 UTC
Created attachment 1179646 [details]
foreman-debug

Comment 3 Ivan Necas 2016-07-14 08:59:37 UTC
The problem is the logged message contains characters, that are not escaped. I will try to sanitize the output. Anyway, it might be an indicator of an error on pulp side as well: which produced the error message that we are not able to display, looking more deeply into it

Comment 4 Ivan Necas 2016-07-14 09:07:02 UTC
It's a regression from https://bugzilla.redhat.com/show_bug.cgi?id=1354631, where we started showing the pulp output to the end user, but we missed some escaping, the fix on it's way

Comment 6 Ivan Necas 2016-07-14 10:26:51 UTC
Created redmine issue http://projects.theforeman.org/issues/15683 from this bug

Comment 7 Ivan Necas 2016-07-14 10:55:23 UTC
A proposed fix https://github.com/Katello/katello-installer/pull/374

Comment 8 Sachin Ghai 2016-07-15 11:37:31 UTC
*** Bug 1356950 has been marked as a duplicate of this bug. ***

Comment 9 Sachin Ghai 2016-07-18 11:58:20 UTC
Verified with upgrade from sat6.1.9 -> sat6.2 GA snap20.2

The reported issue has been fixed. Upgrade completed successfully. Please see the console logs for verification.

Comment 10 Sachin Ghai 2016-07-18 12:00:02 UTC
Created attachment 1181025 [details]
upgrade successfully completed

Comment 11 Brad Buckingham 2016-07-19 14:38:38 UTC
*** Bug 1357217 has been marked as a duplicate of this bug. ***

Comment 12 Bryan Kearney 2016-07-27 11:27:12 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2016:1501