Note: This bug is displayed in read-only format because
the product is no longer active in Red Hat Bugzilla.
Red Hat Satellite engineering is moving the tracking of its product development work on Satellite to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "Satellite project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs will be migrated starting at the end of May. If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "Satellite project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/SAT-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.
Description of problem:
Upgrade from up2date 6.3 to 6.4.0 Snap15: Systemd start for puppetserver failed!
Version-Release number of selected component (if applicable):
satellite-6.4.0-10.beta.el7sat.noarch (snap15)
How reproducible:
1 of 1
Steps to Reproduce:
1. Have 6.3, fully up2date
2. Configure all the repos required for 6.4 snap15 and attempt to yum upgrade - it fails with some dependency issues around puppet - this reminds you to upgrade to puppet 4 first
3. # subscription-manager repos --enable rhel-7-server-satellite-6.3-puppet4-rpms
# satellite-installer --upgrade-puppet
4. # yum -y upgrade
5. # satellite-installer --upgrade
Actual results:
[...]
Upgrade Step: mongo_mmapv1_check...
systemctl stop mongod finished successfully!
Upgrade Step: Running installer...
Systemd start for puppetserver failed!
journalctl log for puppetserver:
-- Logs begin at Fri 2018-07-27 08:33:48 EDT, end at Wed 2018-08-01 13:00:03 EDT. --
Aug 01 12:58:14 host-8-251-131.host.centralci.eng.rdu2.redhat.com systemd[1]: Starting puppetserver Service...
Aug 01 12:58:14 host-8-251-131.host.centralci.eng.rdu2.redhat.com puppetserver[27516]: OpenJDK 64-Bit Server VM warning: ignoring option MaxPermSize=256m; support was removed in 8.0
Aug 01 13:00:02 host-8-251-131.host.centralci.eng.rdu2.redhat.com puppetserver[27516]: Exception in thread "main" java.lang.IllegalArgumentException: service :puppetlabs.trapperkeeper.services.metrics.metrics-service/metrics-webservice does not appear in configuration
Aug 01 13:00:02 host-8-251-131.host.centralci.eng.rdu2.redhat.com puppetserver[27516]: at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
Aug 01 13:00:02 host-8-251-131.host.centralci.eng.rdu2.redhat.com puppetserver[27516]: at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
Aug 01 13:00:02 host-8-251-131.host.centralci.eng.rdu2.redhat.com puppetserver[27516]: at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
Aug 01 13:00:02 host-8-251-131.host.centralci.eng.rdu2.redhat.com puppetserver[27516]: at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
Aug 01 13:00:02 host-8-251-131.host.centralci.eng.rdu2.redhat.com puppetserver[27516]: at clojure.lang.Reflector.invokeConstructor(Reflector.java:180)
Aug 01 13:00:02 host-8-251-131.host.centralci.eng.rdu2.redhat.com puppetserver[27516]: at puppetlabs.trapperkeeper.services.webrouting.webrouting_service_core$get_endpoint_and_server_from_config.invokeStatic(webrouting_service_core.clj:54)
Aug 01 13:00:02 host-8-251-131.host.centralci.eng.rdu2.redhat.com puppetserver[27516]: at puppetlabs.trapperkeeper.services.webrouting.webrouting_service_core$get_endpoint_and_server_from_config.invoke(webrouting_service_core.clj:38)
Aug 01 13:00:02 host-8-251-131.host.centralci.eng.rdu2.redhat.com puppetserver[27516]: at puppetlabs.trapperkeeper.services.webrouting.webrouting_service_core$get_route.invokeStatic(webrouting_service_core.clj:99)
Aug 01 13:00:02 host-8-251-131.host.centralci.eng.rdu2.redhat.com puppetserver[27516]: at puppetlabs.trapperkeeper.services.webrouting.webrouting_service_core$get_route.invoke(webrouting_service_core.clj:96)
Aug 01 13:00:02 host-8-251-131.host.centralci.eng.rdu2.redhat.com puppetserver[27516]: at puppetlabs.trapperkeeper.services.webrouting.webrouting_service$reify__36960$service_fnk__5266__auto___positional$reify__36968.get_route(webrouting_service.clj:37)
Aug 01 13:00:02 host-8-251-131.host.centralci.eng.rdu2.redhat.com puppetserver[27516]: at puppetlabs.trapperkeeper.services.webrouting.webrouting_service$eval36690$fn__36854$G__36674__36860.invoke(webrouting_service.clj:10)
Aug 01 13:00:02 host-8-251-131.host.centralci.eng.rdu2.redhat.com puppetserver[27516]: at puppetlabs.trapperkeeper.services.webrouting.webrouting_service$eval36690$fn__36854$G__36673__36867.invoke(webrouting_service.clj:10)
Aug 01 13:00:02 host-8-251-131.host.centralci.eng.rdu2.redhat.com puppetserver[27516]: at clojure.core$partial$fn__4759.invoke(core.clj:2515)
Aug 01 13:00:02 host-8-251-131.host.centralci.eng.rdu2.redhat.com puppetserver[27516]: at puppetlabs.trapperkeeper.services.metrics.metrics_service$reify__41924$service_fnk__5266__auto___positional$reify__41935.init(metrics_service.clj:64)
Aug 01 13:00:02 host-8-251-131.host.centralci.eng.rdu2.redhat.com puppetserver[27516]: at puppetlabs.trapperkeeper.services$eval5068$fn__5069$G__5056__5072.invoke(services.clj:9)
Aug 01 13:00:02 host-8-251-131.host.centralci.eng.rdu2.redhat.com puppetserver[27516]: at puppetlabs.trapperkeeper.services$eval5068$fn__5069$G__5055__5076.invoke(services.clj:9)
Aug 01 13:00:02 host-8-251-131.host.centralci.eng.rdu2.redhat.com puppetserver[27516]: at puppetlabs.trapperkeeper.internal$eval13835$run_lifecycle_fn_BANG___13842$fn__13843.invoke(internal.clj:198)
Aug 01 13:00:02 host-8-251-131.host.centralci.eng.rdu2.redhat.com puppetserver[27516]: at puppetlabs.trapperkeeper.internal$eval13835$run_lifecycle_fn_BANG___13842.invoke(internal.clj:181)
Aug 01 13:00:02 host-8-251-131.host.centralci.eng.rdu2.redhat.com puppetserver[27516]: at puppetlabs.trapperkeeper.internal$eval13864$run_lifecycle_fns__13869$fn__13870.invoke(internal.clj:231)
Aug 01 13:00:02 host-8-251-131.host.centralci.eng.rdu2.redhat.com puppetserver[27516]: at puppetlabs.trapperkeeper.internal$eval13864$run_lifecycle_fns__13869.invoke(internal.clj:208)
Aug 01 13:00:02 host-8-251-131.host.centralci.eng.rdu2.redhat.com puppetserver[27516]: at puppetlabs.trapperkeeper.internal$eval14346$build_app_STAR___14355$fn$reify__14365.init(internal.clj:583)
Aug 01 13:00:02 host-8-251-131.host.centralci.eng.rdu2.redhat.com puppetserver[27516]: at puppetlabs.trapperkeeper.internal$eval14392$boot_services_for_app_STAR__STAR___14399$fn__14400$fn__14402.invoke(internal.clj:611)
Aug 01 13:00:02 host-8-251-131.host.centralci.eng.rdu2.redhat.com puppetserver[27516]: at puppetlabs.trapperkeeper.internal$eval14392$boot_services_for_app_STAR__STAR___14399$fn__14400.invoke(internal.clj:610)
Aug 01 13:00:02 host-8-251-131.host.centralci.eng.rdu2.redhat.com puppetserver[27516]: at puppetlabs.trapperkeeper.internal$eval14392$boot_services_for_app_STAR__STAR___14399.invoke(internal.clj:604)
Aug 01 13:00:02 host-8-251-131.host.centralci.eng.rdu2.redhat.com puppetserver[27516]: at clojure.core$partial$fn__4761.invoke(core.clj:2521)
Aug 01 13:00:02 host-8-251-131.host.centralci.eng.rdu2.redhat.com puppetserver[27516]: at puppetlabs.trapperkeeper.internal$eval13905$initialize_lifecycle_worker__13916$fn__13917$fn__14013$state_machine__11875__auto____14014$fn__14016.invoke(internal.clj:251)
Aug 01 13:00:02 host-8-251-131.host.centralci.eng.rdu2.redhat.com puppetserver[27516]: at puppetlabs.trapperkeeper.internal$eval13905$initialize_lifecycle_worker__13916$fn__13917$fn__14013$state_machine__11875__auto____14014.invoke(internal.clj:251)
Aug 01 13:00:02 host-8-251-131.host.centralci.eng.rdu2.redhat.com puppetserver[27516]: at clojure.core.async.impl.ioc_macros$run_state_machine.invokeStatic(ioc_macros.clj:1012)
Aug 01 13:00:02 host-8-251-131.host.centralci.eng.rdu2.redhat.com puppetserver[27516]: at clojure.core.async.impl.ioc_macros$run_state_machine.invoke(ioc_macros.clj:1011)
Aug 01 13:00:02 host-8-251-131.host.centralci.eng.rdu2.redhat.com puppetserver[27516]: at clojure.core.async.impl.ioc_macros$run_state_machine_wrapped.invokeStatic(ioc_macros.clj:1016)
Aug 01 13:00:02 host-8-251-131.host.centralci.eng.rdu2.redhat.com puppetserver[27516]: at clojure.core.async.impl.ioc_macros$run_state_machine_wrapped.invoke(ioc_macros.clj:1014)
Aug 01 13:00:02 host-8-251-131.host.centralci.eng.rdu2.redhat.com puppetserver[27516]: at clojure.core.async$ioc_alts_BANG_$fn__12043.invoke(async.clj:383)
Aug 01 13:00:02 host-8-251-131.host.centralci.eng.rdu2.redhat.com puppetserver[27516]: at clojure.core.async$do_alts$fn__11989$fn__11992.invoke(async.clj:252)
Aug 01 13:00:02 host-8-251-131.host.centralci.eng.rdu2.redhat.com puppetserver[27516]: at clojure.core.async.impl.channels.ManyToManyChannel$fn__6799$fn__6800.invoke(channels.clj:95)
Aug 01 13:00:02 host-8-251-131.host.centralci.eng.rdu2.redhat.com puppetserver[27516]: at clojure.lang.AFn.run(AFn.java:22)
Aug 01 13:00:02 host-8-251-131.host.centralci.eng.rdu2.redhat.com puppetserver[27516]: at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
Aug 01 13:00:02 host-8-251-131.host.centralci.eng.rdu2.redhat.com puppetserver[27516]: at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
Aug 01 13:00:02 host-8-251-131.host.centralci.eng.rdu2.redhat.com puppetserver[27516]: at java.lang.Thread.run(Thread.java:748)
Aug 01 13:00:03 host-8-251-131.host.centralci.eng.rdu2.redhat.com systemd[1]: puppetserver.service: control process exited, code=exited status=1
Aug 01 13:00:03 host-8-251-131.host.centralci.eng.rdu2.redhat.com systemd[1]: Failed to start puppetserver Service.
Aug 01 13:00:03 host-8-251-131.host.centralci.eng.rdu2.redhat.com systemd[1]: Unit puppetserver.service entered failed state.
Aug 01 13:00:03 host-8-251-131.host.centralci.eng.rdu2.redhat.com systemd[1]: puppetserver.service failed.
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/provider/service/systemd.rb:167:in `rescue in start'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/provider/service/systemd.rb:163:in `start'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/type/service.rb:103:in `block (3 levels) in <module:Puppet>'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/property.rb:491:in `set'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/property.rb:568:in `sync'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/type/service.rb:114:in `sync'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/transaction/resource_harness.rb:236:in `sync'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/transaction/resource_harness.rb:134:in `sync_if_needed'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/transaction/resource_harness.rb:80:in `perform_changes'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/transaction/resource_harness.rb:21:in `evaluate'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/transaction.rb:257:in `apply'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/transaction.rb:277:in `eval_resource'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/transaction.rb:181:in `call'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/transaction.rb:181:in `block (2 levels) in evaluate'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/util.rb:513:in `block in thinmark'
/opt/puppetlabs/puppet/lib/ruby/2.4.0/benchmark.rb:308:in `realtime'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/util.rb:512:in `thinmark'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/transaction.rb:181:in `block in evaluate'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/graph/relationship_graph.rb:121:in `traverse'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/transaction.rb:171:in `evaluate'
/usr/share/gems/gems/kafo-2.1.0/modules/kafo_configure/lib/puppet/parser/functions/add_progress.rb:30:in `evaluate_with_trigger'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/resource/catalog.rb:239:in `block (2 levels) in apply'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/util.rb:513:in `block in thinmark'
/opt/puppetlabs/puppet/lib/ruby/2.4.0/benchmark.rb:308:in `realtime'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/util.rb:512:in `thinmark'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/resource/catalog.rb:238:in `block in apply'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/util/log.rb:156:in `with_destination'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/transaction/report.rb:146:in `as_logging_destination'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/resource/catalog.rb:237:in `apply'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/configurer.rb:187:in `block (2 levels) in apply_catalog'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/util.rb:513:in `block in thinmark'
/opt/puppetlabs/puppet/lib/ruby/2.4.0/benchmark.rb:308:in `realtime'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/util.rb:512:in `thinmark'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/configurer.rb:186:in `block in apply_catalog'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/util.rb:232:in `block in benchmark'
/opt/puppetlabs/puppet/lib/ruby/2.4.0/benchmark.rb:308:in `realtime'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/util.rb:231:in `benchmark'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/configurer.rb:185:in `apply_catalog'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/configurer.rb:368:in `run_internal'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/configurer.rb:239:in `block in run'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/context.rb:65:in `override'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet.rb:260:in `override'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/configurer.rb:213:in `run'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/application/apply.rb:354:in `apply_catalog'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/application/apply.rb:280:in `block (2 levels) in main'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/context.rb:65:in `override'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet.rb:260:in `override'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/application/apply.rb:280:in `block in main'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/context.rb:65:in `override'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet.rb:260:in `override'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/application/apply.rb:233:in `main'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/application/apply.rb:174:in `run_command'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/application.rb:375:in `block in run'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/util.rb:661:in `exit_on_fail'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/application.rb:375:in `run'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/util/command_line.rb:137:in `run'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/util/command_line.rb:73:in `execute'
/opt/puppetlabs/puppet/bin/puppet:5:in `<main>'
/Stage[main]/Puppet::Server::Service/Service[puppetserver]/ensure: change from 'stopped' to 'running' failed: Systemd start for puppetserver failed!
journalctl log for puppetserver:
-- Logs begin at Fri 2018-07-27 08:33:48 EDT, end at Wed 2018-08-01 13:00:03 EDT. --
Aug 01 12:58:14 host-8-251-131.host.centralci.eng.rdu2.redhat.com systemd[1]: Starting puppetserver Service...
Aug 01 12:58:14 host-8-251-131.host.centralci.eng.rdu2.redhat.com puppetserver[27516]: OpenJDK 64-Bit Server VM warning: ignoring option MaxPermSize=256m; support was removed in 8.0
Aug 01 13:00:02 host-8-251-131.host.centralci.eng.rdu2.redhat.com puppetserver[27516]: Exception in thread "main" java.lang.IllegalArgumentException: service :puppetlabs.trapperkeeper.services.metrics.metrics-service/metrics-webservice does not appear in configuration
Aug 01 13:00:02 host-8-251-131.host.centralci.eng.rdu2.redhat.com puppetserver[27516]: at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
Aug 01 13:00:02 host-8-251-131.host.centralci.eng.rdu2.redhat.com puppetserver[27516]: at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
Aug 01 13:00:02 host-8-251-131.host.centralci.eng.rdu2.redhat.com puppetserver[27516]: at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
Aug 01 13:00:02 host-8-251-131.host.centralci.eng.rdu2.redhat.com puppetserver[27516]: at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
Aug 01 13:00:02 host-8-251-131.host.centralci.eng.rdu2.redhat.com puppetserver[27516]: at clojure.lang.Reflector.invokeConstructor(Reflector.java:180)
Aug 01 13:00:02 host-8-251-131.host.centralci.eng.rdu2.redhat.com puppetserver[27516]: at puppetlabs.trapperkeeper.services.webrouting.webrouting_service_core$get_endpoint_and_server_from_config.invokeStatic(webrouting_service_core.clj:54)
Aug 01 13:00:02 host-8-251-131.host.centralci.eng.rdu2.redhat.com puppetserver[27516]: at puppetlabs.trapperkeeper.services.webrouting.webrouting_service_core$get_endpoint_and_server_from_config.invoke(webrouting_service_core.clj:38)
Aug 01 13:00:02 host-8-251-131.host.centralci.eng.rdu2.redhat.com puppetserver[27516]: at puppetlabs.trapperkeeper.services.webrouting.webrouting_service_core$get_route.invokeStatic(webrouting_service_core.clj:99)
Aug 01 13:00:02 host-8-251-131.host.centralci.eng.rdu2.redhat.com puppetserver[27516]: at puppetlabs.trapperkeeper.services.webrouting.webrouting_service_core$get_route.invoke(webrouting_service_core.clj:96)
Aug 01 13:00:02 host-8-251-131.host.centralci.eng.rdu2.redhat.com puppetserver[27516]: at puppetlabs.trapperkeeper.services.webrouting.webrouting_service$reify__36960$service_fnk__5266__auto___positional$reify__36968.get_route(webrouting_service.clj:37)
Aug 01 13:00:02 host-8-251-131.host.centralci.eng.rdu2.redhat.com puppetserver[27516]: at puppetlabs.trapperkeeper.services.webrouting.webrouting_service$eval36690$fn__36854$G__36674__36860.invoke(webrouting_service.clj:10)
Aug 01 13:00:02 host-8-251-131.host.centralci.eng.rdu2.redhat.com puppetserver[27516]: at puppetlabs.trapperkeeper.services.webrouting.webrouting_service$eval36690$fn__36854$G__36673__36867.invoke(webrouting_service.clj:10)
Aug 01 13:00:02 host-8-251-131.host.centralci.eng.rdu2.redhat.com puppetserver[27516]: at clojure.core$partial$fn__4759.invoke(core.clj:2515)
Aug 01 13:00:02 host-8-251-131.host.centralci.eng.rdu2.redhat.com puppetserver[27516]: at puppetlabs.trapperkeeper.services.metrics.metrics_service$reify__41924$service_fnk__5266__auto___positional$reify__41935.init(metrics_service.clj:64)
Aug 01 13:00:02 host-8-251-131.host.centralci.eng.rdu2.redhat.com puppetserver[27516]: at puppetlabs.trapperkeeper.services$eval5068$fn__5069$G__5056__5072.invoke(services.clj:9)
Aug 01 13:00:02 host-8-251-131.host.centralci.eng.rdu2.redhat.com puppetserver[27516]: at puppetlabs.trapperkeeper.services$eval5068$fn__5069$G__5055__5076.invoke(services.clj:9)
Aug 01 13:00:02 host-8-251-131.host.centralci.eng.rdu2.redhat.com puppetserver[27516]: at puppetlabs.trapperkeeper.internal$eval13835$run_lifecycle_fn_BANG___13842$fn__13843.invoke(internal.clj:198)
Aug 01 13:00:02 host-8-251-131.host.centralci.eng.rdu2.redhat.com puppetserver[27516]: at puppetlabs.trapperkeeper.internal$eval13835$run_lifecycle_fn_BANG___13842.invoke(internal.clj:181)
Aug 01 13:00:02 host-8-251-131.host.centralci.eng.rdu2.redhat.com puppetserver[27516]: at puppetlabs.trapperkeeper.internal$eval13864$run_lifecycle_fns__13869$fn__13870.invoke(internal.clj:231)
Aug 01 13:00:02 host-8-251-131.host.centralci.eng.rdu2.redhat.com puppetserver[27516]: at puppetlabs.trapperkeeper.internal$eval13864$run_lifecycle_fns__13869.invoke(internal.clj:208)
Aug 01 13:00:02 host-8-251-131.host.centralci.eng.rdu2.redhat.com puppetserver[27516]: at puppetlabs.trapperkeeper.internal$eval14346$build_app_STAR___14355$fn$reify__14365.init(internal.clj:583)
Aug 01 13:00:02 host-8-251-131.host.centralci.eng.rdu2.redhat.com puppetserver[27516]: at puppetlabs.trapperkeeper.internal$eval14392$boot_services_for_app_STAR__STAR___14399$fn__14400$fn__14402.invoke(internal.clj:611)
Aug 01 13:00:02 host-8-251-131.host.centralci.eng.rdu2.redhat.com puppetserver[27516]: at puppetlabs.trapperkeeper.internal$eval14392$boot_services_for_app_STAR__STAR___14399$fn__14400.invoke(internal.clj:610)
Aug 01 13:00:02 host-8-251-131.host.centralci.eng.rdu2.redhat.com puppetserver[27516]: at puppetlabs.trapperkeeper.internal$eval14392$boot_services_for_app_STAR__STAR___14399.invoke(internal.clj:604)
Aug 01 13:00:02 host-8-251-131.host.centralci.eng.rdu2.redhat.com puppetserver[27516]: at clojure.core$partial$fn__4761.invoke(core.clj:2521)
Aug 01 13:00:02 host-8-251-131.host.centralci.eng.rdu2.redhat.com puppetserver[27516]: at puppetlabs.trapperkeeper.internal$eval13905$initialize_lifecycle_worker__13916$fn__13917$fn__14013$state_machine__11875__auto____14014$fn__14016.invoke(internal.clj:251)
Aug 01 13:00:02 host-8-251-131.host.centralci.eng.rdu2.redhat.com puppetserver[27516]: at puppetlabs.trapperkeeper.internal$eval13905$initialize_lifecycle_worker__13916$fn__13917$fn__14013$state_machine__11875__auto____14014.invoke(internal.clj:251)
Aug 01 13:00:02 host-8-251-131.host.centralci.eng.rdu2.redhat.com puppetserver[27516]: at clojure.core.async.impl.ioc_macros$run_state_machine.invokeStatic(ioc_macros.clj:1012)
Aug 01 13:00:02 host-8-251-131.host.centralci.eng.rdu2.redhat.com puppetserver[27516]: at clojure.core.async.impl.ioc_macros$run_state_machine.invoke(ioc_macros.clj:1011)
Aug 01 13:00:02 host-8-251-131.host.centralci.eng.rdu2.redhat.com puppetserver[27516]: at clojure.core.async.impl.ioc_macros$run_state_machine_wrapped.invokeStatic(ioc_macros.clj:1016)
Aug 01 13:00:02 host-8-251-131.host.centralci.eng.rdu2.redhat.com puppetserver[27516]: at clojure.core.async.impl.ioc_macros$run_state_machine_wrapped.invoke(ioc_macros.clj:1014)
Aug 01 13:00:02 host-8-251-131.host.centralci.eng.rdu2.redhat.com puppetserver[27516]: at clojure.core.async$ioc_alts_BANG_$fn__12043.invoke(async.clj:383)
Aug 01 13:00:02 host-8-251-131.host.centralci.eng.rdu2.redhat.com puppetserver[27516]: at clojure.core.async$do_alts$fn__11989$fn__11992.invoke(async.clj:252)
Aug 01 13:00:02 host-8-251-131.host.centralci.eng.rdu2.redhat.com puppetserver[27516]: at clojure.core.async.impl.channels.ManyToManyChannel$fn__6799$fn__6800.invoke(channels.clj:95)
Aug 01 13:00:02 host-8-251-131.host.centralci.eng.rdu2.redhat.com puppetserver[27516]: at clojure.lang.AFn.run(AFn.java:22)
Aug 01 13:00:02 host-8-251-131.host.centralci.eng.rdu2.redhat.com puppetserver[27516]: at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
Aug 01 13:00:02 host-8-251-131.host.centralci.eng.rdu2.redhat.com puppetserver[27516]: at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
Aug 01 13:00:02 host-8-251-131.host.centralci.eng.rdu2.redhat.com puppetserver[27516]: at java.lang.Thread.run(Thread.java:748)
Aug 01 13:00:03 host-8-251-131.host.centralci.eng.rdu2.redhat.com systemd[1]: puppetserver.service: control process exited, code=exited status=1
Aug 01 13:00:03 host-8-251-131.host.centralci.eng.rdu2.redhat.com systemd[1]: Failed to start puppetserver Service.
Aug 01 13:00:03 host-8-251-131.host.centralci.eng.rdu2.redhat.com systemd[1]: Unit puppetserver.service entered failed state.
Aug 01 13:00:03 host-8-251-131.host.centralci.eng.rdu2.redhat.com systemd[1]: puppetserver.service failed.
Installing Done [100%] [...........................................................................................................................................................]
Something went wrong! Check the log for ERROR-level output
The full log is at /var/log/foreman-installer/satellite.log
Upgrade Step: remove_legacy_mongo...
yum remove -y mongodb-2* mongodb-server-2* > /dev/null 2>&1 finished successfully!
rm -rf /etc/mongod.conf /var/log/mongodb finished successfully!
There are unfinished transactions remaining. You might consider running yum-complete-transaction, or "yum-complete-transaction --cleanup-only" and "yum history redo last", first to finish them. If those don't work you'll have to try removing/installing packages by hand (maybe package-cleanup can help).
yum install -y -q rh-mongodb34-syspaths finished successfully!
Upgrade failed during the installation phase. Fix the error and re-run the upgrade.
Expected results:
Should pass
Seems like this is connected to the failed upgrade due to puppet 3, and the sat 6.4 repos already enabled. The preupgrade check for the puppet 6.4 is implemented in foreman-maintain and it's part of the documented upgrade process https://bugzilla.redhat.com/show_bug.cgi?id=1571859
I recommend re-testing this with upgrading to puppet 4 before enabling the 6.4 repositories.
So, clean upgrade passed. I'm sorry for the confusion!
After discussing this on the scrum, what happened was probably this: when I was upgrading puppet on 6.3 (step 4. in initial comment), I had 6.4 repos enabled and that gor Satellite into inconsistent state which later caused 6.3 -> 6.4 upgrade error.