Root cause is that the /etc/foreman/logging.yaml is not upgraded (=rewritten by puppet) to include the layout config parameter.

That means 2 issues are here:
- Default config without layout parameter is not correcyly handled, either a nice error needs to be giving or the default (empty) value handling shall be corrected
- Add upgrade steps to upgrade the logging.yaml file (and also settings.yaml) files.

Peter

Ewoud: it looks like we're currently [1] marking all files in /etc/foreman as %config(noreplace), I would argue, that the /etc/foreman/logging.yaml should be marked just as %config, so that the upgrade in case of any change saves the old file if edited as rpmsave, instead of keeping the old config in place.

The reasoning for this is that the logging.yaml is used more as a place for keeping the defaults or some temporary changes, while the main user-facing logging changes are in /etc/foreman/settings.yaml

Thoughts?

[1] https://github.com/theforeman/foreman-packaging/blob/f207e3b478cf4fff85d91cb75dbdf7f85e7d7b25/packages/foreman/foreman/foreman.spec#L1203

If logging.yaml isn't meant to be modified by users but more the hardcoded defaults then it shouldn't live in /etc/foreman. IMHO /etc is user-owned and yum packages shouldn't overwrite them in pretty much all cases

I'd also argue that the Foreman code should never blindly rely on certain content in config files. If it relies on certain content then it should check and fail with a clean error message.

I would be probably ok with not symlinking /usr/share/foreman/config/logging.yaml to /etc/foreman

For me as the original reporter of this issue because i had 'debug' logging enabled on my sat6.3 for issue-troubleshooting.

As an end user i prefer a clear separation then between (mandatory) application defaults and custom user changes

The current config files looks to have default values that i as end user should not change to not risk breaking the application.

An Idea is to add a user owned file for customization like /etc/foreman/custom-settings.yaml that is then symlinked into the /usr/share/foreman/config and applied as last.

This file contains only the delta and has a better chance to survive/not-break upgrades from both user perspective and application perspective.

Peter

Peter: the log level setting, as well as other logigng options, can be set in /etc/foreman/settings.yaml, in `:logging` key (https://github.com/theforeman/foreman/blob/104318494560b2ec1ec998a403691980192d7c40/config/application.rb#L209 for more details), so there shouldn't be any need for exposing the logging.yaml, and we should not expose it in /etc/foreman directory, to not cause any additional confusion to the users.

Ivan,

I have seen that the new provided settings.yaml of sat6.4 and it contains many good examples.

Still it is not clear what parts of the settings.yaml i can change for customization and which cannot be touched 

Peter

The settings.yaml is managed by the installer, so what one can do is having a look at /usr/share/foreman-installer/modules/foreman/templates/settings.yaml.erb and then at the output of `satellite-installer --full-help` to find the option that one wants to change.

Of course, you can go directly to installer --full-help, but if one starts looking at the /etc/foreman/settings.yaml first and already knowns what option want to tweak. then looking at the installer's template can be helpful mid-step.

The only thing one needs to check is that the puppet variables are transformed to installer options by replacing `::` and `_` to `-` (foreman::logging_level -> --foreman-logging-level)

If you think that there is a gap in the documentation, please file another bug against documentation. This BZ should stay focused on the actually migration issue.

Ivan,

Only when i change manual the loglevel in the logging.yaml the log level is changed. Using the --foreman-logging-level correctly updated the settings.yaml, but this was not being honored. It looks like the logging.yaml is overriding the logging settings of the seettings.yaml.

Peter

I can't confirm this behaviour: I've just checked with 6.4, and the log level in settings.yml was overriding properly the logging.yaml file. If you would have more specific reproducer steps, it would be very helpful to investigate this issue more deeply.

First change was made using 'satellite-installer -v --foreman-logging-level=debug'
Later i made the changes directly in the settings.yaml and then did a 'service httpd restart'

Did not work, logging still at info:
------
# Log settings for the current environment can be adjusted by adding them
# here. For example, if you want to increase the log level.
:logging:
  :level: debug
  :production:
    :type: file
    :layout: pattern
------

Did not work, logging still at info:
------
# Log settings for the current environment can be adjusted by adding them
# here. For example, if you want to increase the log level.
:logging:
  :default:
    :level: debug
  :production:
    :type: file
    :layout: pattern
------

Works, logging is with debug:
------
# Log settings for the current environment can be adjusted by adding them
# here. For example, if you want to increase the log level.
:logging:
  :production:
    :type: file
    :layout: pattern
    :level: debug
------

I used a fresh OS and Sat6.4 installation.

Thanks Peter for the reproducing steps: I actually had commented out the #production part in the settings log: that's why I've not seen this issue.

That seems like a regression caused by https://projects.theforeman.org/issues/22559/ introduced in 6.4 and I will file a separate BZ for that.

Here it is: https://bugzilla.redhat.com/show_bug.cgi?id=1623090

This was also reported upstream at https://community.theforeman.org/t/foreman-upgrade-1-18-0-1-18-1-failed/10954

Created redmine issue https://projects.theforeman.org/issues/24943 from this bug

Upstream bug assigned to ekohlvan

Upstream bug assigned to ekohlvan

Moving this bug to POST for triage into Satellite 6 since the upstream issue https://projects.theforeman.org/issues/24943 has been resolved.

snap 23, not 63

Verified!

@ Satelite 6.4 snap 23


Steps:
--------

1. On satellite 6.3, Updated the log level using 'satellite-installer -v --foreman-logging-level=debug'.
2. Also, Updated the /et/foreman/settings.yaml as:
:logging:
  :production:
    :type: file
    :layout: pattern
    :level: debug
3. Upgraded the sat 6.3 to 6.4 snap 23


Behavior:
-----------
1. The upgrade is completed successfully
2. Post-Upgrade, the settings yaml is updated / newly created with logging settings as:
:logging:
  :level: debug
  :production:
    :type: file
    :layout: pattern
3. The migrate foreman step is successful:

Upgrade Step: migrate_foreman...
/usr/share/foreman/lib/foreman.rb:8: warning: already initialized constant Foreman::UUID_REGEXP
/usr/share/foreman/lib/foreman.rb:8: warning: previous definition of UUID_REGEXP was here
/usr/share/foreman/lib/core_extensions.rb:182: warning: already initialized constant ActiveSupport::MessageEncryptor::DEFAULT_CIPHER
/opt/theforeman/tfm-ror51/root/usr/share/gems/gems/activesupport-5.1.6/lib/active_support/message_encryptor.rb:22: warning: previous definition of DEFAULT_CIPHER was here
API controllers newer than Apipie cache! Run apipie:cache rake task to regenerate cache.
== 20160302091113 ChangeEnvironmentPrior: migrating ===========================
-- add_column(:katello_environment_priors, :id, :primary_key)
   -> 0.1119s
== 20160302091113 ChangeEnvironmentPrior: migrated (0.1119s) ==================

== 20160705082036 CreateAnsibleRole: migrating ================================
-- adapter_name()
   -> 0.0000s
-- adapter_name()
   -> 0.0000s
-- create_table(:ansible_roles, {:id=>:integer})
   -> 0.0286s
== 20160705082036 CreateAnsibleRole: migrated (0.0287s) =======================

== 20160706074540 CreateJoinTableHostsAnsibleRoles: migrating =================
-- create_join_table(:ansible_roles, :hosts, {:column_options=>{:type=>:integer}})
   -> 0.0538s
== 20160706074540 CreateJoinTableHostsAnsibleRoles: migrated (0.0538s) ========

== 20160707195442 CreateHostAnsibleRoles: migrating ===========================
-- rename_table(:ansible_roles_hosts, :host_ansible_roles)
   -> 0.0085s
-- add_column(:host_ansible_roles, :id, :primary_key)
   -> 0.0875s
== 20160707195442 CreateHostAnsibleRoles: migrated (0.0961s) ==================

== 20160729094457 AddColumnsToAnsibleRole: migrating ==========================
-- add_column(:ansible_roles, :created_at, :datetime, {:default=>2018-09-25 08:40:50 UTC})
   -> 0.0285s
-- add_column(:ansible_roles, :updated_at, :datetime, {:default=>2018-09-25 08:40:50 UTC})
   -> 0.0250s
== 20160729094457 AddColumnsToAnsibleRole: migrated (0.0536s) =================

== 20160802153302 CreateJoinTableHostgroupAnsibleRoles: migrating =============
-- create_join_table(:hostgroup, :ansible_roles, {:table_name=>"hostgroup_ansible_roles", :column_options=>{:type=>:integer}})
   -> 0.0451s
== 20160802153302 CreateJoinTableHostgroupAnsibleRoles: migrated (0.0451s) ====

== 20160805094233 AddPrimaryKeyHostgroupAnsibleRoles: migrating ===============
-- add_column(:hostgroup_ansible_roles, :id, :primary_key)
   -> 0.0626s
== 20160805094233 AddPrimaryKeyHostgroupAnsibleRoles: migrated (0.0627s) ======

== 20160818062936 RenamePuppetMailNotifications: migrating ====================
== 20160818062936 RenamePuppetMailNotifications: migrated (0.0145s) ===========

== 20161122154057 AutomaticallySetRoleTimestamps: migrating ===================
-- change_column(:ansible_roles, :created_at, :datetime, {:null=>true, :default=>nil})
   -> 0.0028s
-- change_column(:ansible_roles, :updated_at, :datetime, {:null=>true, :default=>nil})
   -> 0.0024s
== 20161122154057 AutomaticallySetRoleTimestamps: migrated (0.0053s) ==========

== 20170109115157 FixLookupKeyAuditableType: migrating ========================
== 20170109115157 FixLookupKeyAuditableType: migrated (0.0172s) ===============

== 20170112175131 MigrateTemplateToParametersMacros: migrating ================
== 20170112175131 MigrateTemplateToParametersMacros: migrated (0.0465s) =======

== 20170404134531 AddDescriptionToSubnets: migrating ==========================
-- add_column(:subnets, :description, :text)
   -> 0.0008s
== 20170404134531 AddDescriptionToSubnets: migrated (0.0008s) =================

== 20170424131346 AddDescriptionToHostgroup: migrating ========================
-- add_column(:hostgroups, :description, :text)
   -> 0.0007s
== 20170424131346 AddDescriptionToHostgroup: migrated (0.0008s) ===============

== 20170508130316 AddVerifySslOptionToDockerRegistries: migrating =============
-- add_column(:docker_registries, :verify_ssl, :boolean, {:default=>true})
   -> 0.0373s
== 20170508130316 AddVerifySslOptionToDockerRegistries: migrated (0.0374s) ====

== 20170525112713 ChangeAuditedChangesInAudits: migrating =====================
== 20170525112713 ChangeAuditedChangesInAudits: migrated (0.0001s) ============

== 20170604082313 AddComputeResourceToHostgroup: migrating ====================
-- add_column(:hostgroups, :compute_resource_id, :integer)
   -> 0.0005s
-- add_foreign_key(:hostgroups, :compute_resources)
   -> 0.0024s
== 20170604082313 AddComputeResourceToHostgroup: migrated (0.0030s) ===========

== 20170610132326 CreatePersonalAccessTokens: migrating =======================
-- adapter_name()
   -> 0.0000s
-- adapter_name()
   -> 0.0000s
-- create_table(:personal_access_tokens, {:id=>:integer})
   -> 0.0857s
== 20170610132326 CreatePersonalAccessTokens: migrated (0.0859s) ==============

== 20170622011347 AddHttpProxies: migrating ===================================
-- adapter_name()
   -> 0.0000s
-- adapter_name()
   -> 0.0000s
-- create_table(:http_proxies, {:id=>:integer})
   -> 0.0511s
-- add_reference(:compute_resources, :http_proxy, {:type=>:integer, :index=>false})
   -> 0.0006s
== 20170622011347 AddHttpProxies: migrated (0.0519s) ==========================

== 20170714144125 AddDeb: migrating ===========================================
-- adapter_name()
   -> 0.0000s
-- adapter_name()
   -> 0.0000s
-- create_table("katello_debs", {:id=>:integer})
   -> 0.0422s
-- add_index(:katello_debs, :uuid, {:unique=>true})
   -> 0.0250s
-- add_index(:katello_debs, [:id, :uuid, :name, :version, :architecture, :version_sortable], {:name=>"katello_debs_fields_index"})
   -> 0.0252s
-- adapter_name()
   -> 0.0000s
-- adapter_name()
   -> 0.0000s
-- create_table("katello_repository_debs", {:id=>:integer})
   -> 0.0250s
-- add_index(:katello_repository_debs, [:deb_id, :repository_id], {:unique=>true})
   -> 0.0251s
-- add_foreign_key("katello_repository_debs", "katello_debs", {:column=>"deb_id"})
   -> 0.0012s
-- add_foreign_key("katello_repository_debs", "katello_repositories", {:column=>"repository_id"})
   -> 0.0019s
== 20170714144125 AddDeb: migrated (0.1460s) ==================================

== 20170720142145 AddDebReleaseParametersToRepositories: migrating ============
-- add_column(:katello_repositories, :deb_releases, :string, {:limit=>255})
   -> 0.0007s
-- add_column(:katello_repositories, :deb_components, :string, {:limit=>255})
   -> 0.0006s
-- add_column(:katello_repositories, :deb_architectures, :string, {:limit=>255})
   -> 0.0005s
== 20170720142145 AddDebReleaseParametersToRepositories: migrated (0.0019s) ===

== 20170815130257 AddIndexToSshKeys: migrating ================================
-- add_index(:ssh_keys, :user_id)
   -> 0.0239s
== 20170815130257 AddIndexToSshKeys: migrated (0.0239s) =======================

== 20170920211135 FixHostAuditableType: migrating =============================
== 20170920211135 FixHostAuditableType: migrated (0.0026s) ====================

== 20171010172724 AddDockerManifestList: migrating ============================
-- adapter_name()
   -> 0.0000s
-- adapter_name()
   -> 0.0000s
-- create_table(:katello_docker_manifest_lists, {:id=>:integer})
   -> 0.0373s
-- adapter_name()
   -> 0.0000s
-- adapter_name()
   -> 0.0000s
-- create_table(:katello_repository_docker_manifest_lists, {:id=>:integer})
   -> 0.0251s
-- adapter_name()
   -> 0.0000s
-- adapter_name()
   -> 0.0000s
-- create_table(:katello_docker_manifest_list_manifests, {:id=>:integer})
   -> 0.0250s
-- remove_foreign_key(:katello_docker_tags, {:column=>:docker_manifest_id})
   -> 0.0044s
-- rename_column(:katello_docker_tags, :docker_manifest_id, :docker_taggable_id)
   -> 0.0024s
-- add_column(:katello_docker_tags, :docker_taggable_type, :string, {:limit=>255, :default=>"Katello::DockerManifest"})
   -> 0.0773s
-- add_index(:katello_docker_tags, [:docker_taggable_id, :docker_taggable_type], {:name=>"docker_taggable_type_index"})
   -> 0.0246s
-- add_index(:katello_repository_docker_manifest_lists, [:docker_manifest_list_id, :repository_id], {:name=>:katello_repo_docker_manifest_list_repo_id, :unique=>true})
   -> 0.0167s
-- add_index(:katello_docker_manifest_list_manifests, [:docker_manifest_list_id, :docker_manifest_id], {:name=>:katello_docker_manifest_lisst_manifest, :unique=>true})
   -> 0.0168s
-- add_foreign_key(:katello_repository_docker_manifest_lists, :katello_repositories, {:column=>:repository_id})
   -> 0.0014s
-- add_foreign_key(:katello_docker_manifest_list_manifests, :katello_docker_manifests, {:column=>:docker_manifest_id})
   -> 0.0013s
== 20171010172724 AddDockerManifestList: migrated (0.2330s) ===================

== 20171014051810 RemoveDockerManifestName: migrating =========================
-- remove_column(:katello_docker_manifests, :name)
   -> 0.0005s
== 20171014051810 RemoveDockerManifestName: migrated (0.0006s) ================

== 20171016125613 AddContentTitleUniqueConstraint: migrating ==================
-- index_exists?(:foreman_openscap_scap_contents, :title, {:name=>"index_scaptimony_scap_contents_on_title"})
   -> 0.0022s
-- index_name(:foreman_openscap_scap_contents, {:name=>"index_scaptimony_scap_contents_on_title"})
   -> 0.0000s
-- index_name_exists?(:foreman_openscap_scap_contents, "index_scaptimony_scap_contents_on_title")
   -> 0.0007s
-- remove_index(:foreman_openscap_scap_contents, {:name=>"index_scaptimony_scap_contents_on_title"})
   -> 0.0006s
-- add_index(:foreman_openscap_scap_contents, :title, {:unique=>true})
   -> 0.0185s
== 20171016125613 AddContentTitleUniqueConstraint: migrated (0.0445s) =========

== 20171016202300 IncreaseFactValueSize: migrating ============================
-- change_column(:fact_values, :value, :text, {:limit=>16777215})
   -> 0.0007s
== 20171016202300 IncreaseFactValueSize: migrated (0.0008s) ===================

== 20171025163149 RemoveUsePulpOauthSetting: migrating ========================
== 20171025163149 RemoveUsePulpOauthSetting: migrated (0.0015s) ===============

== 20171026082635 AddTaskAction: migrating ====================================
-- add_column(:foreman_tasks_tasks, :action, :string)
   -> 0.0007s
== 20171026082635 AddTaskAction: migrated (0.0008s) ===========================

== 20171110082124 AddSslCertsToProductsAndRepos: migrating ====================
-- add_column(:katello_products, :ssl_ca_cert_id, :integer, {:null=>true})
   -> 0.0006s
-- add_index(:katello_products, :ssl_ca_cert_id)
   -> 0.0175s
-- add_foreign_key(:katello_products, :katello_gpg_keys, {:name=>"katello_products_ssl_ca_cert_id_fk", :column=>:ssl_ca_cert_id, :primary_key=>:id})
   -> 0.0015s
-- add_column(:katello_products, :ssl_client_cert_id, :integer, {:null=>true})
   -> 0.0005s
-- add_index(:katello_products, :ssl_client_cert_id)
   -> 0.0314s
-- add_foreign_key(:katello_products, :katello_gpg_keys, {:name=>"katello_products_ssl_client_cert_id_fk", :column=>:ssl_client_cert_id, :primary_key=>:id})
   -> 0.0013s
-- add_column(:katello_products, :ssl_client_key_id, :integer, {:null=>true})
   -> 0.0005s
-- add_index(:katello_products, :ssl_client_key_id)
   -> 0.0233s
-- add_foreign_key(:katello_products, :katello_gpg_keys, {:name=>"katello_products_ssl_client_key_id_fk", :column=>:ssl_client_key_id, :primary_key=>:id})
   -> 0.0012s
-- add_column(:katello_repositories, :ssl_ca_cert_id, :integer, {:null=>true})
   -> 0.0006s
-- add_index(:katello_repositories, :ssl_ca_cert_id)
   -> 0.0232s
-- add_foreign_key(:katello_repositories, :katello_gpg_keys, {:name=>"katello_repositories_ssl_ca_cert_id_fk", :column=>:ssl_ca_cert_id, :primary_key=>:id})
   -> 0.0013s
-- add_column(:katello_repositories, :ssl_client_cert_id, :integer, {:null=>true})
   -> 0.0006s
-- add_index(:katello_repositories, :ssl_client_cert_id)
   -> 0.0231s
-- add_foreign_key(:katello_repositories, :katello_gpg_keys, {:name=>"katello_repositories_ssl_client_cert_id_fk", :column=>:ssl_client_cert_id, :primary_key=>:id})
   -> 0.0013s
-- add_column(:katello_repositories, :ssl_client_key_id, :integer, {:null=>true})
   -> 0.0005s
-- add_index(:katello_repositories, :ssl_client_key_id)
   -> 0.0232s
-- add_foreign_key(:katello_repositories, :katello_gpg_keys, {:name=>"katello_repositories_ssl_client_key_id_fk", :column=>:ssl_client_key_id, :primary_key=>:id})
   -> 0.0013s
-- add_column(:katello_gpg_keys, :content_type, :string, {:null=>false, :default=>"gpg_key", :limit=>255})
   -> 0.0579s
== 20171110082124 AddSslCertsToProductsAndRepos: migrated (0.2403s) ===========

== 20171112174357 CreateKatelloContent: migrating =============================
-- adapter_name()
   -> 0.0000s
-- adapter_name()
   -> 0.0000s
-- create_table(:katello_contents, {:id=>:integer})
   -> 0.2097s
== 20171112174357 CreateKatelloContent: migrated (0.2098s) ====================

== 20171112174358 CreateKatelloProductContent: migrating ======================
-- adapter_name()
   -> 0.0000s
-- adapter_name()
   -> 0.0000s
-- create_table(:katello_product_contents, {:id=>:integer})
   -> 0.0431s
-- add_foreign_key("katello_product_contents", "katello_products", {:name=>"katello_product_content_product_id_fk", :column=>"product_id"})
   -> 0.0015s
-- add_foreign_key("katello_product_contents", "katello_contents", {:name=>"katello_product_content_content_id_fk", :column=>"content_id"})
   -> 0.0012s
== 20171112174358 CreateKatelloProductContent: migrated (0.0461s) =============

== 20171119094913 AddMtuToSubnet: migrating ===================================
-- add_column(:subnets, :mtu, :integer, {:default=>1500, :null=>false, :limit=>8})
   -> 0.0600s
== 20171119094913 AddMtuToSubnet: migrated (0.0601s) ==========================

== 20171121082256 UpdateCentosInstallationMedia: migrating ====================
== 20171121082256 UpdateCentosInstallationMedia: migrated (0.0062s) ===========

== 20171121111333 ChangeAuthSourceResourceType: migrating =====================
== 20171121111333 ChangeAuthSourceResourceType: migrated (0.0063s) ============

== 20171126131104 RemoveDuplicateTaxableTaxonomies: migrating =================
-- index_name(:taxable_taxonomies, {:name=>"taxable_index"})
   -> 0.0000s
-- index_name_exists?(:taxable_taxonomies, "taxable_index")
   -> 0.0009s
-- remove_index(:taxable_taxonomies, {:name=>"taxable_index"})
   -> 0.0004s
-- add_index(:taxable_taxonomies, [:taxable_id, :taxable_type, :taxonomy_id], {:name=>"taxable_index", :unique=>true})
   -> 0.0308s
== 20171126131104 RemoveDuplicateTaxableTaxonomies: migrated (0.0410s) ========

== 20171129103615 AddSecretsToJobInvocations: migrating =======================
-- add_column(:job_invocations, :password, :string)
   -> 0.0007s
-- add_column(:job_invocations, :key_passphrase, :string)
   -> 0.0004s
== 20171129103615 AddSecretsToJobInvocations: migrated (0.0011s) ==============

== 20171208113210 RemoveUseGravatarSetting: migrating =========================
== 20171208113210 RemoveUseGravatarSetting: migrated (0.0011s) ================

== 20171214050230 AddAutoPublishToContentViews: migrating =====================
-- add_column(:katello_content_views, :auto_publish, :boolean, {:null=>false, :default=>false})
   -> 0.1172s
== 20171214050230 AddAutoPublishToContentViews: migrated (0.1173s) ============

== 20171222120314 AddConstraintsOnDiscoveryRulesHostgroups: migrating =========
-- add_foreign_key("discovery_rules", "hostgroups", {:name=>"discovery_rules_hostgroup_id_fk", :column=>"hostgroup_id"})
   -> 0.0016s
== 20171222120314 AddConstraintsOnDiscoveryRulesHostgroups: migrated (0.0188s) 
== 20171225122601 AddVersionToAuditableIndex: migrating =======================
-- index_name(:audits, {:name=>"auditable_index"})
   -> 0.0000s
-- index_name_exists?(:audits, "auditable_index")
   -> 0.0008s
-- remove_index(:audits, {:name=>"auditable_index"})
   -> 0.0006s
-- add_index(:audits, [:auditable_id, :auditable_type, :version], {:name=>"auditable_index"})
   -> 0.0221s
== 20171225122601 AddVersionToAuditableIndex: migrated (0.0237s) ==============

== 20171231134017 ChangeVlanToInt: migrating ==================================
-- change_column(:subnets, :vlanid, :integer, {:using=>"vlanid::integer"})
   -> 0.0596s
== 20171231134017 ChangeVlanToInt: migrated (0.0658s) =========================

== 20180102082705 AddTaxonomyIndexToHosts: migrating ==========================
-- add_index(:hosts, [:type, :organization_id, :location_id])
   -> 0.0231s
-- add_index(:hosts, [:type, :location_id])
   -> 0.0252s
== 20180102082705 AddTaxonomyIndexToHosts: migrated (0.0483s) =================

== 20180105090316 RemoveDockerSearchPermission: migrating =====================
== 20180105090316 RemoveDockerSearchPermission: migrated (0.0477s) ============

== 20180110104432 RenameTemplateInvocationPermission: migrating ===============
== 20180110104432 RenameTemplateInvocationPermission: migrated (0.0180s) ======

== 20180111130853 AddConfigReportsOrigin: migrating ===========================
-- add_column(:reports, :origin, :string)
   -> 0.0008s
== 20180111130853 AddConfigReportsOrigin: migrated (0.0009s) ==================

== 20180119205740 ChangeUserTimezoneEmptyToNil: migrating =====================
== 20180119205740 ChangeUserTimezoneEmptyToNil: migrated (0.0102s) ============

== 20180123140634 RemoveLimitLdapFilter: migrating ============================
-- change_column("auth_sources", :ldap_filter, :text, {:limit=>nil})
   -> 0.0006s
== 20180123140634 RemoveLimitLdapFilter: migrated (0.0007s) ===================

== 20180202072115 AddNotificationBuilderToRemoteExecutionFeature: migrating ===
-- add_column(:remote_execution_features, :notification_builder, :string)
   -> 0.0007s
== 20180202072115 AddNotificationBuilderToRemoteExecutionFeature: migrated (0.0007s) 
== 20180202123215 AddFeatureIdToJobInvocation: migrating ======================
-- add_column(:job_invocations, :remote_execution_feature_id, :integer, {:index=>true})
   -> 0.0007s
-- add_foreign_key(:job_invocations, :remote_execution_features, {:column=>:remote_execution_feature_id})
   -> 0.0016s
== 20180202123215 AddFeatureIdToJobInvocation: migrated (0.0024s) =============

== 20180207150921 AddRemoteTasks: migrating ===================================
-- adapter_name()
   -> 0.0000s
-- adapter_name()
   -> 0.0000s
-- create_table(:foreman_tasks_remote_tasks, {:id=>:integer})
   -> 0.0558s
== 20180207150921 AddRemoteTasks: migrated (0.0560s) ==========================

== 20180207232901 AddTriggeredByToContentViewHistory: migrating ===============
-- add_column(:katello_content_view_histories, :triggered_by_id, :integer, {:null=>true})
   -> 0.0007s
-- add_foreign_key("katello_content_view_histories", "katello_content_view_versions", {:name=>"katello_cv_history_versions_triggered_by_fk", :column=>"triggered_by_id"})
   -> 0.0015s
== 20180207232901 AddTriggeredByToContentViewHistory: migrated (0.0023s) ======

== 20180208053256 CreateTablePreferences: migrating ===========================
-- create_table(:table_preferences)
   -> 0.0467s
-- add_foreign_key(:table_preferences, :users, {:name=>"table_preferences_user_id_fk"})
   -> 0.0013s
-- add_index(:table_preferences, [:user_id, :name], {:unique=>true})
   -> 0.0320s
== 20180208053256 CreateTablePreferences: migrated (0.0801s) ==================

== 20180216092715 UseUuid: migrating ==========================================
-- change_table(:foreman_tasks_tasks, {})
   -> 0.6344s
-- change_table(:foreman_tasks_task_group_members, {})
   -> 0.0588s
-- change_table(:foreman_tasks_locks, {})
   -> 0.1681s
== 20180216092715 UseUuid: migrated (0.8614s) =================================

== 20180216094550 AddTemplateToSubnets: migrating =============================
-- add_column(:subnets, :template_id, :integer)
   -> 0.0006s
== 20180216094550 AddTemplateToSubnets: migrated (0.0022s) ====================

== 20180218191449 AddManifestRefreshedAtToOrganization: migrating =============
-- add_column(:taxonomies, :manifest_refreshed_at, :datetime)
   -> 0.0006s
== 20180218191449 AddManifestRefreshedAtToOrganization: migrated (0.0007s) ====

== 20180226095631 ChangeTaskIdToUuid: migrating ===============================
-- change_table(:job_invocations, {})
   -> 0.1088s
-- change_table(:template_invocations, {})
   -> 0.0836s
== 20180226095631 ChangeTaskIdToUuid: migrated (0.1925s) ======================

== 20180227094827 UseUuidForTaskId: migrating =================================
-- change_table(:katello_content_view_histories, {})
   -> 0.0998s
== 20180227094827 UseUuidForTaskId: migrated (0.0998s) ========================

== 20180228132500 RenameTrustedHosts: migrating ===============================
== 20180228132500 RenameTrustedHosts: migrated (0.0075s) ======================

== 20180312080251 ChangeDigestsLimit: migrating ===============================
-- change_column(:messages, :digest, :string, {:limit=>40})
   -> 0.1007s
-- change_column(:sources, :digest, :string, {:limit=>40})
   -> 0.1003s
== 20180312080251 ChangeDigestsLimit: migrated (0.2012s) ======================

== 20180323175122 AddRegistryNamePatternToEnvironment: migrating ==============
-- add_column(:katello_environments, :registry_name_pattern, :string, {:limit=>255, :null=>true})
   -> 0.0006s
-- change_column(:katello_repositories, :container_repository_name, :string, {:unique=>true})
   -> 0.0006s
== 20180323175122 AddRegistryNamePatternToEnvironment: migrated (0.0013s) =====

== 20180326145716 SubscriptionCpIdChange: migrating ===========================
-- remove_column(:katello_subscriptions, :cp_id)
   -> 0.0007s
-- rename_column(:katello_subscriptions, :product_id, :cp_id)
   -> 0.0015s
== 20180326145716 SubscriptionCpIdChange: migrated (0.0023s) ==================

== 20180326150339 MoveStackingId: migrating ===================================
-- add_column(:katello_pools, :stacking_id, :string)
   -> 0.0006s
-- remove_column(:katello_subscriptions, :stacking_id)
   -> 0.0005s
== 20180326150339 MoveStackingId: migrated (0.0375s) ==========================

== 20180326190408 MoveSubscriptionProducts: migrating =========================
-- create_table("katello_pool_products")
   -> 0.0663s
-- add_index("katello_pool_products", [:pool_id, :product_id], {:unique=>true, :name=>"index_katello_pool_products_on_subs_id_prod_id"})
   -> 0.0250s
-- add_foreign_key("katello_pool_products", "katello_pools", {:name=>"katello_pool_products_pool_id_fk", :column=>"pool_id"})
   -> 0.0014s
-- add_foreign_key("katello_pool_products", "katello_products", {:name=>"katello_pool_products_product_id_fk", :column=>"product_id"})
   -> 0.0012s
-- drop_table("katello_subscription_products")
   -> 0.0017s
== 20180326190408 MoveSubscriptionProducts: migrated (0.1990s) ================

== 20180402160223 CleanUpForcePostSyncActionSetting: migrating ================
== 20180402160223 CleanUpForcePostSyncActionSetting: migrated (0.0010s) =======

== 20180403144853 ConvertVmAttrsToHash: migrating =============================
-- Starting serialized attributes conversion, this can take long time based on data amount
-- Converting Nics, total: 19
-- Converting Compute Attributes, total: 0
-- Converting Compute Resources, total 0
-- Converting Lookup Keys, total: 0
-- Converting Lookup Values, total: 0
-- All conversions finished
== 20180403144853 ConvertVmAttrsToHash: migrated (0.0684s) ====================

== 20180404082603 RemoveV3FromOvirtCrUrl: migrating ===========================
== 20180404082603 RemoveV3FromOvirtCrUrl: migrated (0.0019s) ==================

== 20180410125416 RenameAnsibleJobCategories: migrating =======================
== 20180410125416 RenameAnsibleJobCategories: migrated (0.0336s) ==============

== 20180410140909 AddOrganizationIdToPool: migrating ==========================
-- add_column(:katello_pools, :organization_id, :integer)
   -> 0.0010s
-- add_foreign_key("katello_pools", "taxonomies", {:name=>"katello_pools_organization_id", :column=>"organization_id"})
   -> 0.0014s
== 20180410140909 AddOrganizationIdToPool: migrated (0.0907s) =================

== 20180411160809 AddSudoPasswordToJobInvocation: migrating ===================
-- add_column(:job_invocations, :sudo_password, :string)
   -> 0.0007s
== 20180411160809 AddSudoPasswordToJobInvocation: migrated (0.0008s) ==========

== 20180412124505 AddPriorityScoreToDiscoveryRules: migrating =================
-- change_column(:discovery_rules, :priority, :integer, {:null=>false})
   -> 0.0012s
-- add_index(:discovery_rules, :priority, {:unique=>true})
   -> 0.0300s
== 20180412124505 AddPriorityScoreToDiscoveryRules: migrated (0.0399s) ========

== 20180417031215 AddIgnorableContentToRepository: migrating ==================
-- add_column(:katello_repositories, :ignorable_content, :text, {:null=>true})
   -> 0.0007s
== 20180417031215 AddIgnorableContentToRepository: migrated (0.0008s) =========

== 20180516103339 UpdateIdmParams: migrating ==================================
== 20180516103339 UpdateIdmParams: migrated (0.0030s) =========================

== 20180521200848 DropRepoRpmsTimestampNotNull: migrating =====================
-- change_column(:katello_repository_docker_manifest_lists, :created_at, :datetime, {:null=>true})
   -> 0.0007s
-- change_column(:katello_repository_docker_manifest_lists, :updated_at, :datetime, {:null=>true})
   -> 0.0006s
-- change_column(:katello_repository_docker_manifests, :created_at, :datetime, {:null=>true})
   -> 0.0008s
-- change_column(:katello_repository_docker_manifests, :updated_at, :datetime, {:null=>true})
   -> 0.0006s
-- change_column(:katello_repository_errata, :created_at, :datetime, {:null=>true})
   -> 0.0008s
-- change_column(:katello_repository_errata, :updated_at, :datetime, {:null=>true})
   -> 0.0006s
-- change_column(:katello_repository_files, :created_at, :datetime, {:null=>true})
   -> 0.0008s
-- change_column(:katello_repository_files, :updated_at, :datetime, {:null=>true})
   -> 0.0006s
-- change_column(:katello_repository_ostree_branches, :created_at, :datetime, {:null=>true})
   -> 0.0007s
-- change_column(:katello_repository_ostree_branches, :updated_at, :datetime, {:null=>true})
   -> 0.0006s
-- change_column(:katello_repository_package_groups, :created_at, :datetime, {:null=>true})
   -> 0.0007s
-- change_column(:katello_repository_package_groups, :updated_at, :datetime, {:null=>true})
   -> 0.0005s
-- change_column(:katello_repository_puppet_modules, :created_at, :datetime, {:null=>true})
   -> 0.0007s
-- change_column(:katello_repository_puppet_modules, :updated_at, :datetime, {:null=>true})
   -> 0.0007s
-- change_column(:katello_repository_rpms, :created_at, :datetime, {:null=>true})
   -> 0.0007s
-- change_column(:katello_repository_rpms, :updated_at, :datetime, {:null=>true})
   -> 0.0006s
-- change_column(:katello_repository_srpms, :created_at, :datetime, {:null=>true})
   -> 0.0008s
-- change_column(:katello_repository_srpms, :updated_at, :datetime, {:null=>true})
   -> 0.0007s
== 20180521200848 DropRepoRpmsTimestampNotNull: migrated (0.0129s) ============

== 20180605150932 DeprecateSatelliteTemplates: migrating ======================
== 20180605150932 DeprecateSatelliteTemplates: migrated (0.2912s) =============

== 20180612163403 AddForeignKeyToHypervisorId: migrating ======================
-- add_foreign_key(:katello_pools, :hosts, {:name=>"katello_pools_hypervisor_fk", :column=>"hypervisor_id"})
   -> 0.0017s
== 20180612163403 AddForeignKeyToHypervisorId: migrated (0.0065s) =============

== 20180612164926 AddContentOrgId: migrating ==================================
-- add_column(:katello_contents, :organization_id, :integer, {:null=>true})
   -> 0.0010s
-- add_foreign_key(:katello_contents, :taxonomies, {:column=>:organization_id, :primary_key=>:id})
   -> 0.0014s
-- change_column(:katello_contents, :organization_id, :integer, {:null=>false})
   -> 0.0017s
== 20180612164926 AddContentOrgId: migrated (0.0198s) =========================

== 20180612165011 RemoveContentFieldsFromHost: migrating ======================
-- remove_column(:hosts, :content_view_id)
   -> 0.0007s
-- remove_column(:hosts, :lifecycle_environment_id)
   -> 0.0005s
== 20180612165011 RemoveContentFieldsFromHost: migrated (0.0013s) =============

== 20180626160422 AddUpstreamPoolIdToKatelloPool: migrating ===================
-- add_column(:katello_pools, :upstream_pool_id, :string)
   -> 0.0006s
== 20180626160422 AddUpstreamPoolIdToKatelloPool: migrated (0.0006s) ==========

== 20180724062531 ChangeOutOfSyncDefault: migrating ===========================
== 20180724062531 ChangeOutOfSyncDefault: migrated (0.0052s) ==================

== 20180724152638 AdjustPuppetOutOfSyncInterval: migrating ====================
== 20180724152638 AdjustPuppetOutOfSyncInterval: migrated (0.0040s) ===========

== 20180725083800 ChangeRepositoryPasswordLength: migrating ===================
-- change_column(:katello_repositories, :upstream_password, :string, {:limit=>1024})
   -> 0.0009s
== 20180725083800 ChangeRepositoryPasswordLength: migrated (0.0009s) ==========

== 20180806151925 AddSubnetNameUniqueConstraint: migrating ====================
-- Deduplicating the following subnet names: []
   -> 0.0000s
-- add_index(:subnets, :name, {:unique=>true, :length=>{:name=>255}})
   -> 0.0160s
== 20180806151925 AddSubnetNameUniqueConstraint: migrated (0.0204s) ===========

== 20180816134832 CastLookupKeyValues: migrating ==============================
== 20180816134832 CastLookupKeyValues: migrated (0.0051s) =====================

== 20180917173645 AddSourceRepoChecksumTypeToKatelloRepositories: migrating ===
-- add_column(:katello_repositories, :source_repo_checksum_type, :string)
   -> 0.0006s
== 20180917173645 AddSourceRepoChecksumTypeToKatelloRepositories: migrated (0.0006s) 
== 20180918135943 ChangeDefaultPxeItems: migrating ============================
== 20180918135943 ChangeDefaultPxeItems: migrated (0.0073s) ===================

foreman-rake db:migrate finished successfully!

(In reply to Jitendra Yejare from comment #23)
> Verified!
> 
> @ Satelite 6.4 snap 23
> 
> 
> Steps:
> --------
> 
> 1. On satellite 6.3, Updated the log level using 'satellite-installer -v
> --foreman-logging-level=debug'.
> 2. Also, Updated the /et/foreman/settings.yaml as:
> :logging:
>   :production:
>     :type: file
>     :layout: pattern
>     :level: debug
> 3. Upgraded the sat 6.3 to 6.4 snap 23

It sounds like you're missing a critical step here. You need to make a modification to /etc/foreman/logging.yaml so during the upgrade it won't replace the file. Before this patch it would error out, with it /etc/foreman/logging.yaml should no longer be used.

Verified again!

@ Satellite 6.4 snap 23


Steps:
1. On satellite 6.3, Updated the log level using 'satellite-installer -v --foreman-logging-level=debug'.
2. Also, Updated the /et/foreman/settings.yaml as:
:logging:
  :production:
    :type: file
    :layout: pattern
    :level: debug
3. Updated the /etc/foreman/logging.yaml by adding comment line
4. Upgraded the sat 6.3 to 6.4 snap 23 using foreman maintain


Observation:
1. An upgrade to 6.4 is completed successfully including the migrate foreman step.
2. Post-Upgrade, the settings yaml is updated / newly created with logging settings as:
:logging:
  :level: debug
  :production:
    :type: file
    :layout: pattern
3. Post-upgrade the /etc/foreman/logging.rpmsave can be seen having the changed I made before the upgrade in logging.yaml
4. The /etc/foreman/logging.yaml not exists post upgrade.
4. The logging.yaml is now a file instead of link in /usr/share as :
# ll /usr/share/foreman/config/logging.yaml instead of a link to /etc/foreman/logging.yaml
-rw-r--r--. 1 root root 4033 Sep 21 15:40 /usr/share/foreman/config/logging.yaml

Changing the state to Verified !

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2018:2927