Bug 1649961

Summary: Corrupt Roles after upgrade to 6.4
Product: Red Hat Satellite Reporter: Jason Dickerson <jdickers>
Component: Satellite MaintainAssignee: Anurag Patel <apatel>
Status: CLOSED ERRATA QA Contact: Mirek Długosz <mzalewsk>
Severity: medium Docs Contact:
Priority: unspecified    
Version: 6.4CC: apatel, egolov, ehelms, inecas, jdickers, kgaikwad, lzap, mbacovsk, mhulan, mzalewsk, oprazak, zhunting
Target Milestone: 6.5.0Keywords: Triaged, Upgrades
Target Release: Unused   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: rubygem-foreman_maintain-0.3.2 Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2019-05-14 12:38:50 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
migration to corrupt filters none

Description Jason Dickerson 2018-11-15 00:02:28 UTC 
Description of problem:

There may be other permissions that have moved from one filter to another, but the ones effecting this customer are:

view_arf_reports moved from "(Miscellaneous)" to "Satellite openscap/arf report"

view_subscriptions moved from "Organization" to "Subscriptions"

Any role created prior to upgrade to 6.4, which contains a permission that has moved filters, will be "corrupt".  You cannot save changes to the role, until the bad filters are removed and recreated correctly, and add the new filters where the permissions currently reside, post 6.4 upgrade.


Version-Release number of selected component (if applicable):

Satellite 6.4


How reproducible:

Consistently

Steps to Reproduce:
1. On Satellite 6.3 create a custom role with the permission view_arf_reports from "(Miscellaneous)" and the permission view_arf_reports from 
"(Miscellaneous)".  
2. Upgrade to Satellite 6.4
3. Try to edit the custom role.

Actual results:
You will receive errors

Expected results:
The changes to the role will be saved.

Additional info:
Comment 2 Marek Hulan 2018-11-15 08:37:54 UTC 
The moving of the permission is correct, but I suppose the original filter remains existing without any permissions? Which makes it invalid? Or what exactly is the error? view_art_reports should be now linked to ArfReport resource.
Comment 3 Jason Dickerson 2018-11-15 15:40:23 UTC 
If you go into the "bad" filter and just submit it without making changes you get:

must be of same resource type (Organization,Katello::Subscription) - Role (Server Engineer)

Also if I alter something on the Role Tab and try to Submit, I get this:

Unable to save
One or more of the associated filters are invalid which prevented the role to be saved

Please let me know if you need more information
Comment 5 Jason Dickerson 2018-11-15 15:51:50 UTC 
Here are the 2 errors in the production.log

2018-11-15T15:36:21 [I|app|6cbbb] Processing by FiltersController#update as HTML
2018-11-15T15:36:21 [I|app|6cbbb]   Parameters: {"utf8"=>"✓", "authenticity_token"=>"6OrJ4EQSm4aJXjW0glnldc8UduLOYJ/D9Q5cn+AiZ0292bPxi4tAFOKhp67J/uGuJrnElX9ViqnHt/xrMc1/qQ==", "filter"=>{"role_id"=>"22", "resource_type"=>"Organization", "permission_ids"=>["212", "", "103", "99"], "override"=>"0", "unlimited"=>"1", "location_ids"=>[""], "organization_ids"=>[""]}, "commit"=>"Submit", "id"=>"189"}
2018-11-15T15:36:21 [I|app|6cbbb] Current user: jadic1 (administrator)
2018-11-15T15:36:21 [E|app|6cbbb] Failed to save: Permissions must be of same resource type (Organization,Katello::Subscription) - Role (Server Engineer)
2018-11-15T15:36:21 [I|app|6cbbb]   Rendering filters/edit.html.erb within layouts/application
2018-11-15T15:36:21 [I|app|6cbbb]   Rendered filters/_form.html.erb (26.6ms)
2018-11-15T15:36:21 [I|app|6cbbb]   Rendered filters/edit.html.erb within layouts/application (27.8ms)
2018-11-15T15:36:21 [I|app|6cbbb]   Rendered layouts/_application_content.html.erb (0.3ms)
2018-11-15T15:36:21 [I|app|6cbbb]   Rendering layouts/base.html.erb
2018-11-15T15:36:21 [I|app|6cbbb]   Rendered home/_organization_dropdown.html.erb (1.8ms)
2018-11-15T15:36:21 [I|app|6cbbb]   Rendered home/_location_dropdown.html.erb (2.0ms)
2018-11-15T15:36:21 [I|app|6cbbb]   Rendered home/_org_switcher.html.erb (4.4ms)
2018-11-15T15:36:21 [I|app|6cbbb]   Rendered home/_user_dropdown.html.erb (1.1ms)
2018-11-15T15:36:21 [I|app|6cbbb]   Rendered home/_topbar.html.erb (6.2ms)
2018-11-15T15:36:21 [I|app|6cbbb]   Rendered home/_vertical_menu.html.erb (2.4ms)
2018-11-15T15:36:21 [I|app|6cbbb]   Rendered home/_vertical_menu.html.erb (2.7ms)
2018-11-15T15:36:21 [I|app|6cbbb]   Rendered home/_vertical_menu.html.erb (0.9ms)
2018-11-15T15:36:21 [I|app|6cbbb]   Rendered home/_vertical_menu.html.erb (4.0ms)
2018-11-15T15:36:21 [I|app|6cbbb]   Rendered home/_vertical_menu.html.erb (2.4ms)
2018-11-15T15:36:21 [I|app|6cbbb]   Rendered home/_vertical_menu.html.erb (1.9ms)
2018-11-15T15:36:21 [I|app|6cbbb]   Rendered home/_vertical_menu.html.erb (1.3ms)
2018-11-15T15:36:21 [I|app|6cbbb]   Rendered home/_vertical_menu.html.erb (2.5ms)
2018-11-15T15:36:21 [I|app|6cbbb]   Rendered home/_vertical_taxonomies.html.erb (2.0ms)
2018-11-15T15:36:21 [I|app|6cbbb]   Rendered home/_vertical_taxonomies.html.erb (1.9ms)
2018-11-15T15:36:21 [I|app|6cbbb]   Rendered home/_vertical_menu.html.erb (0.8ms)
2018-11-15T15:36:21 [I|app|6cbbb]   Rendered home/_navbar.html.erb (26.7ms)
2018-11-15T15:36:21 [I|app|6cbbb]   Rendered layouts/base.html.erb (36.2ms)
2018-11-15T15:36:21 [I|app|6cbbb] Completed 200 OK in 100ms (Views: 62.6ms | ActiveRecord: 9.4ms)


2018-11-15T15:38:37 [I|app|ee8f8] Processing by RolesController#update as HTML
2018-11-15T15:38:37 [I|app|ee8f8]   Parameters: {"utf8"=>"✓", "authenticity_token"=>"lEPt9yyxEdNajrqY+6L0pa5zCejkhGS9ivaldNryn03BcJfm4yjKQTFxKIKwBfB+R967n1Wxcde4TwWACx2HqQ==", "role"=>{"name"=>"Server Engineer", "description"=>"", "location_ids"=>["", "2"], "organization_ids"=>["", "1"]}, "commit"=>"Submit", "id"=>"22-Server Engineer"}
2018-11-15T15:38:37 [I|app|ee8f8] Current user: jadic1 (administrator)
2018-11-15T15:38:37 [I|aud|ee8f8] update event for Role with id 22
2018-11-15T15:38:37 [I|aud|ee8f8] update event for Filter with id 167
2018-11-15T15:38:37 [I|aud|ee8f8] update event for Filter with id 168
2018-11-15T15:38:37 [I|aud|ee8f8] update event for Filter with id 170
2018-11-15T15:38:37 [I|aud|ee8f8] update event for Filter with id 172
2018-11-15T15:38:37 [I|aud|ee8f8] update event for Filter with id 174
2018-11-15T15:38:37 [I|aud|ee8f8] update event for Filter with id 175
2018-11-15T15:38:37 [I|aud|ee8f8] update event for Filter with id 176
2018-11-15T15:38:37 [I|aud|ee8f8] update event for Filter with id 177
2018-11-15T15:38:37 [I|aud|ee8f8] update event for Filter with id 178
2018-11-15T15:38:37 [I|aud|ee8f8] update event for Filter with id 179
2018-11-15T15:38:38 [I|aud|ee8f8] update event for Filter with id 180
2018-11-15T15:38:38 [I|aud|ee8f8] update event for Filter with id 181
2018-11-15T15:38:38 [I|aud|ee8f8] update event for Filter with id 183
2018-11-15T15:38:38 [I|aud|ee8f8] update event for Filter with id 184
2018-11-15T15:38:38 [I|aud|ee8f8] update event for Filter with id 185
2018-11-15T15:38:38 [I|aud|ee8f8] update event for Filter with id 187
2018-11-15T15:38:38 [E|app|ee8f8] Failed to save: One or more of the associated filters are invalid which prevented the role to be saved
2018-11-15T15:38:38 [I|app|ee8f8]   Rendering roles/edit.html.erb within layouts/application
2018-11-15T15:38:38 [I|app|ee8f8]   Rendered roles/_form.html.erb (16.0ms)
2018-11-15T15:38:38 [I|app|ee8f8]   Rendered roles/edit.html.erb within layouts/application (18.1ms)
2018-11-15T15:38:38 [I|app|ee8f8]   Rendered layouts/_application_content.html.erb (0.4ms)
2018-11-15T15:38:38 [I|app|ee8f8]   Rendering layouts/base.html.erb
2018-11-15T15:38:38 [I|app|ee8f8]   Rendered home/_organization_dropdown.html.erb (1.9ms)
2018-11-15T15:38:38 [I|app|ee8f8]   Rendered home/_location_dropdown.html.erb (1.9ms)
2018-11-15T15:38:38 [I|app|ee8f8]   Rendered home/_org_switcher.html.erb (4.3ms)
2018-11-15T15:38:38 [I|app|ee8f8]   Rendered home/_user_dropdown.html.erb (1.1ms)
2018-11-15T15:38:38 [I|app|ee8f8]   Rendered home/_topbar.html.erb (6.2ms)
2018-11-15T15:38:38 [I|app|ee8f8]   Rendered home/_vertical_menu.html.erb (2.5ms)
2018-11-15T15:38:38 [I|app|ee8f8]   Rendered home/_vertical_menu.html.erb (2.7ms)
2018-11-15T15:38:38 [I|app|ee8f8]   Rendered home/_vertical_menu.html.erb (0.9ms)
2018-11-15T15:38:38 [I|app|ee8f8]   Rendered home/_vertical_menu.html.erb (4.0ms)
2018-11-15T15:38:38 [I|app|ee8f8]   Rendered home/_vertical_menu.html.erb (2.4ms)
2018-11-15T15:38:38 [I|app|ee8f8]   Rendered home/_vertical_menu.html.erb (2.0ms)
2018-11-15T15:38:38 [I|app|ee8f8]   Rendered home/_vertical_menu.html.erb (1.3ms)
2018-11-15T15:38:38 [I|app|ee8f8]   Rendered home/_vertical_menu.html.erb (2.5ms)
2018-11-15T15:38:38 [I|app|ee8f8]   Rendered home/_vertical_taxonomies.html.erb (1.9ms)
2018-11-15T15:38:38 [I|app|ee8f8]   Rendered home/_vertical_taxonomies.html.erb (1.9ms)
2018-11-15T15:38:38 [I|app|ee8f8]   Rendered home/_vertical_menu.html.erb (0.8ms)
2018-11-15T15:38:38 [I|app|ee8f8]   Rendered home/_navbar.html.erb (26.9ms)
2018-11-15T15:38:38 [I|app|ee8f8]   Rendered layouts/base.html.erb (36.6ms)
2018-11-15T15:38:38 [I|app|ee8f8] Completed 200 OK in 1261ms (Views: 54.6ms | ActiveRecord: 268.4ms)
Comment 8 Ondřej Pražák 2019-01-09 08:26:39 UTC 
Created redmine issue http://projects.theforeman.org/issues/25808 from this bug
Comment 9 Bryan Kearney 2019-01-10 13:07:13 UTC 
Upstream bug assigned to oprazak
Comment 10 Bryan Kearney 2019-01-22 09:02:24 UTC 
Moving this bug to POST for triage into Satellite 6 since the upstream issue https://projects.theforeman.org/issues/25808 has been resolved.
Comment 11 Mirek Długosz 2019-01-23 13:07:17 UTC 
As far as I can tell, old resource mapping dates back to early 6.2 release.  These filters should be moved to new resources as part of 6.2 update or upgrade to 6.3.

In attempt to reproduce the issue, I installed 6.2.1 and created roles with view_arf_report and view_subscriptions filters. Then I followed upgrade path: 6.2.1 -> 6.2.16 -> 6.3.5 -> 6.4.1.

At this point, I tried to reproduce the issue following steps outlined above. I could not - roles can be saved without problem, both with and without any change to role definition.

However, I did notice that create_arf_reports filter is still assigned to "(Miscellaneous)" resource (instead of "ForemanOpenscap::ArfReport"). This is the only filter that isn't where it should be - destroy_arf_reports, view_arf_reports, view_subscriptions, attach_subscriptions and unattach_subscriptions are all in correct resources (when comparing fresh 6.4.1 install with my box that started as 6.2.1). Each of these was part of some non-default role before upgrade.

@Ondřej
Were you able to reproduce this issue locally? Could you help me with setting up environment where I could reproduce it as well?
After running foreman-maintain with your patch on my box, create_arf_reports should change resource to ForemanOpenscap::ArfReport, is that correct? Or is the thing that I see different issue?
Comment 12 Mirek Długosz 2019-01-23 16:02:30 UTC 
I managed to reproduce one error. On 6.2.1 I created role with view_arf_reports, destroy_arf_reports and create_arf_reports filters. After upgrade to 6.4.1, when I open that role and click "Submit" (without any change), I get 
"Unable to save
One or more of the associated filters are invalid which prevented the role to be saved"

So, this is second error from Comment #3; but I rather expected to see first error in this case.

I think this should give us some base for verification of fix, but I still welcome any help in setting up environment where issue could be reliably reproduced.
Comment 13 Ondřej Pražák 2019-01-24 07:08:59 UTC 
I was not able to reproduce. Because the cause is known, I changed the resource for some of the permissions to get the errors (I can provide a migration which does it if needed). The procedure in foreman-maintain does not change resources for permissions, it should bring the role into consistent state so it can be saved.
Comment 14 Ondřej Pražák 2019-01-24 07:44:24 UTC 
Created attachment 1522978 [details]
migration to corrupt filters

The migration changes the resource type for view_arf_reports permission, so any role having this permission will be corrupted after migration is run.

1) create a role with view_arf_reports, create_arf_reports, destroy_arf_reports permissions
2) add file into the migration folder (/usr/share/foreman/db/migrate)
3) foreman-rake db:migrate
4) systemctl restart httpd
5) role should be corrupted and will not save
Comment 15 Mirek Długosz 2019-01-24 18:25:34 UTC 
Ondřej,

Thanks for migration file and explanations. I do appreciate that.

Since there is no downstream build with your patch yet, I have applied it on top of rubygem-foreman_maintain-0.3.0-1 for quick test.

I brought fresh 6.4.1 VM to state where bug can be reproduced by following your instructions. I ran `foreman-maintain upgrade` and saw check for roles with filters from multiple resources failing. I told foreman-maintain to fix issue automatically and verified in Satellite web UI that bug can no longer be reproduced.

I repeated that on my Satellite box that started as 6.2.1, with the same result.

Once snap with your patch is available, I'll verify that issue is fixed as part of Satellite 6.4.1 -> 6.4.2 and 6.4.1 -> 6.5 upgrade paths. But I believe this will be only formality.
Comment 16 Mirek Długosz 2019-03-20 16:02:27 UTC 
Repeated verification on latest 6.5 snap, which has fix included.

I started with fresh 6.4.1 box that was intentionally brought up to "broken" state (using instructions from comment #14). After installing Satellite 6.5 packages, I ran `foreman-maintain upgrade check` and saw that check for roles filters from multiple resources is failing. I responded that foreman-maintain should fix the issue automatically. After upgrade, I verified in Satellite web UI that bug can't be reproduced (role with invalid filters can be saved without errors).

I repeated these steps on Satellite box that started as 6.2.1 and was upgraded up to 6.4.1, and then to 6.5. Results are the same.


Version:
Satellite 6.5 snap 20
satellite-6.5.0-9.el7sat.noarch
foreman-1.20.1.22-1.el7sat.noarch
pulp-server-2.18.1-3.el7sat.noarch
tfm-rubygem-katello-3.10.0.29-1.el7sat.noarch
rubygem-foreman_maintain-0.3.2-1.el7sat.noarch
Comment 19 errata-xmlrpc 2019-05-14 12:38:50 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2019:1222