Bug 2028572

Summary:	[RFE] retrieve more useful task information from content migration failure
Product:	Red Hat Satellite	Reporter:	Paul Dudley <pdudley>
Component:	Pulp	Assignee:	satellite6-bugs <satellite6-bugs>
Status:	CLOSED WONTFIX	QA Contact:	Satellite QE Team <sat-qe-bz-list>
Severity:	medium	Docs Contact:
Priority:	unspecified
Version:	6.9.7	CC:	dalley, phess
Target Milestone:	Unspecified	Keywords:	FutureFeature, Triaged, Upgrades
Target Release:	Unused
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:		Doc Type:	If docs needed, set a value
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:	2023-08-21 20:25:56 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:

Description Paul Dudley 2021-12-02 16:30:42 UTC

Sometimes an upgrade failure can retrieve useful information like the info highlighted in bz 1998796, but often a pulp3 migration can fail with much simpler information. 

For example, seen in a migration failure:
───────────────────────────────────────────────────────────────────────
var/log/foreman-maintain/foreman-maintain.log:
...
rake aborted!^M
ForemanTasks::TaskError: Task 28fd76c4-d68c-4e50-a188-db:3c:3f:f6:83:86 Katello::Errors::Pulp3Error: 1 subtask(s) failed for task group /pulp/api/v3/task-groups/51172b91-afcd-4865-b093-73:f6:e8:37:2b:16/.^M
/opt/theforeman/tfm/obfuscateduser1/usr/share/gems/gems/katello-128.0.0.95/lib/katello/tasks/pulp3_migration.rake:42:in `block (2 levels) in <top (required)>'^M
/opt/rh/rh-ruby25/obfuscateduser1/usr/share/gems/gems/rake-12.3.3/exe/rake:27:in `<top (required)>'^M
Tasks: TOP => katello:pulp3_migration^M
(See full trace by running task with --trace)^M

E, [2021-11-30 20:39:42-0500 #10271] ERROR -- : Failed executing foreman-rake katello:pulp3_migration, exit status 1 (ForemanMaintain::Error::ExecutionError)
/usr/share/gems/gems/foreman_maintain-0.7.13/lib/foreman_maintain/utils/command_runner.rb:54:in `execution_error'
/usr/share/gems/gems/foreman_maintain-0.7.13/lib/foreman_maintain/concerns/system_helpers.rb:68:in `execute!'
/usr/share/gems/gems/foreman_maintain-0.7.13/definitions/procedures/content/prepare.rb:10:in `run'
/usr/share/gems/gems/foreman_maintain-0.7.13/lib/foreman_maintain/executable.rb:139:in `__run__'
...
───────────────────────────────────────────────────────────────────────
From the dynflow task info:
Error:

Katello::Errors::Pulp3Error

1 subtask(s) failed for task group /pulp/api/v3/task-groups/51172b91-afcd-4865-b093-c5f574b6f388/.
───────────────────────────────────────────────────────────────────────

We can find this task info in Dynflow but that information is also limited, seen here:
───────────────────────────────────────────────────────────────────────
task_groups:
- pulp_href: "/pulp/api/v3/task-groups/51172b91-afcd-4865-b093-c5f574b6f388/"
  description: Migration Sub-tasks
  all_tasks_dispatched: true
  waiting: 0
  skipped: 0
  running: 0
  completed: 103
  canceled: 0
  failed: 1
  group_progress_reports:
  - message: Distribution creation
    code: create.distribution
    total: 183
    done: 182
  - message: Repo version creation
    code: create.repo_version
    total: 103
    done: 103
poll_attempts:
  total: 1162
  failed: 1
───────────────────────────────────────────────────────────────────────

So this helps the user identify that a distributor had trouble migrating, but isn't helpful with other information, like which distributor caused an issue when migrating, and also why that issue occurred is left out.

Comment 1 Paul Dudley 2021-12-02 16:37:30 UTC

In that specific case above of a distributor failing to be migrated we can check the db for which distributors are left:
- # su - postgres -c "psql pulpcore -c \"SELECT pulp_id,pulp2_id,pulp2_config,is_migrated FROM pulp_2to3_migration_pulp2distributor WHERE is_migrated = 'f'\""

Comment 6 Paul Dudley 2022-10-26 01:09:22 UTC

This was filed quite a while ago, with many improvements in the meantime. Some errors still come back almost totally ambiguous as to their meaning, but overall I do think it's likely better to focus on BZs with a known impact and issue, bz 2038995 and bz 2074099 for examples, and leave this as unresolved given the constraints on the final 6.9 release.

Comment 7 Daniel Alley 2022-10-26 01:21:53 UTC

Sounds good, there's some discussion about getting both of those into a release.