Description of problem: While using the ansible playbook from a private repository - We are able to execute the playbook, playbook output is also visible in UI using the credentials - While the issue is seen with the playbook while using the access token So, I reproduced the issue to verify if access token is the limitation for the playbook service catalog execution Version-Release number of selected component (if applicable): Red Hat CFME 5.11 How reproducible: Always Steps to Reproduce: 1. Create private ansible playbook repo 2. Enable embedded ansible and add the repo in cloudforms 3. Create Catalog service for the playbook to execute Actual results: 1. Try using SCM Creds as Username and Password => This works well without error and also displays the playbook output in CloudForms UI 2. Try using SCM Creds as Username and Access Token => This does executed the request and gives the below error Error: [----] I, [2020-02-27T18:55:48.998267 #1958:2b118a9125b4] INFO -- : MIQ(MiqPriorityWorker::Runner#get_message_via_drb) Message id: [1000000536584], MiqWorker id: [1000000000765], Zone: [default], Role: [smartstate], Server: [], MiqTask id: [], Ident: [generic], Target id: [], Instance id: [], Task id: [job_dispatcher], Command: [JobProxyDispatcher.dispatch], Timeout: [600], Priority: [20], State: [dequeue], Deliver On: [], Data: [], Args: [], Dequeued in: [4.37724586] seconds [----] I, [2020-02-27T18:55:48.998383 #1958:2b118a9125b4] INFO -- : Q-task_id([job_dispatcher]) MIQ(MiqQueue#deliver) Message id: [1000000536584], Delivering... [----] D, [2020-02-27T18:55:48.998626 #1958:2b118a9125b4] DEBUG -- : Q-task_id([job_dispatcher]) MIQ(InfraConversionThrottler.start_conversions) InfraConversionThrottler.start_conversio ns [----] I, [2020-02-27T18:55:48.999565 #1958:2b118a9125b4] INFO -- : Q-task_id([job_dispatcher]) MIQ(InfraConversionThrottler.pending_conversion_jobs) Pending InfraConversionJob: 0 [----] I, [2020-02-27T18:55:49.000857 #1958:2b118a9125b4] INFO -- : Q-task_id([job_dispatcher]) MIQ(InfraConversionThrottler.running_conversion_jobs) Running InfraConversionJob: 0 [----] I, [2020-02-27T18:55:49.003794 #1958:2b118a9125b4] INFO -- : Q-task_id([job_dispatcher]) MIQ(JobProxyDispatcher#dispatch) Complete - Timings: {:v2v_dispatching=>0.00164222717285 15625, :v2v_limits=>0.0013384819030761719, :pending_container_jobs=>0.001107931137084961, :container_jobs_to_dispatch_count=>0, :container_dispatching=>0.00112152099609375, :pending_vm_ jobs=>0.00029778480529785156, :vm_jobs_to_dispatch_count=>0, :total_time=>0.005200624465942383} [----] I, [2020-02-27T18:55:49.004056 #1958:2b118a9125b4] INFO -- : Q-task_id([job_dispatcher]) MIQ(MiqQueue#delivered) Message id: [1000000536584], State: [ok], Delivered in [0.005677 176] seconds [----] I, [2020-02-27T18:55:49.018269 #1958:2b118a9125b4] INFO -- : MIQ(MiqPriorityWorker::Runner#get_message_via_drb) Message id: [1000000536585], MiqWorker id: [1000000000765], Zone: [], Role: [embedded_ansible], Server: [], MiqTask id: [1000000002611], Ident: [generic], Target id: [], Instance id: [1000000000008], Task id: [], Command: [ManageIQ::Providers::Embedd edAnsible::AutomationManager::ConfigurationScriptSource.sync], Timeout: [600], Priority: [20], State: [dequeue], Deliver On: [], Data: [], Args: [], Dequeued in: [2.785879702] seconds [----] I, [2020-02-27T18:55:49.018350 #1958:2b118a9125b4] INFO -- : MIQ(MiqQueue#deliver) Message id: [1000000536585], Delivering... [----] I, [2020-02-27T18:55:49.020141 #1958:2b118a9125b4] INFO -- : MIQ(MiqTask#update_status) Task: [1000000002611] [Active] [Ok] [Task starting] [----] I, [2020-02-27T18:55:49.035950 #1958:2b118a9125b4] INFO -- : MIQ(GitRepository#update_repo) Updating https://github.com/mihirlele/test_repo2.git in /var/www/miq/vmdb/data/git_re pos/1000000000008... [----] E, [2020-02-27T18:55:49.992351 #1958:2b118a9125b4] ERROR -- : MIQ(MiqQueue#deliver) Message id: [1000000536585], Error: [Please provide username and password for URL https://gith ub.com/mihirlele/test_repo2.git] [----] I, [2020-02-27T18:55:49.992471 #1958:2b118a9125b4] INFO -- : MIQ(MiqQueue#delivered) Message id: [1000000536585], State: [error], Delivered in [0.974123828] seconds [----] I, [2020-02-27T18:55:49.993306 #1958:2b118a9125b4] INFO -- : MIQ(MiqQueue#m_callback) Message id: [1000000536585], Invoking Callback with args: ["Finished", "error", "Please pro vide username and password for URL https://github.com/mihirlele/test_repo2.git", "nil"] [----] I, [2020-02-27T18:55:49.993452 #1958:2b118a9125b4] INFO -- : MIQ(MiqTask#update_status) Task: [1000000002611] [Finished] [Error] [Please provide username and password for URL https://github.com/mihirlele/test_repo2.git] Additional info:
Adding to the previous comment, The SCM creds I used were created as by just updating the SCM Creds [Username+Password] -> Edit -> SCM Creds [Username+Token] Which did not work But now I tried by not updating the SCM creds but directly creating a new Cred which solves the issue. But now the point is the SCM Creds do not update is the new Bug
Hello Team, I took the remote session with the customer today and below is the summary: In the session, we have noticed the ansible playbook is failing with below exception: Server [EVM] Service [neha-test] Provision Step [check_completed] Status [Error Processing check_completed] I have checked the /tmp directory to get the full log sets about this error message but I didn't find the ansible-runner related directory, this are files present under /tmp directory root@cfme-5-11-3 tmp]# ls ansible_key20200304-374-1p79uy2 subAvail vmware-root_1519-4257134898 ansible_key20200304-381-vnlhex systemd-private-2a32886225e44b77b4ae7132567ddc0a-chronyd.service-gUypaZ vmware-root_1529-4290035493 ansible_key20200305-26995-1u6kqk1 systemd-private-2a32886225e44b77b4ae7132567ddc0a-httpd.service-ouQn9O vmware-root_1531-4281712295 miq_20200304-31524-w086xk systemd-private-2a32886225e44b77b4ae7132567ddc0a-memcached.service-YItgEE vmware-root_1538-834774654 miq_20200304-31807-ftrb2i systemd-private-2a32886225e44b77b4ae7132567ddc0a-postfix.service-TyjsFu worker_monitor20200304-32198-htl9ca MiqVimBroker20200304-494-dyossw vmware-root_1489-4257134867 reregister.sh vmware-root_1497-4290559773 Moreover, in the Services -> provisioning tab, under console output the result is nil. He is able to discover the playbooks/repositories using the SCM credentials using AuthToken but not with SSH credentials due to network restriction. The issue occurs only during the execution of Ansible Playbook. Let me know what more details you needed from customer side or if there is need to arrange remote session. Regards, Neha Chugh
New commits detected on ManageIQ/manageiq/ivanchuk: https://github.com/ManageIQ/manageiq/commit/bf21456e04c25821d91de2836860213de8c22f41 commit bf21456e04c25821d91de2836860213de8c22f41 Author: Jason Frey <jfrey> AuthorDate: Fri Mar 13 17:20:45 2020 +0000 Commit: Jason Frey <jfrey> CommitDate: Fri Mar 13 17:20:45 2020 +0000 Merge pull request #19939 from NickLaMuro/handle-submodules-in-git-repository-and-worktree [GitWorktree/GitRepository] Handle submodules (cherry picked from commit c04bcfe953462f9f029e683c3e3ffb50b304c768) https://bugzilla.redhat.com/show_bug.cgi?id=1807928 app/models/git_repository.rb | 2 +- lib/git_worktree.rb | 36 +- 2 files changed, 33 insertions(+), 5 deletions(-) https://github.com/ManageIQ/manageiq/commit/abbe10c014675aeee0a55936deffdf9e110c4537 commit abbe10c014675aeee0a55936deffdf9e110c4537 Author: Jason Frey <jfrey> AuthorDate: Mon Mar 16 13:05:19 2020 +0000 Commit: Jason Frey <jfrey> CommitDate: Mon Mar 16 13:05:19 2020 +0000 Merge pull request #19959 from NickLaMuro/git-worktree-handle-branches-with-submodules GitRepository/GitWorktree Fix checkout to work with branches + submodules (cherry picked from commit 971dcd9c60ff37c5370996489749c3ea750b10e8) https://bugzilla.redhat.com/show_bug.cgi?id=1807928 lib/git_worktree.rb | 4 +- 1 file changed, 2 insertions(+), 2 deletions(-)
New commits detected on ManageIQ/manageiq/jansa: https://github.com/ManageIQ/manageiq/commit/ed025cce43413a7e43d66376c947fdcc978e4c4d commit ed025cce43413a7e43d66376c947fdcc978e4c4d Author: Jason Frey <jfrey> AuthorDate: Fri Mar 13 17:20:45 2020 +0000 Commit: Jason Frey <jfrey> CommitDate: Fri Mar 13 17:20:45 2020 +0000 Merge pull request #19939 from NickLaMuro/handle-submodules-in-git-repository-and-worktree [GitWorktree/GitRepository] Handle submodules (cherry picked from commit c04bcfe953462f9f029e683c3e3ffb50b304c768) https://bugzilla.redhat.com/show_bug.cgi?id=1807928 app/models/git_repository.rb | 2 +- lib/git_worktree.rb | 36 +- 2 files changed, 33 insertions(+), 5 deletions(-) https://github.com/ManageIQ/manageiq/commit/93ead089a6fcd70a4a6116c7e425630d2aecb8f8 commit 93ead089a6fcd70a4a6116c7e425630d2aecb8f8 Author: Jason Frey <jfrey> AuthorDate: Mon Mar 16 13:05:19 2020 +0000 Commit: Jason Frey <jfrey> CommitDate: Mon Mar 16 13:05:19 2020 +0000 Merge pull request #19959 from NickLaMuro/git-worktree-handle-branches-with-submodules GitRepository/GitWorktree Fix checkout to work with branches + submodules (cherry picked from commit 971dcd9c60ff37c5370996489749c3ea750b10e8) https://bugzilla.redhat.com/show_bug.cgi?id=1807928 lib/git_worktree.rb | 4 +- 1 file changed, 2 insertions(+), 2 deletions(-)
Hi Nick, Could you please provide me wide verification steps? Thanks, Gaurav
Gaurav, I would test the following scenarios: - A playbook git repo over https without auth - A playbook git repo over https with auth - A playbook git repo over ssh For the https w/auth, make sure that it is a private repo, and if you use MFA, that you use an access token in place of your password. What also needs to be checked with this is that submodules work as expected. They won't be particularly fast, since they have to be re-clone with each playbook run. Also, instead of just cloning the playbooks as verification, a playbook run also needs to happen successfully since the clones are done as bare repos. We checkout the code into the file system on each playbook run, or now do a "re-clone" when there is a submodule in the repo, and this is where the hiccup in the reported issue occurred. This means that when a submodule is present, the performance of running the playbook is much slower because of the re-cloning, so ideally we suggest not using submodules in general. Unfortunately this is the best we could do with `rugged` since there isn't full support for submodules in libgit2. -Nick
Thanks Nick for steps and for clarifying description about a bug. I followed steps from Nick for all test scenarios, from which 2 scenarios are working fine like, playbook git repo over https without auth and without auth. But there was some issue with scenario of playbook git repo over ssh key based auth, for that I discussed with Nick and he investigated on the same, and filed a new BZ, https://bugzilla.redhat.com/show_bug.cgi?id=1826410 Hence, I'm marking this BZ as verified for version 5.11.5.1.20200415152414_39b433a for the first two scenarios and will consider above BZ 1826410 for the 3rd scenario
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2020:2020