New commit detected on ManageIQ/manageiq/fine:
https://github.com/ManageIQ/manageiq/commit/aa1d911e5739419040ab87c3ca53ac03a6a56229

commit aa1d911e5739419040ab87c3ca53ac03a6a56229
Author:     Adam Grare <agrare@redhat.com>
AuthorDate: Wed Apr 12 15:34:02 2017 -0400
Commit:     Satoe Imaishi <simaishi@redhat.com>
CommitDate: Thu Apr 13 14:55:39 2017 -0400

    Merge pull request #14711 from durandom/ansible_update_project
    
    targeted refresh for ansible configuration_script_source
    (cherry picked from commit d968839e90e72ed44d6f7af41a5feb7c509ef795)
    
    https://bugzilla.redhat.com/show_bug.cgi?id=1442170

 Gemfile                                            |   2 +-
 app/models/ems_refresh.rb                          |   4 +-
 .../ems_refresh/refreshers/ems_refresher_mixin.rb  |   8 +-
 .../collector/configuration_script_source.rb       |   3 +
 .../parser/configuration_script_source.rb          |   3 +
 .../persister/configuration_script_source.rb       |   3 +
 .../configuration_script_source.rb                 |  21 +
 .../collector/configuration_script_source.rb       |  20 +
 .../shared/inventory/parser/automation_manager.rb  |   2 +-
 .../parser/configuration_script_source.rb          |   8 +
 .../persister/configuration_script_source.rb       |  13 +
 .../collector/configuration_script_source.rb       |   3 +
 .../parser/configuration_script_source.rb          |   3 +
 .../persister/configuration_script_source.rb       |   3 +
 app/models/manager_refresh/inventory.rb            |  14 +
 app/models/manager_refresh/inventory/persister.rb  |   1 +
 spec/factories/authentication.rb                   |   8 +
 .../refresher_configuration_script_source_spec.rb  |   7 +
 .../refresher_configuration_script_source_spec.rb  |   7 +
 .../refresh_configuartion_script_source.rb         |  63 ++
 ...resher_targeted_configuration_script_source.yml | 912 +++++++++++++++++++++
 21 files changed, 1099 insertions(+), 9 deletions(-)
 create mode 100644 app/models/manageiq/providers/ansible_tower/inventory/collector/configuration_script_source.rb
 create mode 100644 app/models/manageiq/providers/ansible_tower/inventory/parser/configuration_script_source.rb
 create mode 100644 app/models/manageiq/providers/ansible_tower/inventory/persister/configuration_script_source.rb
 create mode 100644 app/models/manageiq/providers/ansible_tower/shared/inventory/collector/configuration_script_source.rb
 create mode 100644 app/models/manageiq/providers/ansible_tower/shared/inventory/parser/configuration_script_source.rb
 create mode 100644 app/models/manageiq/providers/ansible_tower/shared/inventory/persister/configuration_script_source.rb
 create mode 100644 app/models/manageiq/providers/embedded_ansible/inventory/collector/configuration_script_source.rb
 create mode 100644 app/models/manageiq/providers/embedded_ansible/inventory/parser/configuration_script_source.rb
 create mode 100644 app/models/manageiq/providers/embedded_ansible/inventory/persister/configuration_script_source.rb
 create mode 100644 spec/models/manageiq/providers/ansible_tower/automation_manager/refresher_configuration_script_source_spec.rb
 create mode 100644 spec/models/manageiq/providers/embedded_ansible/automation_manager/refresher_configuration_script_source_spec.rb
 create mode 100644 spec/support/ansible_shared/automation_manager/refresh_configuartion_script_source.rb
 create mode 100644 spec/vcr_cassettes/manageiq/providers/ansible_tower/automation_manager/refresher_targeted_configuration_script_source.yml

I added a repo with 5 playbooks and removed one. After that checked that repo in the repo list and pressed "Refresh Selected Ansible Repositories". Amount of playbooks didn't change. Then I opened details of that repo and clicked "Refresh this Repository" only after that amount of playbooks had changed. Tested in 5.8.0.12-rc1.20170425180304_4f35996.

After some testing on the same appliance it worked for me. Seems there is some timing issue. 
The first time I tried it with Dmitry's Repo, it did not work. Then with my repo, it worked.

I'll add another sleep to the process :(

https://github.com/ManageIQ/manageiq-ui-classic/pull/1176

Verified in 5.8.0.13-rc2.20170502165848_0f98658. Refresh in repo list and repo details screen works as expected.

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2017:1367