Bug 1596942
| Summary: | Barbican - Creating secret using `` openstack secret store`` is failing `` Failed to establish a new connection: [Errno 111] Connection refused',))`` | ||
|---|---|---|---|
| Product: | Red Hat OpenStack | Reporter: | Nilesh <nchandek> |
| Component: | openstack-barbican | Assignee: | Michele Baldessari <michele> |
| Status: | CLOSED DUPLICATE | QA Contact: | Prasanth Anbalagan <panbalag> |
| Severity: | medium | Docs Contact: | |
| Priority: | medium | ||
| Version: | 13.0 (Queens) | CC: | hrybacki |
| Target Milestone: | --- | ||
| Target Release: | --- | ||
| Hardware: | All | ||
| OS: | All | ||
| Whiteboard: | |||
| Fixed In Version: | Doc Type: | If docs needed, set a value | |
| Doc Text: | Story Points: | --- | |
| Clone Of: | Environment: | ||
| Last Closed: | 2018-07-04 15:01:58 UTC | Type: | Bug |
| Regression: | --- | Mount Type: | --- |
| Documentation: | --- | CRM: | |
| Verified Versions: | Category: | --- | |
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
| Cloudforms Team: | --- | Target Upstream Version: | |
| Embargoed: | |||
This is not a Barbican issue, but rather an HAProxy one, given that an HAProxy restart fixes the issue. IIRC, there was recently a fix to restart HAProxy in this case. Redirecting to PIDONE. *** This bug has been marked as a duplicate of bug 1559105 *** |
Here's the complete procedure to enable barbican on an existing cloud (copying the exact commands I ran for reference): 1. Deploy overcloud ~~~ (overcloud) [stack@undercloud-0 ~]$ cat overcloud_deploy.sh #!/bin/bash openstack overcloud deploy \ --timeout 100 \ --templates /usr/share/openstack-tripleo-heat-templates \ --stack overcloud \ --libvirt-type kvm \ --ntp-server clock.redhat.com \ -e /home/stack/virt/config_lvm.yaml \ -e /usr/share/openstack-tripleo-heat-templates/environments/network-isolation.yaml \ -e /home/stack/virt/network/network-environment.yaml \ -e /home/stack/virt/hostnames.yml \ -e /home/stack/virt/debug.yaml \ -e /home/stack/virt/nodes_data.yaml \ -e /home/stack/virt/docker-images.yaml \ --log-file overcloud_deployment_63.log (overcloud) [stack@undercloud-0 ~]$ ~~~ 2. custom params ~~~ (overcloud) [stack@undercloud-0 ~]$ cat configure-barbican.yaml --- parameter_defaults: BarbicanSimpleCryptoGlobalDefault: true (overcloud) [stack@undercloud-0 ~]$ ~~~ 3. Prepare new images, include custom_params.yaml and the relevant tht files. ~~~ (overcloud) [stack@undercloud-0 ~]$ cat prepares_new_container_images_for_barbican #!/bin/bash openstack overcloud container image prepare \ --namespace registry.access.redhat.com/rhosp13 \ --tag latest \ --push-destination 192.168.24.1:8787 \ --output-images-file ~/container-images-with-barbican.yaml \ -e /home/stack/virt/config_lvm.yaml \ -e /usr/share/openstack-tripleo-heat-templates/environments/network-isolation.yaml \ -e /home/stack/virt/network/network-environment.yaml \ -e /home/stack/virt/hostnames.yml \ -e /home/stack/virt/debug.yaml \ -e /home/stack/virt/nodes_data.yaml \ -e /home/stack/virt/docker-images.yaml \ -e /usr/share/openstack-tripleo-heat-templates/environments/services/barbican.yaml \ -e /usr/share/openstack-tripleo-heat-templates/environments/barbican-backend-simple-crypto.yaml \ -e /home/stack/configure-barbican.yaml (overcloud) [stack@undercloud-0 ~]$ ~~~ 4. upload container images to undercloud registry ~~~ (overcloud) [stack@undercloud-0 ~]$ openstack overcloud container image upload --debug --config-file container-images-with-barbican.yaml(overcloud) [stack@undercloud-0 ~]$ ~~~ 5. prepare the new environment file ~~~ (overcloud) [stack@undercloud-0 ~]$ cat prepares_new_env_file_for_barbican #!/bin/bash openstack overcloud container image prepare \ --tag latest \ --namespace 192.168.24.1:8787/rhosp13 \ --output-env-file ~/container-parameters-with-barbican.yaml \ -e /home/stack/virt/config_lvm.yaml \ -e /usr/share/openstack-tripleo-heat-templates/environments/network-isolation.yaml \ -e /home/stack/virt/network/network-environment.yaml \ -e /home/stack/virt/hostnames.yml \ -e /home/stack/virt/debug.yaml \ -e /home/stack/virt/nodes_data.yaml \ -e /home/stack/virt/docker-images.yaml \ -e /usr/share/openstack-tripleo-heat-templates/environments/services/barbican.yaml \ -e /usr/share/openstack-tripleo-heat-templates/environments/barbican-backend-simple-crypto.yaml \ -e /home/stack/configure-barbican.yaml (overcloud) [stack@undercloud-0 ~]$ ~~~ 6. update overcloud ~~~ (overcloud) [stack@undercloud-0 ~]$ cat overcloud_deploy_with_barbican.sh #!/bin/bash openstack overcloud deploy \ --timeout 100 \ --templates /usr/share/openstack-tripleo-heat-templates \ --stack overcloud \ --libvirt-type kvm \ --ntp-server clock.redhat.com \ -e /home/stack/virt/config_lvm.yaml \ -e /usr/share/openstack-tripleo-heat-templates/environments/network-isolation.yaml \ -e /home/stack/virt/network/network-environment.yaml \ -e /home/stack/virt/hostnames.yml \ -e /home/stack/virt/debug.yaml \ -e /home/stack/virt/nodes_data.yaml \ -e /home/stack/virt/docker-images.yaml \ -e /home/stack/container-parameters-with-barbican.yaml \ -e /usr/share/openstack-tripleo-heat-templates/environments/services/barbican.yaml \ -e /usr/share/openstack-tripleo-heat-templates/environments/barbican-backend-simple-crypto.yaml \ -e /home/stack/configure-barbican.yaml \ --log-file overcloud_deployment_63.log (overcloud) [stack@undercloud-0 ~]$ ~~~ 7. Check heat stack-list ~~~ (undercloud) [stack@undercloud-0 ~]$ heat stack-list WARNING (shell) "heat stack-list" is deprecated, please use "openstack stack list" instead +--------------------------------------+------------+-----------------+----------------------+----------------------+----------------------------------+ | id | stack_name | stack_status | creation_time | updated_time | project | +--------------------------------------+------------+-----------------+----------------------+----------------------+----------------------------------+ | 00ed5d0a-9c09-4db2-b147-282e9058fddf | overcloud | UPDATE_COMPLETE | 2018-06-29T13:49:10Z | 2018-06-30T10:43:24Z | e0ec08746b694d4b8a8a45db95ca0db0 | +--------------------------------------+------------+-----------------+----------------------+----------------------+----------------------------------+ (undercloud) [stack@undercloud-0 ~]$ ~~~ 8. Endpoints created ~~~ (overcloud) [stack@undercloud-0 ~]$ openstack endpoint list |grep -i barbican | 1671984f717e46e9ba39143040f0f340 | regionOne | barbican | key-manager | True | internal | http://172.17.1.15:9311 | | 72c51082a2f64fd08ffd9f295d75abcc | regionOne | barbican | key-manager | True | public | http://10.0.0.104:9311 | | c4b48a09820f4e6da953cc798f475535 | regionOne | barbican | key-manager | True | admin | http://172.17.1.15:9311 | (overcloud) [stack@undercloud-0 ~]$ ~~~ Succesfully installed and configure Barbican but while testing - ~~~ (overcloud) [stack@undercloud-0 ~]$ openstack secret store --name secret01 --payload secretkey Failed to contact the endpoint at http://10.0.0.104:9311 for discovery. Fallback to using that endpoint as the base url. Unable to establish connection to http://10.0.0.104:9311/secrets/: HTTPConnectionPool(host='10.0.0.104', port=9311): Max retries exceeded with url: /secrets/ (Caused by NewConnectionError('<requests.packages.urllib3.connection.HTTPConnection object at 0x7f09b6fbf7d0>: Failed to establish a new connection: [Errno 111] Connection refused',)) (overcloud) [stack@undercloud-0 ~]$ ~~~ ~~~ (overcloud) [stack@undercloud-0 ~]$ openstack endpoint list |grep -i barbican | 1671984f717e46e9ba39143040f0f340 | regionOne | barbican | key-manager | True | internal | http://172.17.1.15:9311 | | 72c51082a2f64fd08ffd9f295d75abcc | regionOne | barbican | key-manager | True | public | http://10.0.0.104:9311 | | c4b48a09820f4e6da953cc798f475535 | regionOne | barbican | key-manager | True | admin | http://172.17.1.15:9311 | (overcloud) [stack@undercloud-0 ~]$ ~~~ It should be listen to all endpoints :- ~~~ [root@controller-0 ~]# netstat -lntpu |grep -i 9311 tcp 0 0 172.17.1.24:9311 0.0.0.0:* LISTEN 51603/httpd [root@controller-0 ~]# ~~~ * Restarted haproxy on all controller's ~~~ [root@controller-1 ~]# docker ps |grep -i haproxy bf6c4d234903 192.168.24.1:8787/rhosp13/openstack-haproxy:pcmklatest "/bin/bash /usr/lo..." 22 hours ago Up 22 hours haproxy-bundle-docker-1 [root@controller-1 ~]# [root@controller-1 ~]# [root@controller-1 ~]# docker restart bf6c4d234903 bf6c4d234903 [root@controller-1 ~]# docker ps |grep -i haproxy bf6c4d234903 192.168.24.1:8787/rhosp13/openstack-haproxy:pcmklatest "/bin/bash /usr/lo..." 22 hours ago Up 2 seconds haproxy-bundle-docker-1 [root@controller-1 ~]# ~~~ * Now its listing on all endpoints ~~~ [root@controller-1 ~]# netstat -lunpta|grep 9311 |grep LISTEN tcp 0 0 172.17.1.15:9311 0.0.0.0:* LISTEN 356869/haproxy tcp 0 0 10.0.0.104:9311 0.0.0.0:* LISTEN 356869/haproxy tcp 0 0 172.17.1.19:9311 0.0.0.0:* LISTEN 1006403/httpd [root@controller-1 ~]# ~~~ * Now able to create secrets