Bug 1434542 - [openstack containers] after upstream (pike) deployment with containers using quickstart the gnocchi-statsd is restarting
Summary: [openstack containers] after upstream (pike) deployment with containers using...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat OpenStack
Classification: Red Hat
Component: openstack-containers
Version: 12.0 (Pike)
Hardware: All
OS: Linux
medium
high
Target Milestone: beta
: 12.0 (Pike)
Assignee: Martin André
QA Contact: Omri Hochman
Andrew Burden
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2017-03-21 17:00 UTC by Artem Hrechanychenko
Modified: 2017-12-13 19:14 UTC (History)
5 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2017-12-13 19:14:33 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
OpenStack gerrit 448077 0 None None None 2017-05-22 15:39:08 UTC
Red Hat Product Errata RHEA-2017:3457 0 normal SHIPPED_LIVE Red Hat OpenStack Platform 12.0 Containers Enhancement Advisory 2017-12-14 04:45:51 UTC

Description Artem Hrechanychenko 2017-03-21 17:00:50 UTC
Description of problem:

After upstream (pike) deployment with containers using  quickstart the gnocchi-statsd is restarting

Version-Release number of selected component (if applicable):


How reproducible:
always

Steps to Reproduce:
(1) from laptop or remote machine : fix the ssh-key by 

      run: ssh-copy-id -i ~/.ssh/id_rsa.pub $VIRTHOST, where $VIRTHOST - your baremetal lab 

(2) to test run : ssh root@$VIRTHOST uname -a
(3) wget https://raw.githubusercontent.com/openstack/tripleo-quickstart/master/quickstart.sh
(4) bash quickstart.sh --install-deps
(5) change target machine to latest (more fit CentOS7.3) repos --> http://pastebin.test.redhat.com/465894
(6) bash quickstart.sh --config /var/tmp/foo/config/general_config/containers_minimal.yml --working-dir /var/tmp/foo --teardown all --tags all --release master-tripleo-ci  $VIRTHOST

7) [heat-admin@overcloud-controller-0 ~]$ sudo docker ps 

Actual results:
73e255c91c59        192.168.24.1:8787/tripleoupstream/centos-binary-gnocchi-statsd:latest              "kolla_start"            18 hours ago        Restarting (1) 4 hours ago                       gnocchi_statsd


[heat-admin@overcloud-controller-0 ~]$ sudo docker exec -it gnocchi_statsd bash
Error response from daemon: Container 73e255c91c59c27db6699ef593e8845490003ce711d4d903702e1a9d0fca685a is restarting, wait until the container is running

Expected results:
Container Status - Up 

Additional info:

Comment 1 Red Hat Bugzilla Rules Engine 2017-03-21 17:01:49 UTC
This bugzilla has been removed from the release and needs to be reviewed and Triaged for another Target Release.

Comment 2 Omri Hochman 2017-03-21 17:08:29 UTC
might be related with :  https://review.openstack.org/#/c/448077/

Comment 3 Artem Hrechanychenko 2017-03-21 17:12:46 UTC
(undercloud) [stack@undercloud ~]$ rpm -qa |grep docker
docker-common-1.12.6-11.el7.centos.x86_64
docker-1.12.6-11.el7.centos.x86_64
python-docker-py-1.10.6-1.el7.noarch
docker-client-1.12.6-11.el7.centos.x86_64
python-heat-agent-docker-cmd-1.0.1-0.20170306124536.3961e3b.el7.centos.noarch
docker-distribution-2.6.0-1.el7.x86_64
python-docker-pycreds-1.10.6-1.el7.noarch
instack-6.0.1-0.20170310015230.db7123e.el7.centos.noarc
instack-undercloud-6.0.0-0.20170317130016.58a4f6c.el7.centos.noarch


heat-admin@overcloud-controller-0 ~]$ sudo docker ps
CONTAINER ID        IMAGE                                                                              COMMAND                  CREATED             STATUS                       PORTS               NAMES
508b01f42d5e        192.168.24.1:8787/tripleoupstream/centos-binary-nova-api:latest                    "kolla_start"            20 hours ago        Up 20 hours                                      nova_api
84ddcd8f0f6c        192.168.24.1:8787/tripleoupstream/centos-binary-heat-api-cfn:latest                "kolla_start"            20 hours ago        Up 20 hours                                      heat_api_cfn
a6784023413c        192.168.24.1:8787/tripleoupstream/centos-binary-neutron-server:latest              "kolla_start"            20 hours ago        Up 20 hours                                      neutron_api
e46e269d13d7        192.168.24.1:8787/tripleoupstream/centos-binary-aodh-listener:latest               "kolla_start"            20 hours ago        Up 20 hours                                      aodh_listener
652f468a9f4f        192.168.24.1:8787/tripleoupstream/centos-binary-swift-container:latest             "kolla_start"            20 hours ago        Up 20 hours                                      swift_container_auditor
41e5ee6ef6f5        192.168.24.1:8787/tripleoupstream/centos-binary-swift-proxy-server:latest          "kolla_start"            20 hours ago        Up 20 hours                                      swift_object_expirer
3171133a3aa0        192.168.24.1:8787/tripleoupstream/centos-binary-swift-object:latest                "kolla_start"            20 hours ago        Up 20 hours                                      swift_object_updater
a2c107ce6b3e        192.168.24.1:8787/tripleoupstream/centos-binary-swift-container:latest             "kolla_start"            20 hours ago        Up 20 hours                                      swift_container_replicator
2e942779bf39        192.168.24.1:8787/tripleoupstream/centos-binary-swift-account:latest               "kolla_start"            20 hours ago        Up 20 hours                                      swift_account_auditor
134fa1d35bad        192.168.24.1:8787/tripleoupstream/centos-binary-aodh-api:latest                    "kolla_start"            20 hours ago        Up 20 hours                                      aodh-api
859bd6ce0520        192.168.24.1:8787/tripleoupstream/centos-binary-swift-account:latest               "kolla_start"            20 hours ago        Up 20 hours                                      swift_account_server
a5e028baf447        192.168.24.1:8787/tripleoupstream/centos-binary-nova-scheduler:latest              "kolla_start"            20 hours ago        Up 20 hours                                      nova_scheduler
91486afbb001        192.168.24.1:8787/tripleoupstream/centos-binary-gnocchi-metricd:latest             "kolla_start"            20 hours ago        Up 20 hours                                      gnocchi_metricd
dbe5a5e22107        192.168.24.1:8787/tripleoupstream/centos-binary-swift-object:latest                "kolla_start"            20 hours ago        Up 20 hours                                      swift_object_replicator
5b527490401c        192.168.24.1:8787/tripleoupstream/centos-binary-neutron-openvswitch-agent:latest   "kolla_start"            20 hours ago        Up 20 hours                                      neutronovsagent
30cca42565b5        192.168.24.1:8787/tripleoupstream/centos-binary-heat-engine:latest                 "kolla_start"            20 hours ago        Up 20 hours                                      heat_engine
9868fdca986b        192.168.24.1:8787/tripleoupstream/centos-binary-neutron-l3-agent:latest            "kolla_start"            20 hours ago        Up 20 hours                                      neutronl3agent
7317ead2ca1a        192.168.24.1:8787/tripleoupstream/centos-binary-gnocchi-api:latest                 "kolla_start"            20 hours ago        Up 20 hours                                      gnocchi-api
5d785638e213        192.168.24.1:8787/tripleoupstream/centos-binary-swift-account:latest               "kolla_start"            20 hours ago        Up 20 hours                                      swift_account_reaper
b6f0be9a5eee        192.168.24.1:8787/tripleoupstream/centos-binary-glance-api:latest                  "kolla_start"            20 hours ago        Up 20 hours                                      glance_api
1aec01aff0e6        192.168.24.1:8787/tripleoupstream/centos-binary-swift-container:latest             "kolla_start"            20 hours ago        Up 20 hours                                      swift_container_server
73e255c91c59        192.168.24.1:8787/tripleoupstream/centos-binary-gnocchi-statsd:latest              "kolla_start"            20 hours ago        Restarting (1) 5 hours ago                       gnocchi_statsd
4959fecd8a41        192.168.24.1:8787/tripleoupstream/centos-binary-aodh-notifier:latest               "kolla_start"            20 hours ago        Up 20 hours                                      aodh_notifier
a5b90ea92a14        192.168.24.1:8787/tripleoupstream/centos-binary-swift-account:latest               "kolla_start"            20 hours ago        Up 20 hours                                      swift_account_replicator
5b762ed96f03        192.168.24.1:8787/tripleoupstream/centos-binary-swift-object:latest                "kolla_start"            20 hours ago        Up 20 hours                                      swift_object_auditor
7edbf94532df        192.168.24.1:8787/tripleoupstream/centos-binary-heat-api:latest                    "kolla_start"            20 hours ago        Up 20 hours                                      heat_api
d90486d0e370        192.168.24.1:8787/tripleoupstream/centos-binary-neutron-dhcp-agent:latest          "kolla_start"            20 hours ago        Up 20 hours                                      neutron_dhcp
abfaa4512d80        192.168.24.1:8787/tripleoupstream/centos-binary-swift-proxy-server:latest          "kolla_start"            20 hours ago        Up 20 hours                                      swift_proxy
0883783d9370        192.168.24.1:8787/tripleoupstream/centos-binary-swift-object:latest                "kolla_start"            20 hours ago        Up 20 hours                                      swift_object_server
8dde6c5f40c2        192.168.24.1:8787/tripleoupstream/centos-binary-nova-conductor:latest              "kolla_start"            20 hours ago        Up 20 hours                                      nova_conductor
8b1e426c0295        192.168.24.1:8787/tripleoupstream/centos-binary-aodh-evaluator:latest              "kolla_start"            20 hours ago        Up 20 hours                                      aodh_evaluator
ee7d66308895        192.168.24.1:8787/tripleoupstream/centos-binary-swift-container:latest             "kolla_start"            20 hours ago        Up 20 hours                                      swift_container_updater
6e634f364d3f        192.168.24.1:8787/tripleoupstream/centos-binary-keystone:latest                    "kolla_start"            20 hours ago        Up 20 hours                                      keystone
4987fb18897a        192.168.24.1:8787/tripleoupstream/centos-binary-nova-placement-api                 "kolla_start"            20 hours ago        Up 20 hours                                      nova_placement
cf846977f5d1        192.168.24.1:8787/tripleoupstream/centos-binary-mariadb:latest                     "kolla_start"            20 hours ago        Up 20 hours                                      mysql
b1497e32ecbc        192.168.24.1:8787/tripleoupstream/centos-binary-mongodb:latest                     "kolla_start"            20 hours ago        Up 20 hours                                      mongodb
77be4e8bbb61        192.168.24.1:8787/tripleoupstream/centos-binary-rabbitmq:latest                    "kolla_start"            20 hours ago        Up 20 hours                                      rabbitmq
c480bc930940        192.168.24.1:8787/tripleoupstream/centos-binary-memcached:latest                   "/bin/bash -c 'source"   20 hours ago        Up 20 hours                                      memcached

Comment 4 Ryan Hallisey 2017-03-21 17:14:03 UTC
Can you report the result of `docker logs gnocchi_statsd`.  Docker logs will grab stdout of pid 1, which will give us a clue into what's going on.

Comment 5 Artem Hrechanychenko 2017-03-21 17:17:16 UTC
http://pastebin.test.redhat.com/466829 docker inspect gnocchi_statsd
docker logs - http://pastebin.test.redhat.com/466830

Comment 6 Ryan Hallisey 2017-03-21 17:39:23 UTC
Looks like the logs were no help. What you will have to do change the run command in the container to '/bin/sleep 10000' then run '/usr/bin/gnocchi-statsd' by hand and see if we can get further output.  If nothing logs to stdout, then check the log file after you try to run it.

Change this command to `/bin/sleep 10000' then exec into the container:
https://github.com/openstack/tripleo-heat-templates/blob/c49fa2625d27dae4dd72d67a013c0e9363d007b9/docker/services/gnocchi-statsd.yaml#L59

Comment 8 Artem Hrechanychenko 2017-03-22 11:33:53 UTC
according to https://github.com/openstack/tripleo-heat-templates/blob/c49fa2625d27dae4dd72d67a013c0e9363d007b9/docker/services/gnocchi-statsd.yaml#L59
I have created new container 

#sudo docker run -ti -v /var/lib/kolla/config_files/gnocchi-statsd.json:/var/lib/kolla/config_files/config.json -v /var/lib/config-data/gnocchi:/var/lib/kolla/config_files/src 192.168.24.1:8787/tripleoupstream/centos-binary-gnocchi-statsd /bin/bash

)[gnocchi@ca2468f0706b /]$ env
KOLLA_CONFIG_STRATEGY=COPY_ALWAYS
KOLLA_BASE_DISTRO=centos
HOSTNAME=ca2468f0706b
KOLLA_INSTALL_TYPE=binary
TERM=xterm
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
PWD=/
PS1=$(tput bold)($(printenv KOLLA_SERVICE_NAME))$(tput sgr0)[$(id -un)@$(hostname -s) $(pwd)]$ 
SHLVL=1
HOME=/var/lib/gnocchi
KOLLA_INSTALL_METATYPE=rdo
_=/usr/bin/env


()[gnocchi@ca2468f0706b /]$ kolla_start 
INFO:__main__:Loading config file at /var/lib/kolla/config_files/config.json
INFO:__main__:Validating config file
INFO:__main__:Kolla config strategy set to: COPY_ALWAYS
INFO:__main__:Copying service configuration files
INFO:__main__:Deleting file /etc/gnocchi/gnocchi.conf
INFO:__main__:Coping file from /var/lib/kolla/config_files/src/etc/gnocchi/gnocchi.conf to /etc/gnocchi/gnocchi.conf
INFO:__main__:Setting file /etc/gnocchi/gnocchi.conf owner to gnocchi:gnocchi
INFO:__main__:Setting file /etc/gnocchi/gnocchi.conf permission to 0640
INFO:__main__:Writing out command to execute
Running command: '/usr/bin/gnocchi-statsd'
()[gnocchi@ca2468f0706b /]$ echo $?
1


from logs I see keystone 404 eror
http://pastebin.test.redhat.com/467126

gnocci conf

()[gnocchi@ca2468f0706b /]$ cat /etc/gnocchi/gnocchi.conf |grep -v -e '^$' |grep -v -e '^#'
[DEFAULT]
log_dir = /var/log/gnocchi
[api]
auth_mode = keystone
max_limit = 1000
[archive_policy]
[cors]
[cors.subdomain]
[database]
[healthcheck]
[incoming]
[indexer]
url = mysql+pymysql://gnocchi:p29RAUdcRT7Qfm7fGArwz6yZU.2.11/gnocchi?read_default_file=/etc/my.cnf.d/tripleo.cnf&read_default_group=tripleo
[metricd]
workers = 1
[oslo_middleware]
enable_proxy_headers_parsing = True
[oslo_policy]
policy_file = /etc/gnocchi/policy.json
[statsd]
resource_id = 0a8b55df-f90f-491c-8cb9-7cdecec6fc26
user_id = 27c0d3f8-e7ee-42f0-8317-72237d1c5ae3
project_id = 6c38cd8d-099a-4cb2-aecf-17be688e8616
archive_policy_name = low
flush_delay = 10
[storage]
driver = swift
swift_auth_version = 3
swift_authurl = http://172.16.2.11:5000
swift_user = service:gnocchi
swift_key = p29RAUdcRT7Qfm7fGArwz6yZU
coordination_url = redis://:HZYT6GPqrWwvYZxaN8QTRzV48.2.6:6379/
[keystone_authtoken]
auth_uri=http://172.16.2.11:5000
auth_type=password
auth_url=http://172.16.2.11:5000
username=gnocchi
password=p29RAUdcRT7Qfm7fGArwz6yZU
user_domain_name=Default
project_name=service
project_domain_name=Default





()[gnocchi@ca2468f0706b /]$ ()[gnocchi@ca2468f0706b /]$ curl -i   -H "Content-Type: application/json"   -d '
{ "auth": {
    "identity": {
      "methods": ["password"],
      "password": {
        "user": {
          "name": "gnocchi",
          "domain": { "id": "Default" },
          "password": "p29RAUdcRT7Qfm7fGArwz6yZU"
        }
      }
    }
  }
}'   http://172.16.2.11:5000/auth/tokens ; echo
HTTP/1.1 404 Not Found
Date: Wed, 22 Mar 2017 11:17:45 GMT
Server: Apache
Vary: X-Auth-Token
Content-Length: 93
Content-Type: application/json

{"error": {"message": "The resource could not be found.", "code": 404, "title": "Not Found"}}

and I don't see any entries in keystone log

but if I change url in request to v3 version
()[gnocchi@ca2468f0706b /]$ curl -i   -H "Content-Type: application/json"   -d '
{ "auth": {
    "identity": {
      "methods": ["password"],
      "password": {
        "user": {
          "name": "gnocchi",
          "domain": { "name": "Default" },
          "password": "p29RAUdcRT7Qfm7fGArwz6yZU"
        }
      }
    }
  }
}'   http://172.16.2.11:5000/v3/auth/tokens ; echo
HTTP/1.1 201 Created
Date: Wed, 22 Mar 2017 11:26:15 GMT
Server: Apache
X-Subject-Token: gAAAAABY0l9Xve1SiiMnGjU5riKy8pmi8YyZYoXsdYoFX20vwqjcJnpI0oVCo6wuaEecsM64vnOXsCBNioVP33F-M_ikC7ZbJ8HUIP1Pbus6sKNLGeEk0ZCxRKA5MmRhzSIbtiUbc9TVeENJxLeeRfWX7Okv_Iroyw
Vary: X-Auth-Token
x-openstack-request-id: req-e70223b7-4449-4149-a2a8-fa999a590bd6
Content-Length: 314
Content-Type: application/json

{"token": {"issued_at": "2017-03-22T11:26:15.000000Z", "audit_ids": ["8xsNDdP0R3miXjfF18BDoQ"], "methods": ["password"], "expires_at": "2017-03-22T12:26:15.000000Z", "user": {"password_expires_at": null, "domain": {"id": "default", "name": "Default"}, "id": "b123ae1d06c54a6dac010885c763b61a", "name": "gnocchi"}}}

and from keystone log

2017-03-22 11:25:40.027 17 INFO keystone.common.wsgi [req-2b62e610-b35d-4747-9b0c-aa8ddc425f2c - - - - -] POST http://172.16.2.11:5000/v3/auth/tokens
2017-03-22 11:25:40.130 17 WARNING keystone.common.fernet_utils [req-2b62e610-b35d-4747-9b0c-aa8ddc425f2c - - - - -] key_repository is world readable: /etc/keystone/fernet-keys


Also I tryed change auth_uri and auth_url to 

auth_uri=http://172.16.2.11:5000/v3/

auth_url=http://172.16.2.11:5000/v3/

but result the same
aking authentication request to http://172.16.2.11:5000/auth/tokens get_auth_ref /usr/lib/python2.7/site-packages/keystoneclient/auth/identity/v3/base.py:189
2017-03-22 11:31:21.379 460 DEBUG keystoneclient.session [-] Request returned failure status: 404 request /usr/lib/python2.7/site-packages/keystoneclient/session.py:444
2017-03-22 11:31:21.379 460 CRITICAL gnocchi [-] ClientException: Authorization Failure. Authorization failed: The resource could not be found. (HTTP 404)

Comment 9 Martin André 2017-03-22 14:44:07 UTC
We're also seing this issue upstream.

According to [1] the patch at https://review.openstack.org/#/c/448077/2 fixes gnocchi-statsd accessing swift but highlighted another issue where the gnocchi_statsd container image missing the redis python module. This was also fixed upstream by adding python-redis as a dependency of python-tooz but the image from https://hub.docker.com/r/tripleoupstream/centos-binary-gnocchi-statsd/ was older than the fix. I'm now uploading a newer image to the tripleoupstream registry.

[1] http://logs.openstack.org/77/448077/2/check-tripleo/gate-tripleo-ci-centos-7-ovb-containers-oooq-nv/a878bdf/logs/oooq/overcloud-controller-0/var/log/extra/docker/containers/gnocchi_statsd/log/gnocchi/gnocchi-statsd.log.txt.gz

Comment 10 Martin André 2017-05-22 15:39:08 UTC
The issues were fixed upstream.

Comment 19 Omri Hochman 2017-10-26 19:48:18 UTC
unable to reproduce downstream puddle : 
OpenStack-12.0-RHEL-7 Puddle: 2017-10-24.3

Comment 23 errata-xmlrpc 2017-12-13 19:14:33 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHEA-2017:3457


Note You need to log in before you can comment on or make changes to this bug.