Bug 1679851

For some reason OSP is not using the latest tag. Working on a patch to workaround this. Meanwhile can you try to deploy with the following parameter:

~~~
parameter_defaults:
    SkydiveVars:
        agents:
            skydive_docker_image_tag: 14.0-48
~~~

*** Bug 1679852 has been marked as a duplicate of this bug. ***

Another option could be 

~~~
parameter_defaults:
    SkydiveVars:
        globals:
            skydive_docker_image_tag: 14.0
~~~

It doesn't seem similar to what I suggested. The goal was to force the use of the agent tag (14.0-48). It seems you specify the wrong tag (analyzer tag).

~~~
10.74.167.35:8787/red_hat_ecs-osp14_containers-skydive-analyzer             14.0-46
10.74.167.35:8787/red_hat_ecs-osp14_containers-skydive-agent                14.0-48
~~~

~~~
parameter_defaults:
    SkydiveVars:
        agents:
            skydive_docker_image_tag: 14.0-48
~~~

Thanks for your test

I will check why skydive_docker_image_tag is not applied correctly under agents section here https://git.openstack.org/cgit/openstack/tripleo-heat-templates/tree/extraconfig/services/skydive-analyzer.yaml?h=refs/changes/06/638606/1#n146


As a workaround, could you both docker tag images to 14.0 and use scenario 2 (globals: skydive_docker_image_tag: 14.0) ?

Hi Nicolas,

The workaround to tag both images to 14.0 and setting the global skydive_docker_image_tag to 14.0 didn't work either. As the line 97 in extraconfig/services/skydive-analyzer.yaml ( skydive_docker_image_tag: {{skydive_analyzer_docker_image | regex_replace(".*:")}}` ) is extracting the tag from analyzer docker image and the playbook is using it as part of extra-vars passed to ansible, which overrides the vars in inventory file.

So the workaround should be, to first to tag the images to 14.0 and then update the DockerSkydiveAnalyzerImage  in overcloud_images.yaml (if you have generated one) with 14.0 

In my lab setup, after setting the global skydive_docker_image_tag to 14.0, the inventory.yml file was correctly populated with 14.0, but the global_defaults.yml file was having 14.0-46. Below are two tests showing the effect --extra-vars.

sudo ansible -i ./inventory.yml  --private-key /var/lib/mistral/overcloud/ssh_private_key agents -m shell -a 'echo "{{skydive_docker_image_tag}}"'
lab-computehci-01 | SUCCESS | rc=0 >>
14.0

sudo ansible -i ./inventory.yml  --extra-vars '@/var/lib/mistral/overcloud/skydive/global_defaults.yml' --private-key /var/lib/mistral/overcloud/ssh_private_key agents -m shell -a 'echo "{{skydive_docker_image_tag}}"'
lab-computehci-02 | SUCCESS | rc=0 >>
14.0-46


On a related note the upstream ansible playbooks for skydive is updated to include skydive_agent_docker_image_tag and skydive_analyzer_docker_image_tag , so the ideal fix would be to pull the latest ansible playbooks to mistral container image and then to update extraconfig/services/skydive-analyzer.yaml to include those vars.

Thanks
M Salih

Sorry, my bad, I had wrong setting in parameter_defaults. Instead of 'globals' I was having 'agents'. So setting globals like following and tagging the images to 14.0 should work.

parameter_defaults:
  SkydiveVars:
    globals:
      skydive_docker_image_tag: 14.0

Hi Mohammed,

(In reply to Mohammed Salih from comment #10)
> Sorry, my bad, I had wrong setting in parameter_defaults. Instead of
> 'globals' I was having 'agents'. So setting globals like following and
> tagging the images to 14.0 should work.
> 
> parameter_defaults:
>   SkydiveVars:
>     globals:
>       skydive_docker_image_tag: 14.0

No luck to pull the skydive image with suggested parameters.

parameter_defaults:
  SkydiveVars:
    globals:
      skydive_docker_image_tag: 14.0


$ grep sky overcloud_deploy.sh 
-e /usr/share/openstack-tripleo-heat-templates/environments/services/skydive-environment.yaml \


Error:

   "fatal: [nfv-compute-0]: FAILED! => {\"changed\": true, \"cmd\": \"docker pull 10.74.167.35:8787/red_hat_ecs-osp14_containers-skydive-agent:14.0\", \"delta\": \"0:00:00.113966\", \"end\": \"2019-03-01 13:48:14.659469\", \"msg\": \"non-zero return code\", \"rc\": 1, \"start\": \"2019-03-01 13:48:14.545503\", \"stderr\": \"Error: image red_hat_ecs-osp14_containers-skydive-agent:14.0 not found\", \"stderr_lines\": [\"Error: image red_hat_ecs-osp14_containers-skydive-agent:14.0 not found\"], \"stdout\": \"Trying to pull repository 10.74.167.35:8787/red_hat_ecs-osp14_containers-skydive-agent ... \\nPulling repository 10.74.167.35:8787/red_hat_ecs-osp14_containers-skydive-agent\", \"stdout_lines\": [\"Trying to pull repository 10.74.167.35:8787/red_hat_ecs-osp14_containers-skydive-agent ... \", \"Pulling repository 10.74.167.35:8787/red_hat_ecs-osp14_containers-skydive-agent\"]}",
    "fatal: [nfv-compute-1]: FAILED! => {\"changed\": true, \"cmd\": \"docker pull 10.74.167.35:8787/red_hat_ecs-osp14_containers-skydive-agent:14.0\", \"delta\": \"0:00:00.119783\", \"end\": \"2019-03-01 13:48:14.734286\", \"msg\": \"non-zero return code\", \"rc\": 1, \"start\": \"2019-03-01 13:48:14.614503\", \"stderr\": \"Error: image red_hat_ecs-osp14_containers-skydive-agent:14.0 not found\", \"stderr_lines\": [\"Error: image red_hat_ecs-osp14_containers-skydive-agent:14.0 not found\"], \"stdout\": \"Trying to pull repository 10.74.167.35:8787/red_hat_ecs-osp14_containers-skydive-agent ... \\nPulling repository 10.74.167.35:8787/red_hat_ecs-osp14_containers-skydive-agent\", \"stdout_lines\": [\"Trying to pull repository 10.74.167.35:8787/red_hat_ecs-osp14_containers-skydive-agent ... \", \"Pulling repository 10.74.167.35:8787/red_hat_ecs-osp14_containers-skydive-agent\"]}",
    "fatal: [nfv-controller-0]: FAILED! => {\"changed\": true, \"cmd\": \"docker pull 10.74.167.35:8787/red_hat_ecs-osp14_containers-skydive-agent:14.0\", \"delta\": \"0:00:00.132242\", \"end\": \"2019-03-01 13:48:14.776990\", \"msg\": \"non-zero return code\", \"rc\": 1, \"start\": \"2019-03-01 13:48:14.644748\", \"stderr\": \"Error: image red_hat_ecs-osp14_containers-skydive-agent:14.0 not found\", \"stderr_lines\": [\"Error: image red_hat_ecs-osp14_containers-skydive-agent:14.0 not found\"], \"stdout\": \"Trying to pull repository 10.74.167.35:8787/red_hat_ecs-osp14_containers-skydive-agent ... \\nPulling repository 10.74.167.35:8787/red_hat_ecs-osp14_containers-skydive-agent\", \"stdout_lines\": [\"Trying to pull repository 10.74.167.35:8787/red_hat_ecs-osp14_containers-skydive-agent ... \", \"Pulling repository 10.74.167.35:8787/red_hat_ecs-osp14_containers-skydive-agent\"]}",


Thanks,
Pradipta

Hi Pradipta,

Have you tagged the agent and analyzer container image with 14.0 , I used the following commands to copy (may be there is a better way to tag the images), of-course you have to change the IP of the registry.

skopeo copy --dest-tls-verify=false --src-tls-verify=false docker://172.16.200.10:8787/rhosp14/openstack-skydive-analyzer:14.0-46 docker://172.16.200.10:8787/rhosp14/openstack-skydive-analyzer:14.0
skopeo copy --dest-tls-verify=false --src-tls-verify=false docker://172.16.200.10:8787/rhosp14/openstack-skydive-agent:14.0-48 docker://172.16.200.10:8787/rhosp14/openstack-skydive-agent:14.0

You can verify the images by 

skopeo inspect --tls-verify=false docker://172.16.200.10:8787/rhosp14/openstack-skydive-analyzer:14.0
skopeo inspect --tls-verify=false docker://172.16.200.10:8787/rhosp14/openstack-skydive-agent:14.0

Once this is done , you can re-deploy overcloud.

Hope it helps.
M Salih

Hi Salih,

Thank you very much for sharing the tips. It helps to fixed skydive integration issue.

But I think for a long term solution, it should work with the non-uniform image tag.

Regards,
Pradipta

From what I can see in the skydive-ansible github repo, I think that is being addressed as we speak, and probably would be addressed in openstack tripleo-heat-templates.

Cheers
M Salih

The version skydive-0.20.3-1.el7ost that will be part of Z should fix the issues.

*** This bug has been marked as a duplicate of bug 1676915 ***