What did you use for your IMAGE_PREFIX and IMAGE_VERSION in the `oc process logging-deployer-template` command?
https://docs.openshift.com/enterprise/3.1/install_config/aggregate_logging.html#deploying-the-efk-stack
It's *supposed* to default to the right thing but maybe it's wrong or you did something non-standard. Definitely the image name shouldn't need manual modification.

(In reply to Luke Meyer from comment #1)
> What did you use for your IMAGE_PREFIX and IMAGE_VERSION in the `oc process
> logging-deployer-template` command?

I didn't set IMAGE_PREFIX and IMAGE_VERSION in the `oc process logging-deployer-template` command. 

I didn't think it is necessary, since it has been set by default:

~~~
  name: IMAGE_PREFIX
  value: "registry.access.redhat.com/openshift3/"
~~~

https://github.com/openshift/openshift-ansible/blob/master/roles/openshift_examples/files/examples/v1.0/infrastructure-templates/enterprise/logging-deployer.yaml#L83-L85

Moving this out of 'must fix' as there is a work around if the default is not being set correctly.  The DC can be manually updated.

I think I understand the issue. I don't think it's a bug, but it points to something else that has gone wrong.

How this works is that when the DC is created, the pod spec has a placeholder image setting. This is supposed to be overwritten immediately by the ImageChange trigger. E.g. if you look at logging-fluentd-template it comes out as:

    template:
      metadata:
        labels:
          component: fluentd
          provider: openshift
        name: fluentd-elasticsearch
      spec:
        containers:
        - image: logging-fluentd     <--- placeholder value
          imagePullPolicy: Always
          name: fluentd-elasticsearch
          [...]
    triggers:
    - type: ConfigChange
    - imageChangeParams:             <--- specifies overwrite
        automatic: true
        containerNames:
        - fluentd-elasticsearch
        from:                      
          kind: ImageStreamTag
          name: logging-fluentd:latest
      type: ImageChange

Once the DC is actually created, the image in the template.spec should be changed to the real value, e.g. registry.access.redhat.com/openshift3/logging-fluentd:latest, which comes from the ImageStream "logging-fluentd". If, however, your IS hasn't been populated for some reason, the trigger won't fire and the DC will just try to use the placeholder.

I'm not sure what would cause ImageStream population to fail if you can then just change the image name to the right value and it starts working. ImageStreams do have their own docker client settings (I know this from trying to use an internal registry with an unknown CA, the IS doesn't use the system docker settings for this) however this is working fine with registry.access.redhat.com in all my tests.

It would probably be helpful to get the output of `oc describe is/logging-fluentd` (and/or any others that are misbehaving) as well as master logs if you see this occur.

Since the customer case is closed now (sorry for the delay) shall we close this bug and see if we can get further information if someone reproduces this?

Thank you for your detail explanation.
I confirmed that the DC has "image: registry.access.redhat.com/openshift3/logging-xxxx" after I ran:

  $ oc process logging-support-template | oc create -f -

As you explained, I also guess that ImageStream population from logging-support-template has been failed.
Please feel free to close this bugzilla ticket.