Bug 1420625 - [intservice_public_324] Logging deployment will fail if openshift_logging_fluentd_nodeselector is specified in inventory file
Summary: [intservice_public_324] Logging deployment will fail if openshift_logging_flu...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: Logging
Version: 3.5.0
Hardware: Unspecified
OS: Unspecified
unspecified
medium
Target Milestone: ---
: 3.5.z
Assignee: Jeff Cantrill
QA Contact: Xia Zhao
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2017-02-09 06:20 UTC by Xia Zhao
Modified: 2021-03-11 14:57 UTC (History)
7 users (show)

Fixed In Version:
Doc Type: No Doc Update
Doc Text:
undefined
Clone Of:
Environment:
Last Closed: 2017-10-25 13:00:48 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)
ansible log (76.05 KB, text/plain)
2017-02-09 06:20 UTC, Xia Zhao
no flags Details


Links
System ID Private Priority Status Summary Last Updated
Red Hat Product Errata RHBA-2017:3049 0 normal SHIPPED_LIVE OpenShift Container Platform 3.6, 3.5, and 3.4 bug fix and enhancement update 2017-10-25 15:57:15 UTC

Description Xia Zhao 2017-02-09 06:20:19 UTC
Created attachment 1248739 [details]
ansible log

Description of problem:
Logging deployment failed when openshift_logging_fluentd_nodeselector is specified in inventory file , can succeeded without openshift_logging_fluentd_nodeselector specified in inventory file.

TASK [openshift_logging : fail] ************************************************
task path: /root/openshift-ansible/roles/openshift_logging/tasks/main.yaml:2

MSG:

The conditional check '{{ openshift_logging_fluentd_nodeselector.keys() | count }} > 1' failed. The error was: error while evaluating conditional ({{ openshift_logging_fluentd_nodeselector.keys() | count }} > 1): 'unicode object' has no attribute 'keys'

The error appears to have been in '/root/openshift-ansible/roles/openshift_logging/tasks/main.yaml': line 2, column 3, but may
be elsewhere in the file depending on the exact syntax problem.

The offending line appears to be:

---
- fail:
  ^ here

Version-Release number of selected component (if applicable):
https://github.com/openshift/openshift-ansible

How reproducible:
Always

Steps to Reproduce:
1. Deploy logging 3.5.0 stacks by ansible with openshift_logging_fluentd_nodeselector specified in inventory file:
openshift_logging_fluentd_nodeselector="logging-infra-fluentd-test": "true"

Actual results:
deployment failed 

Expected results:
deployment should succeed

Additional info:
This is a regression, it used to work with repo https://github.com/openshift/openshift-ansible 1 day before.

Comment 1 Jeff Cantrill 2017-02-10 18:17:03 UTC
This is not really a bug.  A nodeSelector can be a complicated hash and should be passed like labels as described here: https://docs.openshift.com/container-platform/latest/install_config/install/advanced_install.html#configuring-ansible

node1.example.com openshift_node_labels="{'region': 'primary', 'zone': 'east'}"

The 'syntax' has changed from 3.4 to 3.5 because the deployer passed the value as a string that was placed in the template where 3.5 uses a proper hash to allow a more complex selector.

Comment 2 Troy Dawson 2017-02-10 22:43:00 UTC
Putting a bug onto MODIFIED is a trigger for it to go onto an errata.
This bug doesn't look like it's suppose to do that.
What is the next step for this bug?

Comment 3 Troy Dawson 2017-02-13 22:31:56 UTC
Moving to ASSIGNED until we determine what to do with this bug.

Comment 4 Jeff Cantrill 2017-02-16 14:47:09 UTC
Moving to 'ON_QA' for re-evaluation.

Comment 5 Xia Zhao 2017-02-20 06:52:41 UTC
Modified the inventory file to 
openshift_logging_fluentd_nodeselector={"logging-infra-fluentd-test": "true"}
and logging deployment finished successfully (with the repro of https://bugzilla.redhat.com/show_bug.cgi?id=1419811). Set to verified, thanks.

Comment 6 Dave Sullivan 2017-08-15 14:47:42 UTC
It's not clear what the status of this bug is.

I'm currently hitting this issue now with 3.5 Advanced Ansible Install, really OpenShift on AWS refarch install.

The vars are configured using yaml format.

I have other nodeselectors that work fine, so not sure why the fluentd nodeselector is not working.

e.g.

This works

openshift_registry_selector: "role=infraregistry"

This doesn't 

openshift_logging_fluentd_nodeselector: "role=infraregistry"

[ec2-user@adpuse1 playbooks]$ cat openshift-setup.yaml
---
- include: /usr/share/ansible/openshift-ansible/playbooks/byo/config.yml
  vars:
    debug_level: 2
    openshift_debug_level: "{{ debug_level }}"
    openshift_node_debug_level: "{{ node_debug_level | default(debug_level, true) }}"
    openshift_node_kubelet_args:
      node-labels:
      - "role={{ openshift_node_labels.role }}"
    openshift_master_debug_level: "{{ master_debug_level | default(debug_level, true) }}"
    openshift_master_access_token_max_seconds: 2419200
    openshift_master_api_port: "{{ console_port }}"
    openshift_master_console_port: "{{ console_port }}"
    osm_cluster_network_cidr: "{{ ocp_cluster_pod_cidr }}"
    openshift_portal_net: "{{ ocp_cluster_service_cidr }}"
    osm_host_subnet_length: 6
    openshift_registry_selector: "role=infraregistry"
    openshift_logging_install_logging: true
    openshift_hosted_logging_deploy: true
    openshift_logging_curator_nodeselector: "role=infraregistry"
    openshift_logging_kibana_nodeselector: "role=infraregistry"
    openshift_logging_es_nodeselector: "role=infraregistry"
    openshift_logging_fluentd_nodeselector: "role=infraregistry"
    ..l
    openshift_router_selector: "role=infrarouter"
    openshift_hosted_router_replicas: 3
    openshift_hosted_registry_replicas: 3
    openshift_master_cluster_method: native
    openshift_node_local_quota_per_fsgroup: 512Mi
    openshift_master_cluster_hostname: "{{ cluster_shortname }}.{{ public_hosted_zone }}"
    openshift_master_cluster_public_hostname: "{{ cluster_shortname }}.{{ public_hosted_zone }}"
    openshift_default_registry: "docker-registry.default.svc.cluster.local:5000"
    osm_default_subdomain: "{{ wildcard_zone }}"
    openshift_hostname: "{{ inventory_hostname }}"
    openshift_master_default_subdomain: "{{osm_default_subdomain}}"
    osm_default_node_selector: "role=app"
    deployment_type: openshift-enterprise
    os_sdn_network_plugin_name: "redhat/{{ openshift_sdn }}"
    openshift_master_identity_providers:
…
    osm_use_cockpit: true
    containerized: false
    openshift_hosted_registry_storage_kind: object
    openshift_hosted_registry_storage_provider: s3
    openshift_hosted_registry_storage_s3_accesskey: "{{ hostvars['localhost']['s3user_id'] }}"
    openshift_hosted_registry_storage_s3_secretkey: "{{ hostvars['localhost']['s3user_secret'] }}"
    openshift_hosted_registry_storage_s3_bucket: "{{ hostvars['localhost']['s3_bucket_name'] }}"
    openshift_hosted_registry_storage_s3_region: "{{ hostvars['localhost']['region'] }}"
    openshift_hosted_registry_storage_s3_chunksize: 26214400
    openshift_hosted_registry_storage_s3_rootdirectory: /registry
    openshift_hosted_registry_pullthrough: true
    openshift_hosted_registry_acceptschema2: true
    openshift_hosted_registry_enforcequota: true

Comment 7 Dave Sullivan 2017-08-15 14:48:03 UTC
It's not clear what the status of this bug is.

I'm currently hitting this issue now with 3.5 Advanced Ansible Install, really OpenShift on AWS refarch install.

The vars are configured using yaml format.

I have other nodeselectors that work fine, so not sure why the fluentd nodeselector is not working.

e.g.

This works

openshift_registry_selector: "role=infraregistry"

This doesn't 

openshift_logging_fluentd_nodeselector: "role=infraregistry"

[ec2-user@adpuse1 playbooks]$ cat openshift-setup.yaml
---
- include: /usr/share/ansible/openshift-ansible/playbooks/byo/config.yml
  vars:
    debug_level: 2
    openshift_debug_level: "{{ debug_level }}"
    openshift_node_debug_level: "{{ node_debug_level | default(debug_level, true) }}"
    openshift_node_kubelet_args:
      node-labels:
      - "role={{ openshift_node_labels.role }}"
    openshift_master_debug_level: "{{ master_debug_level | default(debug_level, true) }}"
    openshift_master_access_token_max_seconds: 2419200
    openshift_master_api_port: "{{ console_port }}"
    openshift_master_console_port: "{{ console_port }}"
    osm_cluster_network_cidr: "{{ ocp_cluster_pod_cidr }}"
    openshift_portal_net: "{{ ocp_cluster_service_cidr }}"
    osm_host_subnet_length: 6
    openshift_registry_selector: "role=infraregistry"
    openshift_logging_install_logging: true
    openshift_hosted_logging_deploy: true
    openshift_logging_curator_nodeselector: "role=infraregistry"
    openshift_logging_kibana_nodeselector: "role=infraregistry"
    openshift_logging_es_nodeselector: "role=infraregistry"
    openshift_logging_fluentd_nodeselector: "role=infraregistry"
    ..l
    openshift_router_selector: "role=infrarouter"
    openshift_hosted_router_replicas: 3
    openshift_hosted_registry_replicas: 3
    openshift_master_cluster_method: native
    openshift_node_local_quota_per_fsgroup: 512Mi
    openshift_master_cluster_hostname: "{{ cluster_shortname }}.{{ public_hosted_zone }}"
    openshift_master_cluster_public_hostname: "{{ cluster_shortname }}.{{ public_hosted_zone }}"
    openshift_default_registry: "docker-registry.default.svc.cluster.local:5000"
    osm_default_subdomain: "{{ wildcard_zone }}"
    openshift_hostname: "{{ inventory_hostname }}"
    openshift_master_default_subdomain: "{{osm_default_subdomain}}"
    osm_default_node_selector: "role=app"
    deployment_type: openshift-enterprise
    os_sdn_network_plugin_name: "redhat/{{ openshift_sdn }}"
    openshift_master_identity_providers:
…
    osm_use_cockpit: true
    containerized: false
    openshift_hosted_registry_storage_kind: object
    openshift_hosted_registry_storage_provider: s3
    openshift_hosted_registry_storage_s3_accesskey: "{{ hostvars['localhost']['s3user_id'] }}"
    openshift_hosted_registry_storage_s3_secretkey: "{{ hostvars['localhost']['s3user_secret'] }}"
    openshift_hosted_registry_storage_s3_bucket: "{{ hostvars['localhost']['s3_bucket_name'] }}"
    openshift_hosted_registry_storage_s3_region: "{{ hostvars['localhost']['region'] }}"
    openshift_hosted_registry_storage_s3_chunksize: 26214400
    openshift_hosted_registry_storage_s3_rootdirectory: /registry
    openshift_hosted_registry_pullthrough: true
    openshift_hosted_registry_acceptschema2: true
    openshift_hosted_registry_enforcequota: true

Comment 8 Jeff Cantrill 2017-08-30 15:22:45 UTC
@Dan see comment#5 which says to specify the selector as a dict

Comment 10 errata-xmlrpc 2017-10-25 13:00:48 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2017:3049


Note You need to log in before you can comment on or make changes to this bug.