Bug 1538560 - [RFE]rename the project name mux-undefined
Summary: [RFE]rename the project name mux-undefined
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: Logging
Version: 3.9.0
Hardware: Unspecified
OS: Unspecified
high
low
Target Milestone: ---
: 3.10.0
Assignee: Noriko Hosoi
QA Contact: Qiaoling Tang
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2018-01-25 10:14 UTC by Anping Li
Modified: 2018-08-31 06:18 UTC (History)
5 users (show)

Fixed In Version:
Doc Type: Enhancement
Doc Text:
When Mux is configured and it fails to find a project/namespace that a log belongs to, the log was indexed into project.mux-undefined where mux-undefined was a mux default namespace. At the same time, the fluentd (without Mux configuration) puts such logs into the .orphaned.YYYY.MM.DD index. With this fix, such orphaned logs are also indexed into the .orphaned.YYYY.MM.DD index for the Mux case.
Clone Of:
Environment:
Last Closed: 2018-08-31 06:18:10 UTC


Attachments (Terms of Use)


Links
System ID Priority Status Summary Last Updated
Red Hat Product Errata RHBA-2018:2376 None None None 2018-08-31 06:18:58 UTC

Description Anping Li 2018-01-25 10:14:16 UTC
Description of problem:
Openshift preserve the prefix openshift- and kube- for system usage. It is better to add prefix openshift- to mux-undefined.

Version-Release number of selected component (if applicable):
v3.7,v3.8,v3.9

How reproducible:
always

Steps to Reproduce:
1. install openshift
2. Check the project name installed by default.
[root@host3-ha-master-1 ~]# oc get projects
NAME                                DISPLAY NAME   STATUS
default                                            Active
kube-public                                        Active
kube-service-catalog                               Active
kube-system                                        Active
logging                                            Active
management-infra                                   Active
mux-undefined                                      Active
openshift                                          Active
openshift-ansible-service-broker                   Active
openshift-infra                                    Active
openshift-node                                     Active
openshift-template-service-broker                  Active
openshift-web-console                              Active

Actual results:
project name : mux-undefined


Expected results:
project name: openshift-mux-undefined or other

Additional info:

Comment 1 Rich Megginson 2018-01-25 14:39:16 UTC
Or, use .orphan or .orphaned or whatever we are using for records where we could not find the namespace.  It is the same problem - we are attempting to process a record and we do not know what the namespace is.

Comment 3 Noriko Hosoi 2018-04-04 19:49:09 UTC
(In reply to Rich Megginson from comment #1)
> Or, use .orphan or .orphaned or whatever we are using for records where we
> could not find the namespace.  It is the same problem - we are attempting to
> process a record and we do not know what the namespace is.

Could it be possible to have '.' for the namespace/project name?  Just replacing mux-undefined with .orphaned did not work...  Is it ok without '.'?

============================================================================
TASK [openshift_logging_mux : Add mux namespaces] ******************************
task path: /usr/share/ansible/openshift-ansible/roles/openshift_logging_mux/tasks/main.yaml:22
0

<localhost> EXEC /bin/sh -c 'sudo -H -S -n -u root /bin/sh -c '"'"'echo BECOME-SUCCESS-alyossphfguzxwqsgoojjbvzflsmpbzy; /usr/bin/python /home/origin/.ansible/tmp/ansible-tmp-1522862486.42-161394758859912/oc_project.py; rm -rf "/home/origin/.ansible/tmp/ansible-tmp-1522862486.42-161394758859912/" > /dev/null 2>&1'"'"' && sleep 0'
The full traceback is:
  File "/tmp/ansible_bPRBHR/ansible_module_oc_project.py", line 47, in <module>
    import ruamel.yaml as yaml
failed: [localhost] (item=.orphaned) => {
    "changed": false, 
    "generated_timestamp": "2018-04-04 17:21:27.471423", 
    "invocation": {
        "module_args": {
            "admin": null, 
            "admin_role": "admin", 
            "debug": false, 
            "description": null, 
            "display_name": null, 
            "kubeconfig": "/etc/origin/master/admin.kubeconfig", 
            "name": ".orphaned", 
            "node_selector": [
                ""
            ], 
            "state": "present"
        }
    }, 
    "item": ".orphaned", 
    "msg": {
        "cmd": "/bin/oc adm new-project .orphaned --admin-role=admin --node-selector=", 
        "results": {}, 
        "returncode": 1, 
        "stderr": "The Project \".orphaned\" is invalid: metadata.name: Invalid value: \".orphaned\": a DNS-1123 label must consist of lower case alphanumeric characters or '-', and must start and end with an alphanumeric character (e.g. 'my-name',  or '123-abc', regex used for validation is '[a-z0-9]([-a-z0-9]*[a-z0-9])?')\n", 
        "stdout": ""
    }

Comment 4 Rich Megginson 2018-04-04 21:16:44 UTC
Right - .orphaned is not a legal namespace/project name.  It works as an index prefix just like .operations (also not a legal namespace/project name).

I think the way it should work is this - in openshift-ansible:

Remove openshift_logging_mux_default_namespaces

Add a warning to roles/openshift_logging_mux/tasks/main.yaml to say that openshift_logging_mux_default_namespaces is not supported and to use openshift_logging_mux_namespaces instead.

Change any docs in openshift-ansible that refer to openshift_logging_mux_default_namespaces to say that openshift_logging_mux_default_namespaces is not supported and to use openshift_logging_mux_namespaces instead.  Also say that 

In origin-aggregated-logging:

change "mux-undefined" to ".orphaned" - including docs
if there are any tests which create project "mux-undefined" just remove that project checking and creation

Comment 5 Noriko Hosoi 2018-04-05 07:00:15 UTC
I'm a bit confused...

If I pass this [1] to ansible-playbook, it fails with the same error of course...

"The Project \".orphaned\" is invalid: metadata.name: Invalid value: \".orphaned\": a DNS-1123 label must consist of lower case alphanumeric characters or '-', and must start and end with an alphanumeric character (e.g. 'my-name',  or '123-abc', regex used for validation is '[a-z0-9]([-a-z0-9]*[a-z0-9])?')\n",

Then, how the system could know the "orphaned" namespace/project would be ".orphaned"???

[1] -e openshift_logging_mux_namespaces=.orphaned

Comment 7 Rich Megginson 2018-04-05 12:11:03 UTC
There is no ".orphaned" namespace/project, just like there is no ".operations" namespace/project.

Treat ".orphaned" the same way we treat ".operations" - it is used as a prefix of a full Elasticsearch index name.  Except that there may be some sort of dummy orphaned "namespace" id used in the index name - @jcantrill would know.

Comment 8 Noriko Hosoi 2018-04-23 14:55:12 UTC
Per discussion on https://github.com/openshift/origin-aggregated-logging/pull/1065, resetting the target release to 3.10.0.

Comment 9 openshift-github-bot 2018-04-23 18:16:46 UTC
Commits pushed to master at https://github.com/openshift/origin-aggregated-logging

https://github.com/openshift/origin-aggregated-logging/commit/9e4a97ee4f60f43624b80821d48da8ad18f8a7f2
Bug 1538560 - [RFE]rename the project name mux-undefined

The logs which failed to associate with the namespace/project will be
indexed into the ES index file .orphaned.YYYY.MM.DD instead of the
project.mux-undefined.*.

In rewrite_tag_filter (input-post-forward-mux.conf), if mux_namespace_name
is .orphaned, we will skip the effort to get k8s_metadata.

https://github.com/openshift/origin-aggregated-logging/commit/fb557c73e6c78e978759b52eb8336c98afde589c
Merge pull request #1065 from nhosoi/bz1538560

Bug 1538560 - [RFE]rename the project name mux-undefined

Comment 10 Noriko Hosoi 2018-04-23 18:19:58 UTC
Could someone please review this pr https://github.com/openshift/openshift-ansible/pull/7792 ?

Comment 11 Noriko Hosoi 2018-04-23 22:17:39 UTC
(In reply to Noriko Hosoi from comment #10)
> Could someone please review this pr
> https://github.com/openshift/openshift-ansible/pull/7792 ?

Correction: pr/7792 is still WIP.

The openshift-ansible pr/7792 removes openshift_logging_mux_default_namespaces, but keeps openshift_logging_mux_namespaces.  Setting the variable [1] makes the logging deployment fail as follows [2].  We should say "openshift_logging_mux_namespaces is not supported" as well?

[1] -e openshift_logging_mux_namespaces=.myorphaned

[2] 
TASK [openshift_logging_mux : Add mux namespaces] ******************************
task path: /usr/share/ansible/openshift-ansible/roles/openshift_logging_mux/tasks/main.yaml:224
Using module file /usr/share/ansible/openshift-ansible/roles/lib_openshift/library/oc_project.py
   ...
The full traceback is:
  File "/tmp/ansible_qBRqou/ansible_module_oc_project.py", line 47, in <module>
    import ruamel.yaml as yaml
failed: [localhost] (item=.myorphaned) => {
   ...
"stderr": "The Project \".myorphaned\" is invalid: metadata.name: Invalid value: \".myorphaned\": a DNS-1123 label must consist of lower case alphanumeric characters or '-', and must start and end with an alphanumeric character (e.g. 'my-name',  or '123-abc', regex used for validation is '[a-z0-9]([-a-z0-9]*[a-z0-9])?')\n",

Comment 12 Rich Megginson 2018-04-23 23:49:03 UTC
> We should say "openshift_logging_mux_namespaces is not supported" as well?

ovirt/rhv rely on this feature, so we must support it.

If you specify a namespace, it must be a valid OpenShift/Kubernetes namespace.

Comment 13 Noriko Hosoi 2018-04-23 23:54:49 UTC
Sorry.  Please ignore #c11.  It was my mistake in my testing.  It just should not have included '.' in the namespace name.  I'm retesting it now...  (I think the current pr/7792 would be ok after all...)

Comment 14 Noriko Hosoi 2018-04-24 04:51:14 UTC
The ci-tests of https://github.com/openshift/openshift-ansible/pull/7792 have passed.  I removed WIP from the pr.  Could someone please review the pr?
Thanks!

Comment 15 openshift-github-bot 2018-04-24 17:30:31 UTC
Commits pushed to master at https://github.com/openshift/openshift-ansible

https://github.com/openshift/openshift-ansible/commit/56f952d734ec521253182cf33e6ed0017835a05c
Bug 1538560 - [RFE]rename the project name mux-undefined

mux-undefined ==> .orphaned
Removing openshift_logging_mux_default_namespaces.

https://github.com/openshift/openshift-ansible/commit/dc50fd346096e07b78d691cd820549627536807e
Merge pull request #7792 from nhosoi/bz1538560

Bug 1538560 - [RFE]rename the project name mux-undefined

Comment 17 openshift-github-bot 2018-06-20 15:06:38 UTC
Commit pushed to master at https://github.com/openshift/origin-aggregated-logging

https://github.com/openshift/origin-aggregated-logging/commit/d71a51ae633509e4f1338e95eefe3d07cffb95a9
Bug 1538560 - [RFE]rename the project name mux-undefined

The logs which failed to associate with the namespace/project will be
indexed into the ES index file .orphaned.YYYY.MM.DD instead of the
project.mux-undefined.*.

In rewrite_tag_filter (input-post-forward-mux.conf), if mux_namespace_name
is .orphaned, we will skip the effort to get k8s_metadata.

Comment 20 Qiaoling Tang 2018-08-23 01:28:18 UTC
Tested on logging-elasticsearch-v3.10.34-2 and logging-fluentd-v3.10.34-1.
There is no project named mux-undefined and logs which failed to associate with the namespace/project are indexed into the ES index file .orphaned.YYYY.MM.DD.

So move to verified.

Comment 22 errata-xmlrpc 2018-08-31 06:18:10 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2018:2376


Note You need to log in before you can comment on or make changes to this bug.