Description of problem: Collectd records cannot be stored - get the following error in the bulk index response: "status"=>400, "error"=>{ "type"=>"mapper_parsing_exception", "reason"=>"failed to parse [collectd.statsd.host_nic_tx]", "caused_by"=>{"type"=>"json_parse_exception", "reason"=>"Numeric value (68554252414) out of range of int\n at [Source: org.elasticsearch.common.io.stream.InputStreamStreamInput@32b1a580; line: 1, column: 195]"}}} I also see the same error for the field collectd.swap.swap, collectd.processes.ps_vm, collectd.processes.ps_data, and more Version-Release number of selected component (if applicable): How reproducible: Steps to Reproduce: 1. 2. 3. Actual results: Expected results: Additional info: Probably many/most of these fields need to be changed from integer to long: https://github.com/ViaQ/elasticsearch-templates/blob/master/namespaces/collectd.yml as per the Elasticsearch documentation: https://www.elastic.co/guide/en/elasticsearch/reference/2.4/mapping-types.html
https://github.com/ViaQ/elasticsearch-templates/commit/4f7ff2a87ff130d21ee31624638763d13963798c
https://github.com/ViaQ/elasticsearch-templates/releases/tag/0.0.10
https://github.com/openshift/origin-aggregated-logging/pull/591
koji_builds: https://brewweb.engineering.redhat.com/brew/buildinfo?buildID=587926 repositories: brew-pulp-docker01.web.prod.ext.phx2.redhat.com:8888/openshift3/logging-elasticsearch:rhaos-3.6-rhel-7-docker-candidate-27813-20170823203718 brew-pulp-docker01.web.prod.ext.phx2.redhat.com:8888/openshift3/logging-elasticsearch:latest brew-pulp-docker01.web.prod.ext.phx2.redhat.com:8888/openshift3/logging-elasticsearch:v3.6 brew-pulp-docker01.web.prod.ext.phx2.redhat.com:8888/openshift3/logging-elasticsearch:v3.6.173.0.5 brew-pulp-docker01.web.prod.ext.phx2.redhat.com:8888/openshift3/logging-elasticsearch:v3.6.173.0.5-6
@Rich, Could you please help advise the scenario where I can verify the bug fix? Thanks, Xia
Shirly, is this something that RHV QE can help with? - create a namespace oadm new-project ovirt-metrics-collectd-test --node-selector='' - get the uuid of the project uuid=$( oc get project ovirt-metrics-collectd-test -o jsonpath='{ .metadata.uid }' ) - create an index name indexname="project.ovirt-metrics-collectd-test.$uuid.$(date -u +'%Y.%m.%d')" - create a test record cat > test.json <<EOF { "collectd": { "processes": { "ps_vm": 4298063872 } } } EOF - add record cat test.json | oc exec -i $espod -- curl -s -k --cert /etc/elasticsearch/secret/admin-cert --key /etc/elasticsearch/secret/admin-key https://localhost:9200/$indexname/multi-tenancy-test/" -XPOST --data-binary @- | python -mjson.tool
(In reply to Rich Megginson from comment #6) > Shirly, is this something that RHV QE can help with? > > - create a namespace > > oadm new-project ovirt-metrics-collectd-test --node-selector='' > > - get the uuid of the project > > uuid=$( oc get project ovirt-metrics-collectd-test -o jsonpath='{ > .metadata.uid }' ) > > - create an index name > > indexname="project.ovirt-metrics-collectd-test.$uuid.$(date -u > +'%Y.%m.%d')" > > - create a test record > > cat > test.json <<EOF > { > "collectd": { > "processes": { > "ps_vm": 4298063872 > } > } > } > EOF > > - add record > > cat test.json | oc exec -i $espod -- curl -s -k --cert > /etc/elasticsearch/secret/admin-cert --key > /etc/elasticsearch/secret/admin-key > https://localhost:9200/$indexname/multi-tenancy-test/" -XPOST --data-binary > @- | python -mjson.tool How should they install viaq for testing?
(In reply to Shirly Radco from comment #7) > (In reply to Rich Megginson from comment #6) > > Shirly, is this something that RHV QE can help with? > > How should they install viaq for testing? Does this refer to the logging system with have viaq template embedded? I'm having issue inserting the record into es, the cursor hangs at curl command: # oc exec logging-es-data-master-4w1n5zto-1-3xqql -- curl -s -k --cert /etc/elasticsearch/secret/admin-cert --key /etc/elasticsearch/secret/admin-key https://localhost:9200/_cat/indices green open project.default.507ff221-91e0-11e7-8996-fa163e14042b.2017.09.05 1 0 2773 0 1.7mb 1.7mb green open project.install-test.6403e17c-91e1-11e7-8996-fa163e14042b.2017.09.05 1 0 2812 0 1.7mb 1.7mb green open project.logging.085f90a5-91e1-11e7-8996-fa163e14042b.2017.09.05 1 0 2814 0 2.2mb 2.2mb green open .kibana 1 0 1 0 3.1kb 3.1kb green open .searchguard.logging-es-data-master-4w1n5zto 1 0 5 0 28kb 28kb green open .operations.2017.09.05 1 0 642576 0 264.9mb 264.9mb # oc rsh logging-es-data-master-4w1n5zto-1-3xqql sh-4.2$ cat > test.json <<EOF > { > "collectd": { > "processes": { > "ps_vm": 4298063872 > } > } > } > EOF sh-4.2$ ls -al | grep test.json -rw-r--r--. 1 1000060000 root 75 Sep 5 10:17 test.json sh-4.2$ curl -s -k --cert /etc/elasticsearch/secret/admin-cert --key /etc/elasticsearch/secret/admin-key https://localhost:9200/project.logging.085f90a5-91e1-11e7-8996-fa163e14042b.2017.09.05/multi-tenancy-test/" -XPOST --data-binary @test.json > @Shirly Radco Is this something you can help with? Let me know if you need the openshift env with logging stacks installed, I can provide it in case you have the libra.pem key file.
Try running the curl command with `-v` - curl -s -v -k ..... Also, try using `--data` instead of `--data-binary`
The post [1,2] succeed with openshift3/logging-elasticsearch:v3.6.173.0.5. But the post failed when the number is in (2^63-192) to (2^63-1). As per https://www.elastic.co/guide/en/elasticsearch/reference/2.4/number.html, (2^63-1) should be an acceptable value. What happened to the back-end? Is (2^63-193) enough for our requirement? 1. "ps_vm": 4298063872 2. "ps_vm": 9223372036854775807 (2^63-193) 3. "ps_vm": (2^63-192) to (2^63-1) cat test.json| curl -s -k --cert /etc/elasticsearch/secret/admin-cert --key /etc/elasticsearch/secret/admin-key https://localhost:9200/project.ovirt-metrics-collectd-test.7d94b436-945f-11e7-bc00-fa163e8efe63.2017.09.08/multi-tenancy-test/ -XPOST --data-binary @- {"error":{"root_cause":[{"type":"mapper_parsing_exception","reason":"failed to parse [_source]"}],"type":"mapper_parsing_exception","reason":"failed to parse [_source]","caused_by":{"type":"illegal_state_exception","reason":"No matching token for number_type [BIG_INTEGER]"}},"status":400}
(In reply to Anping Li from comment #10) > The post [1,2] succeed with openshift3/logging-elasticsearch:v3.6.173.0.5. > But the post failed when the number is in (2^63-192) to (2^63-1). As per > https://www.elastic.co/guide/en/elasticsearch/reference/2.4/number.html, > (2^63-1) should be an acceptable value. What happened to the back-end? Is > (2^63-193) enough for our requirement? > > 1. "ps_vm": 4298063872 > 2. "ps_vm": 9223372036854775807 (2^63-193) > > 3. "ps_vm": (2^63-192) to (2^63-1) > cat test.json| curl -s -k --cert /etc/elasticsearch/secret/admin-cert --key > /etc/elasticsearch/secret/admin-key > https://localhost:9200/project.ovirt-metrics-collectd-test.7d94b436-945f- > 11e7-bc00-fa163e8efe63.2017.09.08/multi-tenancy-test/ -XPOST --data-binary > @- > {"error":{"root_cause":[{"type":"mapper_parsing_exception","reason":"failed > to parse [_source]"}],"type":"mapper_parsing_exception","reason":"failed to > parse [_source]","caused_by":{"type":"illegal_state_exception","reason":"No > matching token for number_type [BIG_INTEGER]"}},"status":400} Hmm - this sounds like a bug in Elasticsearch - they document that a "long" field can store 2^63-1 @lvlcek @pportant - are you guys aware of any Elasticsearch bugs or issues related to this?
I my ES instance, I see: $ curl -X PUT elasticsearch.example.com/test-big-num/bignum/1 -d '{ "ps_vm": 9223372036854775808 }' {"error":{"root_cause":[{"type":"mapper_parsing_exception","reason":"failed to parse [ps_vm]"}],"type":"mapper_parsing_exception","reason":"failed to parse [ps_vm]","caused_by":{"type":"json_parse_exception","reason":"Numeric value (9223372036854775808) out of range of long (-9223372036854775808 - 9223372036854775807)\n at [Source: org.elasticsearch.common.io.stream.InputStreamStreamInput@c59bdad; line: 1, column: 31]"}},"status":400} $ curl -X GET elasticsearch.example.com/ { "name" : "staging-perf34", "cluster_name" : "elasticsearch.perf-dept", "cluster_uuid" : "3UFFaKFYSOqcR59S-9Zm1A", "version" : { "number" : "2.4.6", "build_hash" : "5376dca9f70f3abef96a77f4bb22720ace8240fd", "build_timestamp" : "2017-07-18T12:17:44Z", "build_snapshot" : false, "lucene_version" : "5.5.4" }, "tagline" : "You Know, for Search" }
From https://github.com/elastic/elasticsearch/issues/21728#issuecomment-262235966, ES does not support unsigned long. You can store the field as text or as a double with a mapping.
(In reply to Peter Portante from comment #13) > From > https://github.com/elastic/elasticsearch/issues/21728#issuecomment-262235966, > ES does not support unsigned long. The value 2^63-1 is 9223372036854775807 This value is within the range of Elasticsearch signed long: -9223372036854775808 - 9223372036854775807 - assuming the range is inclusive, then 2^63-1 is a legal value. This appears to be a bug in Elasticsearch - it doesn't accept a value that it says it should accept. > > You can store the field as text or as a double with a mapping.
curl -X PUT elasticsearch.example.com/test-big-num/bignum/1 -d '{ "ps_vm": 9223372036854775807 }' {"_index":"test-big-num","_type":"bignum","_id":"1","_version":1,"_shards":{"total":1,"successful":1,"failed":0},"created":true} Works fine, so what is the bug?
(In reply to Peter Portante from comment #15) > curl -X PUT elasticsearch.example.com/test-big-num/bignum/1 -d '{ "ps_vm": > 9223372036854775807 }' > {"_index":"test-big-num","_type":"bignum","_id":"1","_version":1,"_shards": > {"total":1,"successful":1,"failed":0},"created":true} > > Works fine, so what is the bug? Anping - can you verify the above result?
It works. move to verified. 2^63-1 is 9223372036854775807. There is a numerical error. The online scientific calculator give me a wrong value (2^63=2^63=9223372036854776000).
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2017:3049