Bug 1351504 - elasticsearch doesn't start after updates
Summary: elasticsearch doesn't start after updates
Keywords:
Status: CLOSED EOL
Alias: None
Product: Fedora
Classification: Fedora
Component: elasticsearch
Version: 28
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: ---
Assignee: Zbigniew Jędrzejewski-Szmek
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
: 1418751 (view as bug list)
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2016-06-30 08:33 UTC by Jiri Popelka
Modified: 2019-05-28 22:38 UTC (History)
18 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2019-05-28 22:38:08 UTC
Type: Bug
Embargoed:


Attachments (Terms of Use)

Description Jiri Popelka 2016-06-30 08:33:30 UTC
Version-Release number of selected component (if applicable):
elasticsearch-1.7.1-3.fc24.noarch
java-1.8.0-openjdk-1.8.0.92-3.b14.fc24.x86_64

Steps to Reproduce:
1. dnf install elasticsearch
2. systemctl start elasticsearch

Actual results:
Jun 30, 2016 10:26:50 AM org.elasticsearch.bootstrap.Natives <clinit>
WARNING: JNA not found. native methods will be disabled.
Jun 30, 2016 10:26:50 AM org.elasticsearch.bootstrap.Natives addConsoleCtrlHandler
WARNING: cannot register console handler because JNA is not available
Jun 30, 2016 10:26:50 AM org.elasticsearch.node.internal.InternalNode <init>
INFO: [D'Ken] version[1.7.1], pid[31648], build[${build/NA]
Jun 30, 2016 10:26:50 AM org.elasticsearch.node.internal.InternalNode <init>
INFO: [D'Ken] initializing ...
Jun 30, 2016 10:26:50 AM org.elasticsearch.plugins.PluginsService <init>
INFO: [D'Ken] loaded [], sites []
Jun 30, 2016 10:26:50 AM org.elasticsearch.env.NodeEnvironment maybeLogPathDetails
INFO: [D'Ken] using [1] data paths, mounts [[/ (/dev/mapper/main-root)]], net usable_space [3.6gb], net total_space [31.3gb], types [ext4]
Jun 30, 2016 10:26:51 AM org.elasticsearch.bootstrap.Bootstrap main
SEVERE: Exception
com.google.common.util.concurrent.ExecutionError: com.google.common.util.concurrent.ExecutionError: java.lang.NoClassDefFoundError: Lcom/carrotsearch/hppc/ByteObjectOpenHashMap;
	at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2201)
	at com.google.common.cache.LocalCache.get(LocalCache.java:3937)
	at com.google.common.cache.LocalCache.getOrLoad(LocalCache.java:3941)
	at com.google.common.cache.LocalCache$LocalLoadingCache.get(LocalCache.java:4824)
	at org.elasticsearch.common.inject.internal.FailableCache.get(FailableCache.java:51)
	at org.elasticsearch.common.inject.ConstructorInjectorStore.get(ConstructorInjectorStore.java:50)
	at org.elasticsearch.common.inject.ConstructorBindingImpl.initialize(ConstructorBindingImpl.java:50)
	at org.elasticsearch.common.inject.InjectorImpl.initializeBinding(InjectorImpl.java:372)
	at org.elasticsearch.common.inject.BindingProcessor$1$1.run(BindingProcessor.java:148)
	at org.elasticsearch.common.inject.BindingProcessor.initializeBindings(BindingProcessor.java:204)
	at org.elasticsearch.common.inject.InjectorBuilder.initializeStatically(InjectorBuilder.java:119)
	at org.elasticsearch.common.inject.InjectorBuilder.build(InjectorBuilder.java:102)
	at org.elasticsearch.common.inject.Guice.createInjector(Guice.java:96)
	at org.elasticsearch.common.inject.Guice.createInjector(Guice.java:70)
	at org.elasticsearch.common.inject.ModulesBuilder.createInjector(ModulesBuilder.java:59)
	at org.elasticsearch.node.internal.InternalNode.<init>(InternalNode.java:210)
	at org.elasticsearch.node.NodeBuilder.build(NodeBuilder.java:159)
	at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:77)
	at org.elasticsearch.bootstrap.Bootstrap.main(Bootstrap.java:245)
	at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:32)
Caused by: com.google.common.util.concurrent.ExecutionError: java.lang.NoClassDefFoundError: Lcom/carrotsearch/hppc/ByteObjectOpenHashMap;
	at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2201)
	at com.google.common.cache.LocalCache.get(LocalCache.java:3937)
	at com.google.common.cache.LocalCache.getOrLoad(LocalCache.java:3941)
	at com.google.common.cache.LocalCache$LocalLoadingCache.get(LocalCache.java:4824)
	at org.elasticsearch.common.inject.internal.FailableCache.get(FailableCache.java:51)
	at org.elasticsearch.common.inject.MembersInjectorStore.get(MembersInjectorStore.java:68)
	at org.elasticsearch.common.inject.ConstructorInjectorStore.createConstructor(ConstructorInjectorStore.java:67)
	at org.elasticsearch.common.inject.ConstructorInjectorStore.access$000(ConstructorInjectorStore.java:29)
	at org.elasticsearch.common.inject.ConstructorInjectorStore$1.create(ConstructorInjectorStore.java:37)
	at org.elasticsearch.common.inject.ConstructorInjectorStore$1.create(ConstructorInjectorStore.java:33)
	at org.elasticsearch.common.inject.internal.FailableCache$1.load(FailableCache.java:39)
	at com.google.common.cache.LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3527)
	at com.google.common.cache.LocalCache$Segment.loadSync(LocalCache.java:2319)
	at com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2282)
	at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2197)
	... 19 more
Caused by: java.lang.NoClassDefFoundError: Lcom/carrotsearch/hppc/ByteObjectOpenHashMap;
	at java.lang.Class.getDeclaredFields0(Native Method)
	at java.lang.Class.privateGetDeclaredFields(Class.java:2583)
	at java.lang.Class.getDeclaredFields(Class.java:1916)
	at org.elasticsearch.common.inject.spi.InjectionPoint$Factory$1.getMembers(InjectionPoint.java:378)
	at org.elasticsearch.common.inject.spi.InjectionPoint$Factory$1.getMembers(InjectionPoint.java:376)
	at org.elasticsearch.common.inject.spi.InjectionPoint.addInjectorsForMembers(InjectionPoint.java:351)
	at org.elasticsearch.common.inject.spi.InjectionPoint.addInjectionPoints(InjectionPoint.java:345)
	at org.elasticsearch.common.inject.spi.InjectionPoint.forInstanceMethodsAndFields(InjectionPoint.java:287)
	at org.elasticsearch.common.inject.MembersInjectorStore.createWithListeners(MembersInjectorStore.java:80)
	at org.elasticsearch.common.inject.MembersInjectorStore.access$000(MembersInjectorStore.java:36)
	at org.elasticsearch.common.inject.MembersInjectorStore$1.create(MembersInjectorStore.java:45)
	at org.elasticsearch.common.inject.MembersInjectorStore$1.create(MembersInjectorStore.java:41)
	at org.elasticsearch.common.inject.internal.FailableCache$1.load(FailableCache.java:39)
	at com.google.common.cache.LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3527)
	at com.google.common.cache.LocalCache$Segment.loadSync(LocalCache.java:2319)
	at com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2282)
	at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2197)
	... 33 more
Caused by: java.lang.ClassNotFoundException: com.carrotsearch.hppc.ByteObjectOpenHashMap
	at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
	at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
	... 50 more

Comment 1 Viacheslav Viniarskyi 2016-07-20 08:19:34 UTC
Had same error. You should try using hppc < 0.7.0, there are API-breaking changes in this version, and that class was renamed. https://github.com/carrotsearch/hppc/blob/0.7.1/CHANGES.txt with hppc-0.6.1-3.fc23.noarch elasticsearch start as usual.

Comment 2 gil cattaneo 2016-07-20 08:22:43 UTC
(In reply to Viacheslav Viniarskyi from comment #1)
> Had same error. You should try using hppc < 0.7.0, there are API-breaking
> changes in this version, and that class was renamed.
> https://github.com/carrotsearch/hppc/blob/0.7.1/CHANGES.txt with
> hppc-0.6.1-3.fc23.noarch elasticsearch start as usual.

Someone already fix this problem. ask in fedora-java irc channel

Comment 3 Sam P. 2016-07-22 21:35:55 UTC
I can reproduce this issue on a clean Fedora 24 installation with the following package versions:

    elasticsearch-1.7.1-3.fc24.noarch
    hppc-0.7.1-3.fc24.noarch

What is this magical solution from the irc channel that is not recorded?

Comment 4 Richard Bradfield 2016-07-26 14:21:34 UTC
I'm seeing the same behaviour on a clean install of F24 (with the same versions as Sam P.)

Manually downgrading to hppc-0.6.1-3.fc23 allows Elasticsearch to start.

When can we expect this IRC Fix to be published?

Comment 5 Sudheera Satyanarayana 2016-08-11 13:52:58 UTC
If you're looking for the commands to downgrade hppc

dnf remove hppc
dnf install https://archive.fedoraproject.org/pub/fedora/linux/releases/23/Everything/i386/os/Packages/h/hppc-0.6.1-3.fc23.noarch.rpm
systemctl start elasticsearch

Comment 6 Steven Webb 2016-09-08 06:06:31 UTC
If you don't want to remove hppc and all of its dependencies you can simply download the package linked by Sudheera Satyanarayana and install it by clicking on it.

Comment 7 Gonéri Le Bouder 2017-02-02 20:20:48 UTC
*** Bug 1418751 has been marked as a duplicate of this bug. ***

Comment 8 Miki Shapiro 2017-02-11 01:28:00 UTC
Encountering this too, though the FC24 resolution proposed here, of downgrading hppc, doesn't seem to fix the issue on FC25 (and therefore maybe bug 1418751 needs to be re-opened?)

[root@ip-172-31-43-40 ~]# systemctl start elasticsearch
[root@ip-172-31-43-40 ~]# systemctl status elasticsearch
● elasticsearch.service - ElasticSearch search engine
   Loaded: loaded (/usr/lib/systemd/system/elasticsearch.service; disabled; vendor preset: disabled)
   Active: failed (Result: exit-code) since Sat 2017-02-11 01:25:06 UTC; 16s ago
     Docs: https://www.elasticsearch.org/guide/en/elasticsearch/guide/current/index.html
  Process: 7908 ExecStart=/usr/libexec/elasticsearch (code=exited, status=1/FAILURE)
 Main PID: 7908 (code=exited, status=1/FAILURE)

Feb 11 01:25:06 ip-172-31-43-40.ec2.internal elasticsearch[7908]:         at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:32)
Feb 11 01:25:06 ip-172-31-43-40.ec2.internal elasticsearch[7908]: Caused by: java.lang.ClassNotFoundException: com.fasterxml.jackson.databind.JsonMappingException
Feb 11 01:25:06 ip-172-31-43-40.ec2.internal elasticsearch[7908]:         at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
Feb 11 01:25:06 ip-172-31-43-40.ec2.internal elasticsearch[7908]:         at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
Feb 11 01:25:06 ip-172-31-43-40.ec2.internal elasticsearch[7908]:         at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331)
Feb 11 01:25:06 ip-172-31-43-40.ec2.internal elasticsearch[7908]:         at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
Feb 11 01:25:06 ip-172-31-43-40.ec2.internal elasticsearch[7908]:         ... 11 more
Feb 11 01:25:06 ip-172-31-43-40.ec2.internal systemd[1]: elasticsearch.service: Main process exited, code=exited, status=1/FAILURE
Feb 11 01:25:06 ip-172-31-43-40.ec2.internal systemd[1]: elasticsearch.service: Unit entered failed state.
Feb 11 01:25:06 ip-172-31-43-40.ec2.internal systemd[1]: elasticsearch.service: Failed with result 'exit-code'.
[root@ip-172-31-43-40 ~]# rpm -q hppc elasticsearch
hppc-0.6.1-3.fc23.noarch
elasticsearch-1.7.1-3.fc24.noarch
[root@ip-172-31-43-40 ~]# cat /etc/fedora-release 
Fedora release 25 (Twenty Five)

Comment 9 Steven Webb 2017-03-15 07:37:23 UTC
I can confirm what Miki said. The hppc solution doesn't work with Fedora 25.

Comment 10 Glen Stark 2017-04-06 12:59:36 UTC
I can verify Steven Webb's confirmation that downgrading hppc does not work with Fedora 25.  I worked around the problem by installing elasticsearch from the elasticsearch repository, as detailed here:  https://www.elastic.co/guide/en/elasticsearch/reference/current/rpm.html

Comment 11 ojab 2017-06-21 07:45:23 UTC
Still broken :/

Comment 12 ojab 2017-06-21 08:26:22 UTC
Hmm, http://pkgs.fedoraproject.org/cgit/rpms/elasticsearch.git/ apparently shows that there was updates for fc25 and fc26, but https://apps.fedoraproject.org/packages/elasticsearch shows 1.7.1-3.fc24 and indeed dnf tries to install 1.7.1-3.fc24 on FC25.
So the question is why it's not updated in repos. AFAIU hubbitus is the pkg owner and he's in CC here.

Comment 13 Fedora End Of Life 2017-07-25 21:27:07 UTC
This message is a reminder that Fedora 24 is nearing its end of life.
Approximately 2 (two) weeks from now Fedora will stop maintaining
and issuing updates for Fedora 24. It is Fedora's policy to close all
bug reports from releases that are no longer maintained. At that time
this bug will be closed as EOL if it remains open with a Fedora  'version'
of '24'.

Package Maintainer: If you wish for this bug to remain open because you
plan to fix it in a currently maintained version, simply change the 'version'
to a later Fedora version.

Thank you for reporting this issue and we are sorry that we were not
able to fix it before Fedora 24 is end of life. If you would still like
to see this bug fixed and are able to reproduce it against a later version
of Fedora, you are encouraged  change the 'version' to a later Fedora
version prior this bug is closed as described in the policy above.

Although we aim to fix as many bugs as possible during every release's
lifetime, sometimes those efforts are overtaken by events. Often a
more recent Fedora release includes newer upstream software that fixes
bugs or makes them obsolete.

Comment 14 Fedora End Of Life 2017-08-08 15:11:42 UTC
Fedora 24 changed to end-of-life (EOL) status on 2017-08-08. Fedora 24 is
no longer maintained, which means that it will not receive any further
security or bug fix updates. As a result we are closing this bug.

If you can reproduce this bug against a currently maintained version of
Fedora please feel free to reopen this bug against that version. If you
are unable to reopen this bug, please file a new report against the
current release. If you experience problems, please add a comment to this
bug.

Thank you for reporting this bug and we are sorry it could not be fixed.

Comment 15 peter ducai 2017-09-21 12:39:41 UTC
same issue for F26:

[root@localhost ~]# systemctl status elasticsearch
● elasticsearch.service - ElasticSearch search engine
   Loaded: loaded (/usr/lib/systemd/system/elasticsearch.service; enabled; vendor preset: disabled)
   Active: failed (Result: exit-code) since Thu 2017-09-21 14:27:37 CEST; 3min 48s ago
     Docs: https://www.elasticsearch.org/guide/en/elasticsearch/guide/current/index.html
  Process: 657 ExecStart=/usr/libexec/elasticsearch (code=exited, status=1/FAILURE)
 Main PID: 657 (code=exited, status=1/FAILURE)

Sep 21 14:27:37 localhost.localdomain elasticsearch[657]:         at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:32)
Sep 21 14:27:37 localhost.localdomain elasticsearch[657]: Caused by: java.lang.ClassNotFoundException: com.fasterxml.jackson.databind.JsonMappingException
Sep 21 14:27:37 localhost.localdomain elasticsearch[657]:         at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
Sep 21 14:27:37 localhost.localdomain elasticsearch[657]:         at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
Sep 21 14:27:37 localhost.localdomain elasticsearch[657]:         at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:335)
Sep 21 14:27:37 localhost.localdomain elasticsearch[657]:         at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
Sep 21 14:27:37 localhost.localdomain elasticsearch[657]:         ... 11 more
Sep 21 14:27:37 localhost.localdomain systemd[1]: elasticsearch.service: Main process exited, code=exited, status=1/FAILURE
Sep 21 14:27:37 localhost.localdomain systemd[1]: elasticsearch.service: Unit entered failed state.
Sep 21 14:27:37 localhost.localdomain systemd[1]: elasticsearch.service: Failed with result 'exit-code'.



Linux localhost.localdomain 4.12.13-300.fc26.x86_64 #1 SMP Thu Sep 14 16:00:38 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux
Fedora release 26 (Twenty Six)

Comment 16 peter ducai 2017-09-21 12:40:03 UTC
same issue for F26:

[root@localhost ~]# systemctl status elasticsearch
● elasticsearch.service - ElasticSearch search engine
   Loaded: loaded (/usr/lib/systemd/system/elasticsearch.service; enabled; vendor preset: disabled)
   Active: failed (Result: exit-code) since Thu 2017-09-21 14:27:37 CEST; 3min 48s ago
     Docs: https://www.elasticsearch.org/guide/en/elasticsearch/guide/current/index.html
  Process: 657 ExecStart=/usr/libexec/elasticsearch (code=exited, status=1/FAILURE)
 Main PID: 657 (code=exited, status=1/FAILURE)

Sep 21 14:27:37 localhost.localdomain elasticsearch[657]:         at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:32)
Sep 21 14:27:37 localhost.localdomain elasticsearch[657]: Caused by: java.lang.ClassNotFoundException: com.fasterxml.jackson.databind.JsonMappingException
Sep 21 14:27:37 localhost.localdomain elasticsearch[657]:         at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
Sep 21 14:27:37 localhost.localdomain elasticsearch[657]:         at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
Sep 21 14:27:37 localhost.localdomain elasticsearch[657]:         at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:335)
Sep 21 14:27:37 localhost.localdomain elasticsearch[657]:         at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
Sep 21 14:27:37 localhost.localdomain elasticsearch[657]:         ... 11 more
Sep 21 14:27:37 localhost.localdomain systemd[1]: elasticsearch.service: Main process exited, code=exited, status=1/FAILURE
Sep 21 14:27:37 localhost.localdomain systemd[1]: elasticsearch.service: Unit entered failed state.
Sep 21 14:27:37 localhost.localdomain systemd[1]: elasticsearch.service: Failed with result 'exit-code'.



Linux localhost.localdomain 4.12.13-300.fc26.x86_64 #1 SMP Thu Sep 14 16:00:38 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux
Fedora release 26 (Twenty Six)

Comment 17 peter ducai 2017-09-21 12:41:18 UTC
[root@localhost ~]# dnf info elasticsearch
Last metadata expiration check: 1:52:57 ago on Thu 21 Sep 2017 12:47:54 PM CEST.
Installed Packages
Name         : elasticsearch
Version      : 1.7.1
Release      : 3.fc24
Arch         : noarch
Size         : 9.2 M
Source       : elasticsearch-1.7.1-3.fc24.src.rpm

Comment 18 Lukas Berk 2018-07-17 15:10:54 UTC
Still hitting this bug, elasticsearch is broken, and requires more manual effort to fix the classpath loading.

Comment 19 Ben Cotton 2019-05-02 20:48:39 UTC
This message is a reminder that Fedora 28 is nearing its end of life.
On 2019-May-28 Fedora will stop maintaining and issuing updates for
Fedora 28. It is Fedora's policy to close all bug reports from releases
that are no longer maintained. At that time this bug will be closed as
EOL if it remains open with a Fedora 'version' of '28'.

Package Maintainer: If you wish for this bug to remain open because you
plan to fix it in a currently maintained version, simply change the 'version' 
to a later Fedora version.

Thank you for reporting this issue and we are sorry that we were not 
able to fix it before Fedora 28 is end of life. If you would still like 
to see this bug fixed and are able to reproduce it against a later version 
of Fedora, you are encouraged  change the 'version' to a later Fedora 
version prior this bug is closed as described in the policy above.

Although we aim to fix as many bugs as possible during every release's 
lifetime, sometimes those efforts are overtaken by events. Often a 
more recent Fedora release includes newer upstream software that fixes 
bugs or makes them obsolete.

Comment 20 Ben Cotton 2019-05-28 22:38:08 UTC
Fedora 28 changed to end-of-life (EOL) status on 2019-05-28. Fedora 28 is
no longer maintained, which means that it will not receive any further
security or bug fix updates. As a result we are closing this bug.

If you can reproduce this bug against a currently maintained version of
Fedora please feel free to reopen this bug against that version. If you
are unable to reopen this bug, please file a new report against the
current release. If you experience problems, please add a comment to this
bug.

Thank you for reporting this bug and we are sorry it could not be fixed.


Note You need to log in before you can comment on or make changes to this bug.