*** Bug 996121 has been marked as a duplicate of this bug. ***

Startup script is here, use this as a reference to see how thumbslug is being shutdown to determine WHICH warn needs to be fixed in Main.java of thumbslug.

https://github.com/Katello/katello/blob/master/script/katello-service?source=cc

I think thumbslug is failing to load it's keystore (/etc/thumbslug/server_keystore) and fails to start and logs the "Shutting down..." warning on the way out.

I can reproduce the WARN message on a system without a server_keystore.

==> /var/log/thumbslug/error.log <==
Aug 22 10:55:18 [main] ERROR org.candlepin.thumbslug.Main - Unable to load the ssl keystore. Check that ssl.keystore and ssl.keystore.password are set correctly.
org.candlepin.thumbslug.ssl.SslKeystoreException: Failed to initialize the server-side SSLContext.
	at org.candlepin.thumbslug.ssl.SslContextFactory.getServerContext(SslContextFactory.java:83)
	at org.candlepin.thumbslug.Main.configureSSL(Main.java:67)
	at org.candlepin.thumbslug.Main.main(Main.java:139)
Caused by: java.io.FileNotFoundException: /etc/thumbslug/server_keystore.p12 (No such file or directory)
	at java.io.FileInputStream.open(Native Method)
	at java.io.FileInputStream.<init>(FileInputStream.java:137)
	at org.candlepin.thumbslug.ssl.SslContextFactory.getServerContext(SslContextFactory.java:68)
	... 2 more
Aug 22 10:55:44 [main] WARN  org.candlepin.thumbslug.Main - Shutting down...
Aug 22 10:55:45 [main] INFO  org.candlepin.thumbslug.ssl.SslContextFactory - reading keystore
Aug 22 10:55:45 [main] ERROR org.candlepin.thumbslug.Main - Unable to load the ssl keystore. Check that ssl.keystore and ssl.keystore.password are set correctly.
org.candlepin.thumbslug.ssl.SslKeystoreException: Failed to initialize the server-side SSLContext.
	at org.candlepin.thumbslug.ssl.SslContextFactory.getServerContext(SslContextFactory.java:83)
	at org.candlepin.thumbslug.Main.configureSSL(Main.java:67)
	at org.candlepin.thumbslug.Main.main(Main.java:139)
Caused by: java.io.FileNotFoundException: /etc/thumbslug/server_keystore.p12 (No such file or directory)
	at java.io.FileInputStream.open(Native Method)
	at java.io.FileInputStream.<init>(FileInputStream.java:137)
	at org.candlepin.thumbslug.ssl.SslContextFactory.getServerContext(SslContextFactory.java:68)
	... 2 more


Presumably missing or incorrect keystore password would be similar.

As best I can tell, nothing in thumbslug actually creates this file.

(In reply to Adrian Likins from comment #4)
> As best I can tell, nothing in thumbslug actually creates this file.

That's correct, thumbslug proper does not. It's created by the katello-installer.

https://github.com/Katello/katello-installer/tree/master/modules/thumbslug

So it sounds like this error only happens if thumbslug is started without being setup first? and katello-installer will set that up?

The WARN is from thumbslug startup, where the default logging is:

log4j.appender.RootAppender=org.apache.log4j.ConsoleAppender
log4j.appender.RootAppender.layout=org.apache.log4j.PatternLayout
log4j.appender.RootAppender.layout.ConversionPattern=%d{MMM dd HH:mm:ss} [%t] %-5p %c - %m%n
log4j.rootLogger=WARN,RootAppender
log4j.logger.org.candlepin.thumbslug=INFO

until configs are loaded anyway, so I think that WARN is intentional and valid in this scenario (though it could use more detail)

I added qualifiers to the 3 print messages in Main.java. The message referenced in this bug is from line 153 of Main.java:

        else {
            if (shouldDaemonize) {
                try {
                    log.debug("Daemonizing..");
                    daemon.daemonize();
                    log.debug("Daemonized, exiting");
                }
                catch (IOException e) {
                    log.error("Unable to daemonize properly", e);
                    System.exit(ERROR_DAEMON_DAEMONIZE);
                }
                log.warn("Daemonized, I guess we're done. Shutting down...");
                System.exit(0);
            }
        }

I added the prefix so I could see which statement it was. After starting up thumbslug I was able to see this:

Starting thumbslug: Sep 05 14:06:15 [main] WARN  org.candlepin.thumbslug.Main - Daemonized, I guess we're done. Shutting down...

Removed offending Shutting down log statement.

Added text to each of the remaining Shutting down log entries which result from
an error during startup. The only one that print Shutting down... alone is the
shutdown hook which is the normal shutdown mechanism for thumbslug.

Configuration error results int he following:
    "Unable to load config! Shutting down..."

Problem trying to daemonize:
    "Error during daemon initialization! Shutting down..."

Valid shutdown message:

    "Shutting down..."

https://github.com/candlepin/thumbslug/pull/28

Failed.

# katello-service restart
Shutting down Katello services...
Stopping katello: 
Stopping elasticsearch: [  OK  ]
Stopping thumbslug: [  OK  ]
Stopping httpd: [  OK  ]
Stopping tomcat6: [  OK  ]
Done.
Starting Katello services...
Starting tomcat6: [  OK  ]
Starting httpd: [  OK  ]
Starting thumbslug: Sep 13 11:53:19 [main] WARN  org.candlepin.thumbslug.Main - Shutting down...
[  OK  ]
Starting elasticsearch: [  OK  ]
Starting katello: [  OK  ]
Starting katello-jobs: [  OK  ]
Done.


Version Tested:
* candlepin-0.8.26-1.el6sam.noarch
* candlepin-scl-1-5.el6_4.noarch
* candlepin-scl-quartz-2.1.5-5.el6_4.noarch
* candlepin-scl-rhino-1.7R3-1.el6_4.noarch
* candlepin-scl-runtime-1-5.el6_4.noarch
* candlepin-selinux-0.8.26-1.el6sam.noarch
* candlepin-tomcat6-0.8.26-1.el6sam.noarch
* elasticsearch-0.19.9-8.el6sat.noarch
* katello-candlepin-cert-key-pair-1.0-1.noarch
* katello-certs-tools-1.4.2-2.el6sat.noarch
* katello-cli-1.4.3-10.el6sat.noarch
* katello-cli-common-1.4.3-10.el6sat.noarch
* katello-common-1.4.3-13.el6sam_splice.noarch
* katello-configure-1.4.4-4.el6sat.noarch
* katello-glue-candlepin-1.4.3-13.el6sam_splice.noarch
* katello-glue-elasticsearch-1.4.3-13.el6sam_splice.noarch
* katello-headpin-1.4.3-13.el6sam_splice.noarch
* katello-headpin-all-1.4.3-13.el6sam_splice.noarch
* katello-selinux-1.4.4-2.el6sat.noarch
* thumbslug-0.0.34-1.el6sam.noarch
* thumbslug-selinux-0.0.34-1.el6sam.noarch

Prematurely moved to ON_QA. The fix is in thumbslug-0.0.35-1.

The release of Satellite 5.8 we are deprecating the support of Subscription Asset Manager. The release notes for 5.8 can be found at https://access.redhat.com/documentation/en-us/red_hat_satellite/5.8/pdf/release_notes/Red_Hat_Satellite-5.8-Release_Notes-en-US.pdf.

I am therefore closing out this bug as WONTFIX. If you believe this to be an error, please feel free tor each out to either Rich Jerrido or Bryan Kearney. Thank you!