Bug 1240495

upstream openstack CI is occasionally [1] failing; e.g.

---
2015-07-04 15:20:33.078 | + sudo /sbin/service httpd start
2015-07-04 15:20:33.095 | Redirecting to /bin/systemctl start  httpd.service
2015-07-04 15:20:33.196 | Job for httpd.service failed. See 'systemctl status httpd.service' and 'journalctl -xn' for details.
2015-07-04 15:20:33.197 | + exit_trap
---

---
Jul 04 15:20:33 devstack-... systemd[1]: Starting The Apache HTTP Server...
Jul 04 15:20:33 devstack-... httpd[6407]: AH00526: Syntax error on line 100 of /etc/httpd/conf.d/ssl.conf:
Jul 04 15:20:33 devstack-... httpd[6407]: SSLCertificateFile: file '/etc/pki/tls/certs/localhost.crt' does not exist or is empty
Jul 04 15:20:33 devstack-... systemd[1]: httpd.service: main process exited, code=exited, status=1/FAILURE
---

However when we trace backwards, we see

---
2015-07-04 15:07:24.037 |   Installing : 1:mod_ssl-2.4.6-31.el7.centos.x86_64                       56/77 
2015-07-04 15:07:24.037 | warning: %post(mod_ssl-1:2.4.6-31.el7.centos.x86_64) scriptlet failed, exit status 1
2015-07-04 15:07:24.039 | Non-fatal POSTIN scriptlet failure in rpm package 1:mod_ssl-2.4.6-31.el7.centos.x86_64
---

The mod_ssl post script [1] doesn't do a lot -- it generates /etc/pki/tls/private/localhost.key and then /etc/pki/tls/certs/localhost.crt

Really, the only place this could cause the script to exit with "1" is if the final call to "openssl req" fails.

The way it is written, the error output of "openssl gen" gets sent to /dev/null so we don't get a smoking gun.  But after a bunch of testing I believe the issue is with the length of the hostname
---
# hostname devstack-centos7-1435932846.template.openstack.org.novalocal
# rm /etc/pki/tls/certs/localhost.crt /etc/pki/tls/private/localhost.key 
# yum remove -y mod_ssl

# yum install -y mod_ssl
...
Resolving Dependencies
--> Running transaction check
---> Package mod_ssl.x86_64 1:2.4.6-31.el7.centos will be installed
Running transaction
  Installing : 1:mod_ssl-2.4.6-31.el7.centos.x86_64
warning: %post(mod_ssl-1:2.4.6-31.el7.centos.x86_64) scriptlet failed, exit status 1
Non-fatal POSTIN scriptlet failure in rpm package 1:mod_ssl-2.4.6-31.el7.centos.x86_64
---

(if you run it on the command-line by hand, it complains the hostname is too long)

I don't know what the right thing to do is : pki/tls/openssl.cnf sets commonName_max to 64 characters so that's where the limit comes from; chopping it off seems easy but maybe wrong, and failing silently in this way is definitely wrong -- any failure output would have helped track this down a lot faster.

[1] search for "Job for httpd.service failed" at logstash.openstack.org
[2] http://logs.openstack.org/96/198496/1/check/check-tempest-dsvm-centos7/5a0897e/logs/devstacklog.txt.gz
[3] http://pkgs.fedoraproject.org/cgit/httpd.git/tree/httpd.spec#n501