Bug 1012001
| Summary: | all openstack services: unable to connect to AMQP server (after upgrading to qpid 0.24) | ||||||
|---|---|---|---|---|---|---|---|
| Product: | [Fedora] Fedora | Reporter: | Matthias Runge <mrunge> | ||||
| Component: | qpid-cpp | Assignee: | Darryl L. Pierce <dpierce> | ||||
| Status: | CLOSED ERRATA | QA Contact: | Fedora Extras Quality Assurance <extras-qa> | ||||
| Severity: | unspecified | Docs Contact: | |||||
| Priority: | unspecified | ||||||
| Version: | 19 | CC: | bderzhavets, derekh, dpierce, itamar, Jan.van.Eldik, jose.p.oliveira.oss, mmagr, nsantos, pbrady, p, tross, zhenqin | ||||
| Target Milestone: | --- | Keywords: | Reopened | ||||
| Target Release: | --- | ||||||
| Hardware: | Unspecified | ||||||
| OS: | Unspecified | ||||||
| Whiteboard: | |||||||
| Fixed In Version: | qpid-cpp-0.24-4.fc20.1 | Doc Type: | Bug Fix | ||||
| Doc Text: | Story Points: | --- | |||||
| Clone Of: | |||||||
| : | 1012814 (view as bug list) | Environment: | |||||
| Last Closed: | 2013-10-09 14:44:53 UTC | Type: | Bug | ||||
| Regression: | --- | Mount Type: | --- | ||||
| Documentation: | --- | CRM: | |||||
| Verified Versions: | Category: | --- | |||||
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |||||
| Cloudforms Team: | --- | Target Upstream Version: | |||||
| Embargoed: | |||||||
| Bug Depends On: | |||||||
| Bug Blocks: | 1012814 | ||||||
| Attachments: |
|
||||||
|
Description
Matthias Runge
2013-09-25 14:20:48 UTC
This is where the Qpid configuration files will live going forward, in /etc/qpid. Darryl, there is a compatibility cost of course with moving configs around. Have you considered that and a possible transition period/process to support for example /etc/qpidd.conf as a symlink into /etc/qpid/ or something? (In reply to Pádraig Brady from comment #2) > Darryl, there is a compatibility cost of course with moving configs around. > Have you considered that and a possible transition period/process to support > for example /etc/qpidd.conf as a symlink into /etc/qpid/ or something? No, that wasn't considered. Having a symlink from /etc/qpid/qpidd.conf to /etc/qpidd.conf wouldn't work since qpidd will look for the configuration in /etc/qpidd. I discussed this with other package maintainers. The general consensus was that, since the upgrade process would leave the file /etc/qpidd.conf.rpmsave behind after an upgrade, along with the release notes for the server, the sysadmin would take the appropriate steps to migrate their existing configuration to the new file location. (In reply to Darryl L. Pierce from comment #3) > (In reply to Pádraig Brady from comment #2) > > Darryl, there is a compatibility cost of course with moving configs around. > > Have you considered that and a possible transition period/process to support > > for example /etc/qpidd.conf as a symlink into /etc/qpid/ or something? > > No, that wasn't considered. Having a symlink from /etc/qpid/qpidd.conf to > /etc/qpidd.conf wouldn't work since qpidd will look for the configuration in > /etc/qpidd. What I mean here is that qpid could use the new /etc/qpid/qpidd.conf while other tools like puppet modules etc. could continue to use /etc/qpidd.conf (symlink) until updated to the new location. > I discussed this with other package maintainers. The general consensus was > that, since the upgrade process would leave the file /etc/qpidd.conf.rpmsave > behind after an upgrade, along with the release notes for the server, the > sysadmin would take the appropriate steps to migrate their existing > configuration to the new file location. I don't think this is acceptable during a release TBH. I.E. the system shouldn't break on (auto) update. IMHO this fits here: http://fedoraproject.org/wiki/Updates_Policy#All_other_updates I even disagree with the request to support both config options (/etc/qpidd.conf) and /etc/qpid/qpidd.conf), because qpid brings it's new, own config to /etc/qpid/qpidd.conf). Given some script like puppet places a new qpid.conf in /etc, which config should have precedence? I see arguments in both directions. So this upgrade in a released version is currently more than just unfortunate. (In reply to Matthias Runge from comment #5) > IMHO this fits here: > http://fedoraproject.org/wiki/Updates_Policy#All_other_updates > > I even disagree with the request to support both config options > (/etc/qpidd.conf) and /etc/qpid/qpidd.conf), because qpid brings it's new, > own config to /etc/qpid/qpidd.conf). Given some script like puppet places a > new qpid.conf in /etc, which config should have precedence? I see arguments > in both directions. > So this upgrade in a released version is currently more than just > unfortunate. I'm willing, for the 0.24 release, to modify the spec to create a symlink from /etc/qpid/qpidd.conf to /etc/qpid.conf if that will smooth the transition. However, anybody who has modified their configuration will still need to migrate their configuration to the new file. Will that provide a sufficient transition? qpid-cpp-0.24-4.fc20.1 has been submitted as an update for Fedora 20. https://admin.fedoraproject.org/updates/qpid-cpp-0.24-4.fc20.1 qpid-cpp-0.24-3.fc19.1 has been submitted as an update for Fedora 19. https://admin.fedoraproject.org/updates/qpid-cpp-0.24-3.fc19.1 qpid-cpp-0.24-3.fc18.1 has been submitted as an update for Fedora 18. https://admin.fedoraproject.org/updates/qpid-cpp-0.24-3.fc18.1 This won't help in case of puppet-qpid module, but I guess we would have to fix that in the module. Package qpid-cpp-0.24-4.fc20.1: * should fix your issue, * was pushed to the Fedora 20 testing repository, * should be available at your local mirror within two days. Update it with: # su -c 'yum update --enablerepo=updates-testing qpid-cpp-0.24-4.fc20.1' as soon as you are able to. Please go to the following url: https://admin.fedoraproject.org/updates/FEDORA-2013-17794/qpid-cpp-0.24-4.fc20.1 then log in and leave karma (feedback). qpid-cpp-0.24-3.fc19.1 has been pushed to the Fedora 19 stable repository. If problems still persist, please make note of it in this bug report. qpid-cpp-0.24-3.fc18.1 has been pushed to the Fedora 18 stable repository. If problems still persist, please make note of it in this bug report. > Package qpid-cpp-0.24-4.fc20.1 * should fix your issue No it doesn't [boris@localhost ~]$ rpm -iv qpid-cpp-0.24-4.fc21.1.src.rpm [boris@localhost ~]$ rpmbuild -bb qpid-cpp.spec [boris@localhost ~]$ cat rpmbuild/RPMS/x*/inst.sh sudo yum install python-qpid-qmf-0.24-4.fc19.1.x86_64.rpm \ qpid-cpp-client-0.24-4.fc19.1.x86_64.rpm \ qpid-cpp-client-devel-0.24-4.fc19.1.x86_64.rpm \ qpid-cpp-client-rdma-0.24-4.fc19.1.x86_64.rpm \ qpid-cpp-debuginfo-0.24-4.fc19.1.x86_64.rpm \ qpid-cpp-server-0.24-4.fc19.1.x86_64.rpm \ qpid-cpp-server-ha-0.24-4.fc19.1.x86_64.rpm \ qpid-cpp-server-rdma-0.24-4.fc19.1.x86_64.rpm \ qpid-cpp-server-store-0.24-4.fc19.1.x86_64.rpm \ qpid-cpp-server-xml-0.24-4.fc19.1.x86_64.rpm \ qpid-qmf-0.24-4.fc19.1.x86_64.rpm \ qpid-qmf-devel-0.24-4.fc19.1.x86_64.rpm \ ruby-qpid-qmf-0.24-4.fc19.1.x86_64.rpm [boris@localhost x86_64]$ ./inst.sh [sudo] password for boris: Loaded plugins: langpacks, refresh-packagekit Examining python-qpid-qmf-0.24-4.fc19.1.x86_64.rpm: python-qpid-qmf-0.24-4.fc19.1.x86_64 Marking python-qpid-qmf-0.24-4.fc19.1.x86_64.rpm to be installed Examining qpid-cpp-client-0.24-4.fc19.1.x86_64.rpm: qpid-cpp-client-0.24-4.fc19.1.x86_64 Marking qpid-cpp-client-0.24-4.fc19.1.x86_64.rpm as an update to qpid-cpp-client-0.24-3.fc19.1.x86_64 Examining qpid-cpp-client-devel-0.24-4.fc19.1.x86_64.rpm: qpid-cpp-client-devel-0.24-4.fc19.1.x86_64 Marking qpid-cpp-client-devel-0.24-4.fc19.1.x86_64.rpm to be installed Examining qpid-cpp-client-rdma-0.24-4.fc19.1.x86_64.rpm: qpid-cpp-client-rdma-0.24-4.fc19.1.x86_64 Marking qpid-cpp-client-rdma-0.24-4.fc19.1.x86_64.rpm as an update to qpid-cpp-client-rdma-0.24-3.fc19.1.x86_64 Examining qpid-cpp-debuginfo-0.24-4.fc19.1.x86_64.rpm: qpid-cpp-debuginfo-0.24-4.fc19.1.x86_64 Marking qpid-cpp-debuginfo-0.24-4.fc19.1.x86_64.rpm to be installed Examining qpid-cpp-server-0.24-4.fc19.1.x86_64.rpm: qpid-cpp-server-0.24-4.fc19.1.x86_64 Marking qpid-cpp-server-0.24-4.fc19.1.x86_64.rpm as an update to qpid-cpp-server-0.24-3.fc19.1.x86_64 Examining qpid-cpp-server-ha-0.24-4.fc19.1.x86_64.rpm: qpid-cpp-server-ha-0.24-4.fc19.1.x86_64 Marking qpid-cpp-server-ha-0.24-4.fc19.1.x86_64.rpm to be installed Examining qpid-cpp-server-rdma-0.24-4.fc19.1.x86_64.rpm: qpid-cpp-server-rdma-0.24-4.fc19.1.x86_64 Marking qpid-cpp-server-rdma-0.24-4.fc19.1.x86_64.rpm to be installed Examining qpid-cpp-server-store-0.24-4.fc19.1.x86_64.rpm: qpid-cpp-server-store-0.24-4.fc19.1.x86_64 Marking qpid-cpp-server-store-0.24-4.fc19.1.x86_64.rpm to be installed Examining qpid-cpp-server-xml-0.24-4.fc19.1.x86_64.rpm: qpid-cpp-server-xml-0.24-4.fc19.1.x86_64 Marking qpid-cpp-server-xml-0.24-4.fc19.1.x86_64.rpm to be installed Examining qpid-qmf-0.24-4.fc19.1.x86_64.rpm: qpid-qmf-0.24-4.fc19.1.x86_64 Marking qpid-qmf-0.24-4.fc19.1.x86_64.rpm to be installed Examining qpid-qmf-devel-0.24-4.fc19.1.x86_64.rpm: qpid-qmf-devel-0.24-4.fc19.1.x86_64 Marking qpid-qmf-devel-0.24-4.fc19.1.x86_64.rpm to be installed Examining ruby-qpid-qmf-0.24-4.fc19.1.x86_64.rpm: ruby-qpid-qmf-0.24-4.fc19.1.x86_64 Marking ruby-qpid-qmf-0.24-4.fc19.1.x86_64.rpm to be installed Resolving Dependencies --> Running transaction check ---> Package python-qpid-qmf.x86_64 0:0.24-4.fc19.1 will be installed ---> Package qpid-cpp-client.x86_64 0:0.24-3.fc19.1 will be updated ---> Package qpid-cpp-client.x86_64 0:0.24-4.fc19.1 will be an update ---> Package qpid-cpp-client-devel.x86_64 0:0.24-4.fc19.1 will be installed ---> Package qpid-cpp-client-rdma.x86_64 0:0.24-3.fc19.1 will be updated ---> Package qpid-cpp-client-rdma.x86_64 0:0.24-4.fc19.1 will be an update ---> Package qpid-cpp-debuginfo.x86_64 0:0.24-4.fc19.1 will be installed ---> Package qpid-cpp-server.x86_64 0:0.24-3.fc19.1 will be updated ---> Package qpid-cpp-server.x86_64 0:0.24-4.fc19.1 will be an update ---> Package qpid-cpp-server-ha.x86_64 0:0.24-4.fc19.1 will be installed ---> Package qpid-cpp-server-rdma.x86_64 0:0.24-4.fc19.1 will be installed ---> Package qpid-cpp-server-store.x86_64 0:0.24-4.fc19.1 will be installed ---> Package qpid-cpp-server-xml.x86_64 0:0.24-4.fc19.1 will be installed ---> Package qpid-qmf.x86_64 0:0.24-4.fc19.1 will be installed ---> Package qpid-qmf-devel.x86_64 0:0.24-4.fc19.1 will be installed ---> Package ruby-qpid-qmf.x86_64 0:0.24-4.fc19.1 will be installed --> Finished Dependency Resolution http://repos.fedorapeople.org/repos/openstack/openstack-grizzly/fedora-19/repodata/repomd.xml: [Errno 14] HTTP Error 404 - Not Found Trying other mirror. Dependencies Resolved ============================================================================================================================== Package Arch Version Repository Size ============================================================================================================================== Installing: python-qpid-qmf x86_64 0.24-4.fc19.1 /python-qpid-qmf-0.24-4.fc19.1.x86_64 7.1 M qpid-cpp-client-devel x86_64 0.24-4.fc19.1 /qpid-cpp-client-devel-0.24-4.fc19.1.x86_64 2.7 M qpid-cpp-debuginfo x86_64 0.24-4.fc19.1 /qpid-cpp-debuginfo-0.24-4.fc19.1.x86_64 140 M qpid-cpp-server-ha x86_64 0.24-4.fc19.1 /qpid-cpp-server-ha-0.24-4.fc19.1.x86_64 652 k qpid-cpp-server-rdma x86_64 0.24-4.fc19.1 /qpid-cpp-server-rdma-0.24-4.fc19.1.x86_64 78 k qpid-cpp-server-store x86_64 0.24-4.fc19.1 /qpid-cpp-server-store-0.24-4.fc19.1.x86_64 0.0 qpid-cpp-server-xml x86_64 0.24-4.fc19.1 /qpid-cpp-server-xml-0.24-4.fc19.1.x86_64 107 k qpid-qmf x86_64 0.24-4.fc19.1 /qpid-qmf-0.24-4.fc19.1.x86_64 1.4 M qpid-qmf-devel x86_64 0.24-4.fc19.1 /qpid-qmf-devel-0.24-4.fc19.1.x86_64 225 k ruby-qpid-qmf x86_64 0.24-4.fc19.1 /ruby-qpid-qmf-0.24-4.fc19.1.x86_64 705 k Updating: qpid-cpp-client x86_64 0.24-4.fc19.1 /qpid-cpp-client-0.24-4.fc19.1.x86_64 4.4 M qpid-cpp-client-rdma x86_64 0.24-4.fc19.1 /qpid-cpp-client-rdma-0.24-4.fc19.1.x86_64 264 k qpid-cpp-server x86_64 0.24-4.fc19.1 /qpid-cpp-server-0.24-4.fc19.1.x86_64 6.3 M Transaction Summary ============================================================================================================================== Install 10 Packages Upgrade 3 Packages Total size: 163 M Is this ok [y/d/N]: y Downloading packages: Running transaction check Running transaction test Transaction test succeeded Running transaction Updating : qpid-cpp-client-0.24-4.fc19.1.x86_64 1/16 Updating : qpid-cpp-client-rdma-0.24-4.fc19.1.x86_64 2/16 Updating : qpid-cpp-server-0.24-4.fc19.1.x86_64 3/16 Installing : qpid-qmf-0.24-4.fc19.1.x86_64 4/16 Installing : qpid-cpp-client-devel-0.24-4.fc19.1.x86_64 5/16 Installing : qpid-qmf-devel-0.24-4.fc19.1.x86_64 6/16 Installing : qpid-cpp-server-ha-0.24-4.fc19.1.x86_64 7/16 Installing : python-qpid-qmf-0.24-4.fc19.1.x86_64 8/16 Installing : ruby-qpid-qmf-0.24-4.fc19.1.x86_64 9/16 Installing : qpid-cpp-server-xml-0.24-4.fc19.1.x86_64 10/16 Installing : qpid-cpp-server-rdma-0.24-4.fc19.1.x86_64 11/16 Installing : qpid-cpp-server-store-0.24-4.fc19.1.x86_64 12/16 Installing : qpid-cpp-debuginfo-0.24-4.fc19.1.x86_64 13/16 Cleanup : qpid-cpp-server-0.24-3.fc19.1.x86_64 14/16 Cleanup : qpid-cpp-client-0.24-3.fc19.1.x86_64 15/16 Cleanup : qpid-cpp-client-rdma-0.24-3.fc19.1.x86_64 16/16 Verifying : qpid-cpp-server-ha-0.24-4.fc19.1.x86_64 1/16 Verifying : qpid-cpp-server-xml-0.24-4.fc19.1.x86_64 2/16 Verifying : qpid-cpp-server-rdma-0.24-4.fc19.1.x86_64 3/16 Verifying : python-qpid-qmf-0.24-4.fc19.1.x86_64 4/16 Verifying : qpid-cpp-server-0.24-4.fc19.1.x86_64 5/16 Verifying : qpid-qmf-devel-0.24-4.fc19.1.x86_64 6/16 Verifying : qpid-qmf-0.24-4.fc19.1.x86_64 7/16 Verifying : qpid-cpp-client-rdma-0.24-4.fc19.1.x86_64 8/16 Verifying : qpid-cpp-debuginfo-0.24-4.fc19.1.x86_64 9/16 Verifying : ruby-qpid-qmf-0.24-4.fc19.1.x86_64 10/16 Verifying : qpid-cpp-client-0.24-4.fc19.1.x86_64 11/16 Verifying : qpid-cpp-server-store-0.24-4.fc19.1.x86_64 12/16 Verifying : qpid-cpp-client-devel-0.24-4.fc19.1.x86_64 13/16 Verifying : qpid-cpp-server-0.24-3.fc19.1.x86_64 14/16 Verifying : qpid-cpp-client-rdma-0.24-3.fc19.1.x86_64 15/16 Verifying : qpid-cpp-client-0.24-3.fc19.1.x86_64 16/16 Installed: python-qpid-qmf.x86_64 0:0.24-4.fc19.1 qpid-cpp-client-devel.x86_64 0:0.24-4.fc19.1 qpid-cpp-debuginfo.x86_64 0:0.24-4.fc19.1 qpid-cpp-server-ha.x86_64 0:0.24-4.fc19.1 qpid-cpp-server-rdma.x86_64 0:0.24-4.fc19.1 qpid-cpp-server-store.x86_64 0:0.24-4.fc19.1 qpid-cpp-server-xml.x86_64 0:0.24-4.fc19.1 qpid-qmf.x86_64 0:0.24-4.fc19.1 qpid-qmf-devel.x86_64 0:0.24-4.fc19.1 ruby-qpid-qmf.x86_64 0:0.24-4.fc19.1 Updated: qpid-cpp-client.x86_64 0:0.24-4.fc19.1 qpid-cpp-client-rdma.x86_64 0:0.24-4.fc19.1 qpid-cpp-server.x86_64 0:0.24-4.fc19.1 Complete! Rerun : [boris@localhost ~]$ sudo packstack --answer-file=/home/boris/packstack-answers-20131008-220433.txt ....... 192.168.1.127_swift.pp : [ DONE ] [ ERROR ] ERROR : Error during puppet run : Error: Could not prefetch quantum_network provider 'quantum': Execution of '/usr/bin/quantum net-list --format=csv --column=id --quote=none' returned 1: [Errno 111] Connection refused Please check log file /var/tmp/packstack/20131008-232557-5lcymK/openstack-setup.log for more information Boris, please verify, this "Connection refused" really comes from qpid, and not from quantum. This one looks suspicious for me, since OpenStack Havana doesn't contain quantum at all. (FYI, quantum was renamed to neutron back in April). Matthias, It comes from /var/log/quantum/server.log or /var/log/neutron/server.log. I've also tried Havana&Neutron. 2013-10-09 09:34:54 ERROR [quantum.openstack.common.rpc.impl_qpid] Unable to connect to AMQP server: connection-forced: Authentication failed(320). Sleeping 60 seconds Created attachment 809672 [details]
/valr/log/quantum/server.log
Thank you Boris. Sounds like a valid qpid issue. qpid-cpp-0.24-4.fc20.1 has been pushed to the Fedora 20 stable repository. If problems still persist, please make note of it in this bug report. (In reply to Matthias Runge from comment #15) > Boris, > > please verify, this "Connection refused" really comes from qpid, and not > from quantum. This one looks suspicious for me, since OpenStack Havana > doesn't contain quantum at all. (FYI, quantum was renamed to neutron back in > April). I did a little googling on the error message and found this: https://bugs.launchpad.net/nova/+bug/1172922 Are you guys sure this a bug in qpid and not in how it's being configured? Is your auth properly setup, etc? Darryl, since the issue went away, after adding auth=no to qpidd.conf the issue went away. Since packstack (the deployment tool) places the config in /etc/qpidd.conf and not at /etc/qpid/qpidd.conf, it's still a config placement issue. IMHO one can verify by downgrading qpid. (In reply to Matthias Runge from comment #21) > Darryl, > > since the issue went away, after adding auth=no to qpidd.conf the issue went > away. > > Since packstack (the deployment tool) places the config in /etc/qpidd.conf > and not at /etc/qpid/qpidd.conf, it's still a config placement issue. IMHO > one can verify by downgrading qpid. Can't you update packstack to now put the qpidd.conf file in /etc/qpid/ where it will live going forward? Yes we can, that's the reason why bug #1012814 was opened. Being able to workaround this is not the point. One can always workaround stuff, but with various costs. There were two issues here. 1. Upstream didn't consider backwards compat when moving configs 2. The incompat change was pushed to F19 1. s bad, but 2. compounded the issue. Please consider backwards compatibility in future. |