Hide Forgot
Description of problem: With the latest beaker we are unable to export the 'Excludes' and the 'Key/Value' pairs. This is going to severely impact the ability to administer the Excluded Distros for the Admins. Version-Release number of selected component (if applicable): beaker-server-25.4-1.el6bkr.hotfix.1.noarch python-beaker-1.3.1-7.el6.noarch beaker-common-25.4-1.el6bkr.hotfix.1.noarch beaker-server-redhat-3-1.el6eng.noarch How reproducible: Try to export via: https://beaker.engineering.redhat.com/csv/action_export?csv_type=exclude -or- https://beaker.engineering.redhat.com/csv/action_export?csv_type=keyvalue Steps to Reproduce: 1. Connect to beaker server beaker.engineering.redhat.com 2. Try to export either 'System Key/Values' or 'System Excluded Families' Actual results: Receive a page with "500 Internal error" Expected results: Should be presented with a CSV file to save/view. Additional info: The following was observed in /var/log/beaker/server-errors.log Jul 12 14:29:05 beaker-server beaker-server[7143]: bkr.server ERROR Exception on /csv/action_export [GET] Jul 12 14:29:05 beaker-server beaker-server[7143]: Traceback (most recent call last): Jul 12 14:29:05 beaker-server beaker-server[7143]: File "/usr/lib/python2.6/site-packages/flask/app.py", line 1817, in wsgi_app Jul 12 14:29:05 beaker-server beaker-server[7143]: response = self.full_dispatch_request() Jul 12 14:29:05 beaker-server beaker-server[7143]: File "/usr/lib/python2.6/site-packages/flask/app.py", line 1479, in full_dispatch_request Jul 12 14:29:05 beaker-server beaker-server[7143]: response = self.process_response(response) Jul 12 14:29:05 beaker-server beaker-server[7143]: File "/usr/lib/python2.6/site-packages/flask/app.py", line 1691, in process_response Jul 12 14:29:05 beaker-server beaker-server[7143]: response = handler(response) Jul 12 14:29:05 beaker-server beaker-server[7143]: File "/usr/lib/python2.6/site-packages/bkr/server/wsgi.py", line 150, in commit_or_rollback_session Jul 12 14:29:05 beaker-server beaker-server[7143]: session.rollback() Jul 12 14:29:05 beaker-server beaker-server[7143]: File "/usr/lib64/python2.6/site-packages/sqlalchemy/orm/scoping.py", line 150, in do Jul 12 14:29:05 beaker-server beaker-server[7143]: return getattr(self.registry(), name)(*args, **kwargs) Jul 12 14:29:05 beaker-server beaker-server[7143]: File "/usr/lib64/python2.6/site-packages/sqlalchemy/orm/session.py", line 740, in rollback Jul 12 14:29:05 beaker-server beaker-server[7143]: self.transaction.rollback() Jul 12 14:29:05 beaker-server beaker-server[7143]: File "/usr/lib64/python2.6/site-packages/sqlalchemy/orm/session.py", line 403, in rollback Jul 12 14:29:05 beaker-server beaker-server[7143]: transaction._rollback_impl() Jul 12 14:29:05 beaker-server beaker-server[7143]: File "/usr/lib64/python2.6/site-packages/sqlalchemy/orm/session.py", line 434, in _rollback_impl Jul 12 14:29:05 beaker-server beaker-server[7143]: self._restore_snapshot(dirty_only=self.nested) Jul 12 14:29:05 beaker-server beaker-server[7143]: File "/usr/lib64/python2.6/site-packages/sqlalchemy/orm/session.py", line 286, in _restore_snapshot Jul 12 14:29:05 beaker-server beaker-server[7143]: for s in self.session.identity_map.all_states(): Jul 12 14:29:05 beaker-server beaker-server[7143]: File "/usr/lib64/python2.6/site-packages/sqlalchemy/orm/identity.py", line 177, in all_states Jul 12 14:29:05 beaker-server beaker-server[7143]: return dict.values(self) Jul 12 14:29:05 beaker-server beaker-server[7143]: MemoryError: <bound method CSV.action_export of <bkr.server.CSV_import_export.CSV object at 0x7f40651edf90>>
This looks like it's the same problem as it was initially reported by Bug 1568751. I would bump the priority of 1568751 since it seems it would benefit from the same solution...
Bumping the priority on this as well as https://bugzilla.redhat.com/show_bug.cgi?id=1568751 since they both seem related. Though they get different errors and I believe the beaker logs get different errors: The key/value gets: 500 Internal error The server encountered an unexpected condition which prevented it from fulfilling the request. System Excluded Families gets: Internal Server Error The server encountered an internal error or misconfiguration and was unable to complete your request. Please contact the server administrator, root@localhost and inform them of the time the error occurred, and anything you might have done that may have caused the error. More information about this error may be available in the server error log.
As a workaround, you can extract a similar CSV to what Beaker would produce for the "System Excluded Families" CSV export by using a Teiid query like this: SELECT 'exclude' AS csv_type, fqdn, arch.arch, osmajor.osmajor as family, NULL as "update", 'True' as excluded FROM Beaker.system INNER JOIN Beaker.exclude_osmajor ON system_id = system.id INNER JOIN Beaker.osmajor ON exclude_osmajor.osmajor_id = osmajor.id INNER JOIN Beaker.arch ON exclude_osmajor.arch_id = arch.id UNION SELECT 'exclude' AS csv_type, fqdn, arch.arch, osmajor.osmajor as family, osversion.osminor as "update", 'True' as excluded FROM Beaker.system INNER JOIN exclude_osversion ON system_id = system.id INNER JOIN osversion ON exclude_osversion.osversion_id = osversion.id INNER JOIN osmajor ON osversion.osmajor_id = osmajor.id INNER JOIN arch ON exclude_osversion.arch_id = arch.id; Unfortunately there is no easy way to make the psql client produce a CSV formatted result from that query. We can kind of fake it, by using the following psql options: psql postgresql://... -o exclude.csv -c '\pset border 1' -c '\pset format unaligned' -c '\pset fieldsep ,' -c '\pset footer off' -c "SELECT ..." Be careful to escape " inside the query from the shell as \". Also note that this trick for faking the CSV output will only work if the values do not contain commas or quotes, because it is not doing any proper CSV escaping. For this particular query (exclude families) it should be fine since system FQDNs and OS major names will not normally contain a comma or quote character. But it won't necessarily work for other Teiid queries.
This is still an issue in prod. https://beaker.engineering.redhat.com/csv/action_export?csv_type=exclude Internal Server Error The server encountered an internal error or misconfiguration and was unable to complete your request. Please contact the server administrator, root@localhost and inform them of the time the error occurred, and anything you might have done that may have caused the error. More information about this error may be available in the server error log. Apache/2.2.15 (Red Hat) Server at beaker.engineering.redhat.com Port 443
Still an issue in Beaker version 26 as well. Should be fixed in version 27, which prod will soon be updated to. Hopefully.
(In reply to Yaniv Liberman from comment #7) > Still an issue in Beaker version 26 as well. > > Should be fixed in version 27, which prod will soon be updated to. Hopefully. This will be not fixed in 27. This will land in Beaker in 28.
Ack. I was told by rebarbos on #beaker that he tested it in v27 development and it worked...