Description of problem: Hello, while i am on the main beaker website, im trying to export the system key/values and i am getting the below HTTP 500 error message. "500 Internal error The server encountered an unexpected condition which prevented it from fulfilling the request. Powered by CherryPy 2.3.0 " Version-Release number of selected component (if applicable): Beaker 25.1 How reproducible: Easy to reproduce. Steps to Reproduce: 1. Go to https://beaker.engineering.redhat.com/ 2. log in 3. Click at the "Admin" tab 4. Export 5. Select "System Key/Values" 6. click the button "Export CSV" Actual results: 500 Internal error Expected results: Downloadable CSV file expected. Additional info: Feel free to ask me for more information if needed. irc:theo email:tapazogl thank you, Theo
Created attachment 1423857 [details] memory error on the server
Found the problem on the server. Seems like while compiling the CSV it ran out of memory.
Dear Theodoros, do you need the key/values of *all* systems, or just a specific subset?
Hello Roman, because of https://redhat.service-now.com/surl.do?n=PNT0187703, i need key/values of *all* systems, so i can find the ones that have "mptsas/mptspi disks and/or e1000 network interfaces" in order to exclude those systems from installing RHEL-8 since these drivers are not supported. Few days ago i did export of *all* systems "excluded families" and worked flawlessly. thanks, Theo PnT & CEE Labs team
There is no easy solution to this problem (too much memory needed to export all systems) and thus it won't be fixed any time soon. Can you instead construct a suitable XML host filter and use `bkr list-systems --xml-filter=` to get a list of relevant FQDNs, and then do the exclusions just for those?
Hi, I think the problem here is really the functionality of the CSV exporting. When you export any of the CSV types it dumps literally everything from the database. Considering that this includes systems that were "Removed" years ago, there are extremely few cases where someone actually needs to export info for ALL the systems. Instead of considering this a bug against a MemoryError maybe we can treat it as an RFE to improve the usability of CSV exports. Is it possible to implement CSV exporting based on system search results? For example: - I search from the Beaker home page (All Systems) for 'Devices/Description' containing the string "be3" - 87 systems are returned - Perhaps, under the "+Add" button there is an "Export CSV" button which will pass the results to the main CSV Export page where you can then select the CSV type that you want to export for just those 87 systems.
Dear Bill, I like the idea. Not sure how much of a change it is with the current code, but the use case sounds reasonable.
Hi Dan/Roman, while running the export of key/values i am checking the cpu/ram resources and i don't see running out of memory. Yes, beaker RAM resources is not in a good shape but, that's why there is the swap memory and i don't see the system to allocate any. I don't know in deep how beaker works at the background that's why i am trying to understand. Exporting is a useful features for my team and the key/values is the only one it doesn't work. I don't know how to construct a suitable XML host filter. Is there an update on this issue or maybe at Bill's proposal ? thanks, Theo
Dear Theodoros, I believe you that this would make your life a lot easier. Thank you as well for your help in checking the memory usage. We haven't had time so far to look into this issue and therefore do not know how much work is involved with it. With that I can't unfortunately give you an estimate when we will ship this. Having said that, I bumped the priority in our board so that we'll be looking at this item next, once all the fire fighting is done (RHEL8). A suitable host filter is whatever you would usually insert in a hostRequires element. You can find examples here: https://beaker-project.org/docs/user-guide/job-xml.html#job-workflow-details Note, that you'll have to scroll down a fair bit. For example, if I would like to find all systems which have a network card with a description of: NetXtreme II BCM5708 Gigabit Ethernet I would use: bkr list-systems --xml-filter='<and><device op="=" type="network" description="NetXtreme II BCM5708 Gigabit Ethernet" /></and>' Mind the single quotes around the filter to avoid my shell interpolating any values away. I do know it's not ideal, but hopefully will get you going in the mean time.
Bumping the priority on this as well as https://bugzilla.redhat.com/show_bug.cgi?id=1600587 since they both seem related. Though they get different errors and I believe the beaker logs get different errors: The key/value gets: 500 Internal error The server encountered an unexpected condition which prevented it from fulfilling the request. System Excluded Families gets: Internal Server Error The server encountered an internal error or misconfiguration and was unable to complete your request. Please contact the server administrator, root@localhost and inform them of the time the error occurred, and anything you might have done that may have caused the error. More information about this error may be available in the server error log.
This is still an issue in prod. https://beaker.engineering.redhat.com/csv/action_export?csv_type=exclude Internal Server Error The server encountered an internal error or misconfiguration and was unable to complete your request. Please contact the server administrator, root@localhost and inform them of the time the error occurred, and anything you might have done that may have caused the error. More information about this error may be available in the server error log. Apache/2.2.15 (Red Hat) Server at beaker.engineering.redhat.com Port 443
After one year, this is still an issue. 500 Internal error The server encountered an unexpected condition which prevented it from fulfilling the request. Powered by CherryPy 2.3.0