Red Hat Bugzilla – Bug 611846
Test bundle subsystem with large bundles
Last modified: 2015-02-01 18:26:20 EST
We need to make sure that no part of the bundle system is susceptible to problems due to large bundle files:
Couple of areas we need to test
1) No OOM errors on agent or server, basically that we don't read the whole bundle file into memory while processing it, this applies to both server and agent
2) Database txn timeouts, make sure that if an upload of a bundle or download of a bundle to an agent takes a long time, due to it being large or the link being slow that we don't end up with a txn timeout, e.g. after 10mins
a) Upload a 1gb bundle using all of the various upload mechanisms
b) Install 1gb bundle on an agent
a) Upload a 1gb bundle using all of the various upload mechanisms, where the source of the bundle (url, browser) does not share a fast network with the JON server, e.g. browser upload from laptop to RH based jon server, RH url upload to laptop based jon server
b) Install 1gb bundle on an agent which does not share a fast network with the JON server
Setting directly to ON_QA
Note: to really test a huge file going from server to agent then what is important is that you have a large file in the distribution, not just a large distribution file. This is because each bundle file will be dragged over individually. Meaning, the 1G distribution zip would need to contain probably 1 huge bundle file (maybe a zip of like 3 EAPs or something :) and the recipe file.
I have 1GB file to upload via web.
After 3 minutes I'm getting this exception:
HTTP Status 500 -
Content is loaded to RAM (increase about 870MB).
I'm fairly sure this large-bundle load failure is not a bug but probably a manifestation of our shared session id behavior. Make sure you absolutely don't have any other rhq browser windows open. Restart the server, start a gui portal war session, nav to bundles and try the upload.
If the coregui has a valid sessionid this issue does not appear for me no matter what size bundle I use. If I invalidate the user's session I can recreate it with any size bundle. (note - to invalidate the session: in another browser tab log into a new gui session as the same user, then log out of it. Then, go back to your bundle test tab, it's session is now invalid.)
Assuming the upload now works the test can continue.
I was able to deploy a bundle with a bundle file ~850MB. It wasn't fast, the streaming into and out of the db is a bit cumbersome, but it succeeded.
Closing this one out, following testing. Dev will open a separate bz on the timeout issue.
Mass-closure of verified bugs against JON.