Hide Forgot
Created attachment 1196458 [details] runtime stack Description of problem: When doing a trying to list images, we've got an our of memory error : oc get image fatal error: runtime: out of memory Providing the runtime stack in attached runtime.txt file Version-Release number of selected component (if applicable): 3.2.0 How reproducible: On customer end Steps to Reproduce: 1.oc get images 2. 3. Actual results: oc get images gives out of memory error Expected results: oc get images shall work fine Additional info: We are having a lot of memory consumption in production
Miheer, When they tried to prune or list images, how much free memory did the system have where they were running the command? How much total memory does that system have? Do you know how many images they have in etcd? Do you know what the size of their etcd database is? Thanks, Andy
FYI if an attempt to run 'oc get images' or 'oadm prune images' runs out of memory, as we see in https://bugzilla.redhat.com/attachment.cgi?id=1196458, it is the system where the oc/oadm command is executed that is running out of memory and not necessarily the system where atomic-openshift-* is running.
Here is the code for a script that can display the etcd usage sorted by largest entries: https://github.com/ncdc/etcdstats I'm also attaching a precompiled binary you can use
I can't attach the binary as it's larger than the limit. Will need to find an alternative.
Binary: https://github.com/ncdc/etcdstats/releases/download/v0.1.0/etcdstats-v0.1.0-x86_64.tar.gz