Bug 1139688
Summary: | [scale] - "Unable to evaluate payload" when loading cluster policy | ||||||||
---|---|---|---|---|---|---|---|---|---|
Product: | Red Hat Enterprise Virtualization Manager | Reporter: | Eldad Marciano <emarcian> | ||||||
Component: | ovirt-engine | Assignee: | Liran Zelkha <lzelkha> | ||||||
Status: | CLOSED CURRENTRELEASE | QA Contact: | Eldad Marciano <emarcian> | ||||||
Severity: | high | Docs Contact: | |||||||
Priority: | unspecified | ||||||||
Version: | 3.4.1-1 | CC: | aberezin, bazulay, ecohen, emarcian, gklein, iheim, lpeer, oourfali, rbalakri, Rhev-m-bugs, yeylon, yobshans | ||||||
Target Milestone: | --- | ||||||||
Target Release: | 3.5.0 | ||||||||
Hardware: | x86_64 | ||||||||
OS: | Linux | ||||||||
Whiteboard: | infra | ||||||||
Fixed In Version: | org.ovirt.engine-root-3.5.0-15 | Doc Type: | Bug Fix | ||||||
Doc Text: | Story Points: | --- | |||||||
Clone Of: | Environment: | ||||||||
Last Closed: | 2015-02-17 17:10:44 UTC | Type: | Bug | ||||||
Regression: | --- | Mount Type: | --- | ||||||
Documentation: | --- | CRM: | |||||||
Verified Versions: | Category: | --- | |||||||
oVirt Team: | Infra | RHEL 7.3 requirements from Atomic Host: | |||||||
Cloudforms Team: | --- | Target Upstream Version: | |||||||
Embargoed: | |||||||||
Attachments: |
|
Description
Eldad Marciano
2014-09-09 12:57:29 UTC
Please give much more info: 1. How many simulatenous users 2. Attach 3 jstack logs for server 3. Attach full engine.log Liran from some reason, at some point nothing writing to server log there is no traffic at this log since 10-Sep. Once i reproduced the bug i wanted to add the server log since we saw the error \ exception there this bug might be side-affect of another issue, to me it looks like resources lack. please advise. Created attachment 937512 [details]
jstack logs
Created attachment 937605 [details]
follow thread "ajp-/127.0.0.1:8702-10"
Following the thread dump, it seems all your HTTP threads are blocking on 0x00000007025f87d0, which is a lock by RollingFile of the Log4J infrastructure we use. The weird thing is that no thread is locking that file. I suggest the following: 1. Please enclose the full log. 2. Please check the free file handles and free size of your disk. 3. I'll check if there are issues with our log4j The log you sent has data in Sep-15, so I'm not following you. It seems like a network issue from the server to your browser... Is it always reproducible, and only for this specific action? (In reply to Liran Zelkha from comment #6) > The log you sent has data in Sep-15, so I'm not following you. > It seems like a network issue from the server to your browser... > Is it always reproducible, and only for this specific action? Yep. Are the jstack files from exactly when you activate this scenario? Because it doesn't show anything related... not correlated to description. because of that i comment the jstcks files to "follow thread "ajp-/127.0.0.1:8702-10" It seems like there are 2 DB queries that execute once the Edit Cluster option is selected: SELECT * FROM ((SELECT distinct vms.* FROM vms WHERE vms.vds_group_name LIKE 'PerfCluster' ) ORDER BY vm_name ASC ) as T1 OFFSET (1 -1) LIMIT 2147483647 SELECT * FROM ((SELECT distinct vds.* FROM vds WHERE vds.vds_group_name LIKE 'PerfCluster' ) ORDER BY vds_name,vds_name ASC ) as T1 OFFSET (1 -1) LIMIT 2147483647 These are heavy queries, and probably cause the timeout. I will check why these are called. Eldad, can you run both queries (change the cluster name to your cluster name) and tell me how long it takes them to execute? see the results using pgadmin: SELECT * FROM ((SELECT distinct vms.* FROM vms WHERE vms.vds_group_name LIKE 'dc_fake' ) ORDER BY vm_name ASC ) as T1 OFFSET (1 -1) LIMIT 2147483647 results first run 32 sec, second run 26 sec. ============================================================== SELECT * FROM ((SELECT distinct vds.* FROM vds WHERE vds.vds_group_name LIKE 'dc_fake' ) ORDER BY vds_name,vds_name ASC ) as T1 OFFSET (1 -1) LIMIT 214748364 results first run 4 sec, second run 3.8 sec. looks like using vds much more faster. which query is it? i want to replace it im mine 3.4 setup The change I did is for 3.5, not 3.4. Can't be easily backported. verified on VT13.7 rhev 3.5.0 was released. closing. |