DescriptionGreg Rodriguez II
2019-05-21 15:47:26 UTC
Description of problem:
Customer is facing issues making calls using services in OCP 3.10.XX. When they see these issues, these are the errors from their application:
~~~~
2019-05-02 15:09:11.940 INFO 9 --- [xtgen.svc:27017] org.mongodb.driver.cluster: Exception in monitor thread while connecting to server mongodb.ode-b92590-nextgen.svc:27017com.mongodb.MongoSocketOpenException: Exception opening socket
~~~~
When the sdn pod logs are reviewed, it was found that the service endpoints were removed and not added back. Restarting the sdn pod resolves the issues.
NOTE: There also were issues with xtables locks as this environment has >16k iptables rules. To resolve these issues, the iptables min-sync-period parameter was increased to 30s. In addition, BZ 1669311 was hit and the limits/requests of the sdn and ovs pods were changed to prevent oom-killer events.
Version-Release number of selected component (if applicable):
OCP v3.11.111
Additional info:
Providing sdn and ovs pod logs, as well as sosreport from affected node to private update
Comment 4Greg Rodriguez II
2019-05-21 16:23:04 UTC
(In reply to Greg Rodriguez II from comment #0)
Correction to BZ description, this is for OCP v3.10.111 and confirmed from customer.