Bug 1679973 - Sometimes can not connect to api server
Summary: Sometimes can not connect to api server
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: Master
Version: 4.1.0
Hardware: Unspecified
OS: Unspecified
unspecified
medium
Target Milestone: ---
: 4.1.0
Assignee: Michal Fojtik
QA Contact: Xingxing Xia
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2019-02-22 11:49 UTC by Junqi Zhao
Modified: 2019-06-04 10:44 UTC (History)
5 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2019-06-04 10:44:26 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Product Errata RHBA-2019:0758 0 None None None 2019-06-04 10:44:33 UTC

Description Junqi Zhao 2019-02-22 11:49:25 UTC
Description of problem:
run the following loop, when the number reaches to 8, "the server could not find the requested resource (get projectrequests.project.openshift.io)" and later connected to api server.

$ for i in {1..19}; do echo $i; oc new-project test$i; oc new-app centos/ruby-25-centos7~https://github.com/sclorg/ruby-ex.git; sleep 15s; done


8
Error from server (NotFound): the server could not find the requested resource (get projectrequests.project.openshift.io)
warning: Cannot find git. Ensure that it is installed and in your path. Git is required to work with git repositories.
....
9
Now using project "test9" on server "https://api.**.qe.devcluster.openshift.com:6443".

project "test8" are not created
$ oc get ns | grep test
test1                                        Active   38m
test10                                       Active   36m
test11                                       Active   36m
test12                                       Active   35m
test13                                       Active   35m
test14                                       Active   35m
test15                                       Active   35m
test16                                       Active   34m
test17                                       Active   34m
test18                                       Active   34m
test19                                       Active   33m
test2                                        Active   38m
test3                                        Active   38m
test4                                        Active   38m
test5                                        Active   37m
test6                                        Active   37m
test7                                        Active   37m
test9                                        Active   36m



Version-Release number of selected component (if applicable):
$ oc get clusterversion
NAME      VERSION                             AVAILABLE   PROGRESSING   SINCE   STATUS
version   4.0.0-0.nightly-2019-02-21-215247   True        False         7h      Cluster version is 4.0.0-0.nightly-2019-02-21-215247


How reproducible:
Sometimes

Steps to Reproduce:
1. See the description part
2.
3.

Actual results:
Sometimes can not connect to api server

Expected results:
api server should be stable

Additional info:

Comment 1 David Eads 2019-02-26 12:56:08 UTC
Where are you running this from.

1. master node -> direct to master node
2. pod -> service network -> master
3. pod -> pod network -> master
4. external -> elb -> master

Comment 2 Junqi Zhao 2019-02-26 13:51:59 UTC
(In reply to David Eads from comment #1)
> Where are you running this from.
> 
> 1. master node -> direct to master node
> 2. pod -> service network -> master
> 3. pod -> pod network -> master
> 4. external -> elb -> master

from master node

Comment 3 David Eads 2019-02-26 13:57:19 UTC
From master node using which route to the server?  Directly to the master?  Doing that, you would expect API requests to be dropped before the server is healthy.  It accepts requests before it is healthy, but does not join the service until passes a health check

Comment 4 Junqi Zhao 2019-02-27 00:36:23 UTC
(In reply to David Eads from comment #3)
> From master node using which route to the server?  Directly to the master? 
> Doing that, you would expect API requests to be dropped before the server is
> healthy.  It accepts requests before it is healthy, but does not join the
> service until passes a health check

I sshed to the master, and run the commands there, will try again today, since it's not always happen,
not sure whether it can be reproduced or not

Comment 5 Junqi Zhao 2019-02-27 07:09:42 UTC
Not reproduced with
# oc get clusterversion
NAME      VERSION                             AVAILABLE   PROGRESSING   SINCE   STATUS
version   4.0.0-0.nightly-2019-02-26-125216   True        False         6h12m   Cluster version is 4.0.0-0.nightly-2019-02-26-125216

Comment 6 Michal Fojtik 2019-03-05 09:34:25 UTC
I believe this was fixed recently as we now wait for discovery to contain all CRD and api's before reporting ready from apiserver.

Comment 7 Junqi Zhao 2019-03-07 10:29:35 UTC
no such issue with 4.0.0-0.nightly-2019-03-06-074438, api-server performs well

Comment 10 errata-xmlrpc 2019-06-04 10:44:26 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2019:0758


Note You need to log in before you can comment on or make changes to this bug.