Hide Forgot
Description of problem: update scheduler.json of master can not take effect. Version-Release number of selected component (if applicable): oc v3.11.0-0.10.0 openshift v3.11.0-0.10.0 kubernetes v1.11.0+d4cacc0 How reproducible: always Steps to Reproduce: 1.modify /etc/origin/master/master-config.yaml, add kubernetesMasterConfig: .... schedulerArguments: feature-gates: - BalanceAttachedNodeVolumes=true 2.modify /etc/origin/master/scheduler.json, as follows: scheduler.json as follows: { "apiVersion": "v1", "kind": "Policy", "predicates": [ { "name": "GeneralPredicates" } ], "priorities": [ { "name": "BalancedResourceAllocation", "weight": 4 } ] } 3.modify /etc/origin/master/master.env: DEBUG_LOGLEVEL=10 4.restart master controllers #master-restart controllers 5.create a pvc apiVersion: v1 kind: PersistentVolumeClaim metadata: name: ebs spec: accessModes: - ReadWriteOnce resources: requests: storage: 1Gi persistentVolumeReclaimPolicy: Retain 6.print contrllers log, and then deploy a pod #master-logs controllers controllers 1> ./controller.txt 2>&1 kind: Pod apiVersion: v1 metadata: name: mypod labels: name: frontendhttp spec: containers: - name: myfrontend image: jhou/hello-openshift ports: - containerPort: 80 name: "http-server" volumeMounts: - mountPath: "/tmp" name: aws volumes: - name: aws persistentVolumeClaim: claimName: ebs Actual results: check controller log , as follows: I0802 05:55:32.739516 1 server.go:126] Version: v1.11.0+d4cacc0 I0802 05:55:32.746712 1 factory.go:960] Creating scheduler from configuration: {{ } [{NoVolumeZoneConflict <nil>} {MaxEBSVolumeCount <nil>} {MaxGCEPDVolumeCount <nil>} {MaxAzureDiskVolumeCount <nil>} {MatchInterPodAffinity <nil>} {NoDiskConflict <nil>} {GeneralPredicates <nil>} {PodToleratesNodeTaints <nil>} {CheckNodeMemoryPressure <nil>} {CheckNodeDiskPressure <nil>} {CheckVolumeBinding <nil>} {Region 0xc421019520}] [{SelectorSpreadPriority 1 <nil>} {InterPodAffinityPriority 1 <nil>} {LeastRequestedPriority 1 <nil>} {BalancedResourceAllocation 1 <nil>} {NodePreferAvoidPodsPriority 10000 <nil>} {NodeAffinityPriority 1 <nil>} {TaintTolerationPriority 1 <nil>} {Zone 2 0xc42063baa0}] [] 0 false} I0802 05:55:32.746757 1 factory.go:977] Registering predicate: NoVolumeZoneConflict I0802 05:55:32.746770 1 plugins.go:224] Predicate type NoVolumeZoneConflict already registered, reusing. I0802 05:55:32.746776 1 factory.go:977] Registering predicate: MaxEBSVolumeCount I0802 05:55:32.746782 1 plugins.go:224] Predicate type MaxEBSVolumeCount already registered, reusing. I0802 05:55:32.746787 1 factory.go:977] Registering predicate: MaxGCEPDVolumeCount I0802 05:55:32.746793 1 plugins.go:224] Predicate type MaxGCEPDVolumeCount already registered, reusing. I0802 05:55:32.746798 1 factory.go:977] Registering predicate: MaxAzureDiskVolumeCount I0802 05:55:32.746803 1 plugins.go:224] Predicate type MaxAzureDiskVolumeCount already registered, reusing. I0802 05:55:32.746808 1 factory.go:977] Registering predicate: MatchInterPodAffinity I0802 05:55:32.746814 1 plugins.go:224] Predicate type MatchInterPodAffinity already registered, reusing. I0802 05:55:32.746819 1 factory.go:977] Registering predicate: NoDiskConflict I0802 05:55:32.746824 1 plugins.go:224] Predicate type NoDiskConflict already registered, reusing. I0802 05:55:32.746829 1 factory.go:977] Registering predicate: GeneralPredicates I0802 05:55:32.746835 1 plugins.go:224] Predicate type GeneralPredicates already registered, reusing. I0802 05:55:32.746840 1 factory.go:977] Registering predicate: PodToleratesNodeTaints I0802 05:55:32.746845 1 plugins.go:224] Predicate type PodToleratesNodeTaints already registered, reusing. I0802 05:55:32.746851 1 factory.go:977] Registering predicate: CheckNodeMemoryPressure I0802 05:55:32.746856 1 plugins.go:224] Predicate type CheckNodeMemoryPressure already registered, reusing. I0802 05:55:32.746863 1 factory.go:977] Registering predicate: CheckNodeDiskPressure I0802 05:55:32.746868 1 plugins.go:224] Predicate type CheckNodeDiskPressure already registered, reusing. I0802 05:55:32.746874 1 factory.go:977] Registering predicate: CheckVolumeBinding I0802 05:55:32.746879 1 plugins.go:224] Predicate type CheckVolumeBinding already registered, reusing. I0802 05:55:32.746884 1 factory.go:977] Registering predicate: Region I0802 05:55:32.746897 1 factory.go:992] Registering priority: SelectorSpreadPriority I0802 05:55:32.746904 1 plugins.go:336] Priority type SelectorSpreadPriority already registered, reusing. I0802 05:55:32.746916 1 factory.go:992] Registering priority: InterPodAffinityPriority I0802 05:55:32.746922 1 plugins.go:336] Priority type InterPodAffinityPriority already registered, reusing. I0802 05:55:32.746930 1 factory.go:992] Registering priority: LeastRequestedPriority I0802 05:55:32.746935 1 plugins.go:336] Priority type LeastRequestedPriority already registered, reusing. I0802 05:55:32.746942 1 factory.go:992] Registering priority: BalancedResourceAllocation I0802 05:55:32.746948 1 plugins.go:336] Priority type BalancedResourceAllocation already registered, reusing. I0802 05:55:32.746955 1 factory.go:992] Registering priority: NodePreferAvoidPodsPriority I0802 05:55:32.746961 1 plugins.go:336] Priority type NodePreferAvoidPodsPriority already registered, reusing. I0802 05:55:32.746968 1 factory.go:992] Registering priority: NodeAffinityPriority I0802 05:55:32.746975 1 plugins.go:336] Priority type NodeAffinityPriority already registered, reusing. I0802 05:55:32.746982 1 factory.go:992] Registering priority: TaintTolerationPriority I0802 05:55:32.746988 1 plugins.go:336] Priority type TaintTolerationPriority already registered, reusing. I0802 05:55:32.746995 1 factory.go:992] Registering priority: Zone t:{} MaxAzureDiskVolumeCount:{} NoDiskConflict:{} PodToleratesNodeTaints:{} CheckNodeDiskPressure:{} CheckVolumeBinding:{} NoVolumeZoneConflict:{} MaxEBSVolumeCount:{} MatchInterPodAffinity:{} GeneralPredicates:{}]' and priority functions 'map[LeastRequestedPriority:{} BalancedResourceAllocation:{} NodePreferAvoidPodsPriority:{} NodeAffinityPriority:{} TaintTolerationPriority:{} Zone:{} SelectorSpreadPriority:{} InterPodAffinityPriority:{}]' Expected results: logging updated scheduler policy according to scheduler.json Additional info:
this problem happen definitely in aws env, template: http://git.app.eng.bos.redhat.com/git/openshift-misc.git/plain/v3-launch-templates/functionality-testing/aos-3_11/vars-aws/rpm-rhel7-s3_registry-aws-cloudprovider But in other env, such as gce , it not happen
I am not able to reproduce this, as everything is working as expected. I did following steps and it is evident from the logs, the scheduler was started with updated scheduler.json: 1.Updated debug level in /etc/origin/master/master.env: DEBUG_LOGLEVEL=10 2. Used following scheduler.json: scheduler.json as follows: { "apiVersion": "v1", "kind": "Policy", "predicates": [ { "name": "GeneralPredicates" } ], "priorities": [ { "name": "BalancedResourceAllocation", "weight": 4 } ] } 3. Restarted controllers on master: #/usr/local/bin/master-restart controllers 4. Here are the logs: I0806 20:07:21.733455 1 factory.go:960] Creating scheduler from configuration: {{ } [{GeneralPredicates <nil>}] [{BalancedResourceAllocation 4 <nil>}] [] 0 false} I0806 20:07:21.733522 1 factory.go:977] Registering predicate: GeneralPredicates I0806 20:07:21.733556 1 plugins.go:224] Predicate type GeneralPredicates already registered, reusing. I0806 20:07:21.733584 1 factory.go:992] Registering priority: BalancedResourceAllocation I0806 20:07:21.733632 1 plugins.go:336] Priority type BalancedResourceAllocation already registered, reusing. I0806 20:07:21.733670 1 factory.go:1049] Creating scheduler with fit predicates 'map[GeneralPredicates:{}]' and priority functions 'map[BalancedResourceAllocation:{}]' I have my 3.11 cluster running in AWS in case you want to verify.
I test again in aws cluster When do step4 "master-restart controllers" it print "2". I think this cause scheduler.json not effect. Because I run the command in gce cluster, it not print anything. I want to know if you meet the same problem.
can i get access to your aws cluster?
It seems this issue not happen any more.
As comment 9, move to verified.
verified version: oc v3.11.57 kubernetes v1.11.0+d4cacc0 features: Basic-Auth GSSAPI Kerberos SPNEGO Server https://ip-172-18-15-99.ec2.internal:8443 openshift v3.11.57 kubernetes v1.11.0+d4cacc0
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2019:0758