Bug 1711563 - github.com/openshift/origin/vendor/k8s.io/kubernetes/pkg/scheduler/internal/queue TestHighProirotyFlushUnschedulableQLeftover
Summary: github.com/openshift/origin/vendor/k8s.io/kubernetes/pkg/scheduler/internal/q...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: Master
Version: 4.2.0
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: ---
: 4.2.0
Assignee: ravig
QA Contact: Sunil Choudhary
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2019-05-18 16:59 UTC by Maciej Szulik
Modified: 2019-10-16 06:29 UTC (History)
4 users (show)

Fixed In Version:
Doc Type: No Doc Update
Doc Text:
Clone Of:
Environment:
Last Closed: 2019-10-16 06:29:06 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Product Errata RHBA-2019:2922 0 None None None 2019-10-16 06:29:17 UTC

Description Maciej Szulik 2019-05-18 16:59:28 UTC
In https://github.com/openshift/origin/pull/22833 I've seen this happening a few times:

=== RUN   TestHighProirotyFlushUnschedulableQLeftover
==================
WARNING: DATA RACE
Read at 0x00c00000c408 by goroutine 90:
  github.com/openshift/origin/vendor/k8s.io/kubernetes/pkg/scheduler/internal/queue.(*PriorityQueue).flushUnschedulableQLeftover()
      /go/src/github.com/openshift/origin/_output/local/go/src/github.com/openshift/origin/vendor/k8s.io/kubernetes/pkg/scheduler/internal/queue/scheduling_queue.go:497 +0x24a
  github.com/openshift/origin/vendor/k8s.io/kubernetes/pkg/scheduler/internal/queue.(*PriorityQueue).flushUnschedulableQLeftover-fm()
      /go/src/github.com/openshift/origin/_output/local/go/src/github.com/openshift/origin/vendor/k8s.io/kubernetes/pkg/scheduler/internal/queue/scheduling_queue.go:315 +0x41
  github.com/openshift/origin/vendor/k8s.io/apimachinery/pkg/util/wait.JitterUntil.func1()
      /go/src/github.com/openshift/origin/_output/local/go/src/github.com/openshift/origin/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:152 +0x61
  github.com/openshift/origin/vendor/k8s.io/apimachinery/pkg/util/wait.JitterUntil()
      /go/src/github.com/openshift/origin/_output/local/go/src/github.com/openshift/origin/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:153 +0xce
  github.com/openshift/origin/vendor/k8s.io/apimachinery/pkg/util/wait.Until()
      /go/src/github.com/openshift/origin/_output/local/go/src/github.com/openshift/origin/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:88 +0x5a

Previous write at 0x00c00000c408 by goroutine 88:
  github.com/openshift/origin/vendor/k8s.io/kubernetes/pkg/scheduler/internal/queue.TestHighProirotyFlushUnschedulableQLeftover()
      /go/src/github.com/openshift/origin/_output/local/go/src/github.com/openshift/origin/vendor/k8s.io/kubernetes/pkg/scheduler/internal/queue/scheduling_queue_test.go:979 +0x60d
  testing.tRunner()
      /usr/local/go/src/testing/testing.go:827 +0x162

Goroutine 90 (running) created at:
  github.com/openshift/origin/vendor/k8s.io/kubernetes/pkg/scheduler/internal/queue.(*PriorityQueue).run()
      /go/src/github.com/openshift/origin/_output/local/go/src/github.com/openshift/origin/vendor/k8s.io/kubernetes/pkg/scheduler/internal/queue/scheduling_queue.go:315 +0x18a
  github.com/openshift/origin/vendor/k8s.io/kubernetes/pkg/scheduler/internal/queue.NewPriorityQueueWithClock()
      /go/src/github.com/openshift/origin/_output/local/go/src/github.com/openshift/origin/vendor/k8s.io/kubernetes/pkg/scheduler/internal/queue/scheduling_queue.go:307 +0x875
  github.com/openshift/origin/vendor/k8s.io/kubernetes/pkg/scheduler/internal/queue.NewPriorityQueue()
      /go/src/github.com/openshift/origin/_output/local/go/src/github.com/openshift/origin/vendor/k8s.io/kubernetes/pkg/scheduler/internal/queue/scheduling_queue.go:290 +0x68
  github.com/openshift/origin/vendor/k8s.io/kubernetes/pkg/scheduler/internal/queue.TestHighProirotyFlushUnschedulableQLeftover()
      /go/src/github.com/openshift/origin/_output/local/go/src/github.com/openshift/origin/vendor/k8s.io/kubernetes/pkg/scheduler/internal/queue/scheduling_queue_test.go:933 +0x4f
  testing.tRunner()
      /usr/local/go/src/testing/testing.go:827 +0x162

Goroutine 88 (running) created at:
  testing.(*T).Run()
      /usr/local/go/src/testing/testing.go:878 +0x659
  testing.runTests.func1()
      /usr/local/go/src/testing/testing.go:1119 +0xa8
  testing.tRunner()
      /usr/local/go/src/testing/testing.go:827 +0x162
  testing.runTests()
      /usr/local/go/src/testing/testing.go:1117 +0x4ee
  testing.(*M).Run()
      /usr/local/go/src/testing/testing.go:1034 +0x2ee
  main.main()
      _testmain.go:126 +0x332
==================
==================
WARNING: DATA RACE
Read at 0x00c00000c428 by goroutine 90:
  github.com/openshift/origin/vendor/k8s.io/kubernetes/pkg/scheduler/internal/queue.(*PriorityQueue).flushUnschedulableQLeftover()
      /go/src/github.com/openshift/origin/_output/local/go/src/github.com/openshift/origin/vendor/k8s.io/kubernetes/pkg/scheduler/internal/queue/scheduling_queue.go:497 +0x24a
  github.com/openshift/origin/vendor/k8s.io/kubernetes/pkg/scheduler/internal/queue.(*PriorityQueue).flushUnschedulableQLeftover-fm()
      /go/src/github.com/openshift/origin/_output/local/go/src/github.com/openshift/origin/vendor/k8s.io/kubernetes/pkg/scheduler/internal/queue/scheduling_queue.go:315 +0x41
  github.com/openshift/origin/vendor/k8s.io/apimachinery/pkg/util/wait.JitterUntil.func1()
      /go/src/github.com/openshift/origin/_output/local/go/src/github.com/openshift/origin/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:152 +0x61
  github.com/openshift/origin/vendor/k8s.io/apimachinery/pkg/util/wait.JitterUntil()
      /go/src/github.com/openshift/origin/_output/local/go/src/github.com/openshift/origin/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:153 +0xce
  github.com/openshift/origin/vendor/k8s.io/apimachinery/pkg/util/wait.Until()
      /go/src/github.com/openshift/origin/_output/local/go/src/github.com/openshift/origin/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:88 +0x5a

Previous write at 0x00c00000c428 by goroutine 88:
  github.com/openshift/origin/vendor/k8s.io/kubernetes/pkg/scheduler/internal/queue.TestHighProirotyFlushUnschedulableQLeftover()
      /go/src/github.com/openshift/origin/_output/local/go/src/github.com/openshift/origin/vendor/k8s.io/kubernetes/pkg/scheduler/internal/queue/scheduling_queue_test.go:980 +0x7d7
  testing.tRunner()
      /usr/local/go/src/testing/testing.go:827 +0x162

Goroutine 90 (running) created at:
  github.com/openshift/origin/vendor/k8s.io/kubernetes/pkg/scheduler/internal/queue.(*PriorityQueue).run()
      /go/src/github.com/openshift/origin/_output/local/go/src/github.com/openshift/origin/vendor/k8s.io/kubernetes/pkg/scheduler/internal/queue/scheduling_queue.go:315 +0x18a
  github.com/openshift/origin/vendor/k8s.io/kubernetes/pkg/scheduler/internal/queue.NewPriorityQueueWithClock()
      /go/src/github.com/openshift/origin/_output/local/go/src/github.com/openshift/origin/vendor/k8s.io/kubernetes/pkg/scheduler/internal/queue/scheduling_queue.go:307 +0x875
  github.com/openshift/origin/vendor/k8s.io/kubernetes/pkg/scheduler/internal/queue.NewPriorityQueue()
      /go/src/github.com/openshift/origin/_output/local/go/src/github.com/openshift/origin/vendor/k8s.io/kubernetes/pkg/scheduler/internal/queue/scheduling_queue.go:290 +0x68
  github.com/openshift/origin/vendor/k8s.io/kubernetes/pkg/scheduler/internal/queue.TestHighProirotyFlushUnschedulableQLeftover()
      /go/src/github.com/openshift/origin/_output/local/go/src/github.com/openshift/origin/vendor/k8s.io/kubernetes/pkg/scheduler/internal/queue/scheduling_queue_test.go:933 +0x4f
  testing.tRunner()
      /usr/local/go/src/testing/testing.go:827 +0x162

Goroutine 88 (running) created at:
  testing.(*T).Run()
      /usr/local/go/src/testing/testing.go:878 +0x659
  testing.runTests.func1()
      /usr/local/go/src/testing/testing.go:1119 +0xa8
  testing.tRunner()
      /usr/local/go/src/testing/testing.go:827 +0x162
  testing.runTests()
      /usr/local/go/src/testing/testing.go:1117 +0x4ee
  testing.(*M).Run()
      /usr/local/go/src/testing/testing.go:1034 +0x2ee
  main.main()
      _testmain.go:126 +0x332
==================
==================
WARNING: DATA RACE
Write at 0x00c000412420 by goroutine 90:
  runtime.mapdelete_faststr()
      /usr/local/go/src/runtime/map_faststr.go:281 +0x0
  github.com/openshift/origin/vendor/k8s.io/kubernetes/pkg/scheduler/internal/queue.(*UnschedulablePodsMap).delete()
      /go/src/github.com/openshift/origin/_output/local/go/src/github.com/openshift/origin/vendor/k8s.io/kubernetes/pkg/scheduler/internal/queue/scheduling_queue.go:802 +0xc4
  github.com/openshift/origin/vendor/k8s.io/kubernetes/pkg/scheduler/internal/queue.(*PriorityQueue).movePodsToActiveQueue()
      /go/src/github.com/openshift/origin/_output/local/go/src/github.com/openshift/origin/vendor/k8s.io/kubernetes/pkg/scheduler/internal/queue/scheduling_queue.go:674 +0xaf
  github.com/openshift/origin/vendor/k8s.io/kubernetes/pkg/scheduler/internal/queue.(*PriorityQueue).flushUnschedulableQLeftover()
      /go/src/github.com/openshift/origin/_output/local/go/src/github.com/openshift/origin/vendor/k8s.io/kubernetes/pkg/scheduler/internal/queue/scheduling_queue.go:504 +0x44c
  github.com/openshift/origin/vendor/k8s.io/kubernetes/pkg/scheduler/internal/queue.(*PriorityQueue).flushUnschedulableQLeftover-fm()
      /go/src/github.com/openshift/origin/_output/local/go/src/github.com/openshift/origin/vendor/k8s.io/kubernetes/pkg/scheduler/internal/queue/scheduling_queue.go:315 +0x41
  github.com/openshift/origin/vendor/k8s.io/apimachinery/pkg/util/wait.JitterUntil.func1()
      /go/src/github.com/openshift/origin/_output/local/go/src/github.com/openshift/origin/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:152 +0x61
  github.com/openshift/origin/vendor/k8s.io/apimachinery/pkg/util/wait.JitterUntil()
      /go/src/github.com/openshift/origin/_output/local/go/src/github.com/openshift/origin/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:153 +0xce
  github.com/openshift/origin/vendor/k8s.io/apimachinery/pkg/util/wait.Until()
      /go/src/github.com/openshift/origin/_output/local/go/src/github.com/openshift/origin/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:88 +0x5a

Previous read at 0x00c000412420 by goroutine 88:
  runtime.mapaccess1_faststr()
      /usr/local/go/src/runtime/map_faststr.go:12 +0x0
  github.com/openshift/origin/vendor/k8s.io/kubernetes/pkg/scheduler/internal/queue.TestHighProirotyFlushUnschedulableQLeftover()
      /go/src/github.com/openshift/origin/_output/local/go/src/github.com/openshift/origin/vendor/k8s.io/kubernetes/pkg/scheduler/internal/queue/scheduling_queue_test.go:979 +0x58a
  testing.tRunner()
      /usr/local/go/src/testing/testing.go:827 +0x162

Goroutine 90 (running) created at:
  github.com/openshift/origin/vendor/k8s.io/kubernetes/pkg/scheduler/internal/queue.(*PriorityQueue).run()
      /go/src/github.com/openshift/origin/_output/local/go/src/github.com/openshift/origin/vendor/k8s.io/kubernetes/pkg/scheduler/internal/queue/scheduling_queue.go:315 +0x18a
  github.com/openshift/origin/vendor/k8s.io/kubernetes/pkg/scheduler/internal/queue.NewPriorityQueueWithClock()
      /go/src/github.com/openshift/origin/_output/local/go/src/github.com/openshift/origin/vendor/k8s.io/kubernetes/pkg/scheduler/internal/queue/scheduling_queue.go:307 +0x875
  github.com/openshift/origin/vendor/k8s.io/kubernetes/pkg/scheduler/internal/queue.NewPriorityQueue()
      /go/src/github.com/openshift/origin/_output/local/go/src/github.com/openshift/origin/vendor/k8s.io/kubernetes/pkg/scheduler/internal/queue/scheduling_queue.go:290 +0x68
  github.com/openshift/origin/vendor/k8s.io/kubernetes/pkg/scheduler/internal/queue.TestHighProirotyFlushUnschedulableQLeftover()
      /go/src/github.com/openshift/origin/_output/local/go/src/github.com/openshift/origin/vendor/k8s.io/kubernetes/pkg/scheduler/internal/queue/scheduling_queue_test.go:933 +0x4f
  testing.tRunner()
      /usr/local/go/src/testing/testing.go:827 +0x162

Goroutine 88 (running) created at:
  testing.(*T).Run()
      /usr/local/go/src/testing/testing.go:878 +0x659
  testing.runTests.func1()
      /usr/local/go/src/testing/testing.go:1119 +0xa8
  testing.tRunner()
      /usr/local/go/src/testing/testing.go:827 +0x162
  testing.runTests()
      /usr/local/go/src/testing/testing.go:1117 +0x4ee
  testing.(*M).Run()
      /usr/local/go/src/testing/testing.go:1034 +0x2ee
  main.main()
      _testmain.go:126 +0x332
==================
--- FAIL: TestHighProirotyFlushUnschedulableQLeftover (0.00s)
    testing.go:771: race detected during execution of test

https://openshift-gce-devel.appspot.com/build/origin-ci-test/pr-logs/pull/22833/pull-ci-openshift-origin-master-unit/6570/

https://openshift-gce-devel.appspot.com/build/origin-ci-test/pr-logs/pull/22833/pull-ci-openshift-origin-master-unit/6567/

Comment 1 Michal Fojtik 2019-06-10 13:36:57 UTC
Potential fix: https://github.com/openshift/origin/pull/23094

Comment 2 Maciej Szulik 2019-06-19 14:40:58 UTC
This looks like it should be fixed by now. I'm gonna close it, it should be reopened if re-appears.

Comment 4 errata-xmlrpc 2019-10-16 06:29:06 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2019:2922


Note You need to log in before you can comment on or make changes to this bug.