Created attachment 1127428 [details] Test log Description of problem: Running [test] with docker 1.9 fails consistently Version-Release number of selected component (if applicable): How reproducible: 100% excluding random flakes before getting to this point Steps to Reproduce: vagrant origin-init --stage os --os rhel7 --instance-type m4.large <yourname>-dev vagrant up --provider aws vagrant build-origin-base vagrant clone-upstream-repos --clean vagrant checkout-repos vagrant build-origin-base-images vagrant install-origin-assets-base vagrant install-origin vagrant build-origin-base-images vagrant build-origin --images vagrant build-sti --binary-only vagrant test-origin --all > /tmp/out.txt echo $? Actual results: sent 91399 bytes received 210 bytes 61072.67 bytes/sec total size is 90650 speedup is 0.99 [INFO] Back to 'default' project with 'admin' user... Now using project "default" from context named "default/172-18-8-6:8443/system:admin" on server "https://172.18.8.6:8443". [INFO] Waiting for command to finish: '[[ "$(oc get endpoints router --output-version=v1beta3 --template="{{ if .subsets }}{{ len .subsets }}{{ else }}0{{ end }}" || echo "0")" != "0" ]]'... [INFO] Success running command: '[[ "$(oc get endpoints router --output-version=v1beta3 --template="{{ if .subsets }}{{ len .subsets }}{{ else }}0{{ end }}" || echo "0")" != "0" ]]' after 0 seconds [INFO] Validating privileged pod exec Login successful. Using project "default". Now using project "default" from context named "default/172-18-8-6:8443/system:admin" on server "https://172.18.8.6:8443". [INFO] Validating routed app response... [INFO] Response is invalid: [INFO] Stacked plot for log subset "iops" written to /tmp/openshift/test-end-to-end-docker//logs/iops.pdf [INFO] Stacked plot for log subset "paging" written to /tmp/openshift/test-end-to-end-docker//logs/paging.pdf [INFO] Stacked plot for log subset "cpu" written to /tmp/openshift/test-end-to-end-docker//logs/cpu.pdf [INFO] Stacked plot for log subset "queue" written to /tmp/openshift/test-end-to-end-docker//logs/queue.pdf [INFO] Stacked plot for log subset "memory" written to /tmp/openshift/test-end-to-end-docker//logs/memory.pdf [FAIL] !!!!! Test Failed !!!! Expected results: SUCCESS Additional info:
Failing tests are: TestImageStreamImportRedHatRegistry TestRegistryClientConnectPulpRegistry
(In reply to Maru Newby from comment #1) > Failing tests are: > > TestImageStreamImportRedHatRegistry > TestRegistryClientConnectPulpRegistry I see those tests were passing in the attached test log. Re-running to see if failure is consistent.
I missed a step 1) in the recreate steps. You first need to pull in: https://github.com/openshift/vagrant-openshift/pull/396 and rebuild vagrant-openshift so it will use docker-1.9
I ran again with vagrant-openshift 3.0.3 and saw the missing image failures. Those issues should be resolved now, re-running to see if a clean result is possible.
New error suggests a problem with the test environment unrelated to docker 1.9: Running hack/../hack/../test/end-to-end/core.sh:62: executing 'oc create -f examples/image-streams/image-streams-centos7.json --namespace=openshift' expecting success... FAILURE after 0.660s: hack/../hack/../test/end-to-end/core.sh:62: executing 'oc create -f examples/image-streams/image-streams-centos7.json --namespace=openshift' expecting success: the command returned the wrong error code There was no output from the command. Standard error from the command: Error from server: imagestreams "ruby" already exists Error from server: imagestreams "nodejs" already exists Error from server: imagestreams "perl" already exists Error from server: imagestreams "php" already exists Error from server: imagestreams "python" already exists Error from server: imagestreams "wildfly" already exists Error from server: imagestreams "mysql" already exists Error from server: imagestreams "postgresql" already exists Error from server: imagestreams "mongodb" already exists Error from server: imagestreams "jenkins" already exists !!! Error in hack/../hack/../test/end-to-end/../../hack/cmd_util.sh:195 'return 1' exited with status 1 Call stack: 1: hack/../hack/../test/end-to-end/../../hack/cmd_util.sh:195 os::cmd::expect_success(...) 2: hack/../hack/../test/end-to-end/core.sh:62 main(...) Exiting with status 1
I can't reproduce the problem by running hack/test-end-to-end.sh locally, is there something in the reproduction steps that would be creating the imagestreams in question before test/end-to-end/core.sh is invoked?
Not sure but that error looks familiar. It's probably just a different test flake. The error I got was received 3 or 4 times in a row. I'll run it now and see if anything has changed.
Still getting the same error: There was no output from the command. Standard error from the command: W0223 09:25:26.734739 324 cmd.go:213] -p POD is DEPRECATED and will be removed in a future version. Use exec POD instead. STDIN is not a terminal [INFO] Validating routed app response... [INFO] Response is invalid: [INFO] Stacked plot for log subset "iops" written to /tmp/openshift/test-end-to-end-docker//logs/iops.pdf [INFO] Stacked plot for log subset "paging" written to /tmp/openshift/test-end-to-end-docker//logs/paging.pdf [INFO] Stacked plot for log subset "cpu" written to /tmp/openshift/test-end-to-end-docker//logs/cpu.pdf [INFO] Stacked plot for log subset "queue" written to /tmp/openshift/test-end-to-end-docker//logs/queue.pdf [INFO] Stacked plot for log subset "memory" written to /tmp/openshift/test-end-to-end-docker//logs/memory.pdf [FAIL] !!!!! Test Failed !!!! [INFO] Dumping container logs to /tmp/openshift/test-end-to-end-docker//logs [INFO] Dumping all resources to /tmp/openshift/test-end-to-end-docker//logs/export_all.json [INFO] Dumping etcd contents to /tmp/openshift/test-end-to-end-docker//artifacts/etcd_dump.json [INFO] remove the openshift container origin origin [INFO] Stopping k8s docker containers e72514700216 c6cd07f5a398 e1d796eedba5 c502f30abc12 a8dc360a32b0 77627ef9b638 41fd8ef24e43 b8c8034c834e bc5b3a40344f fbe7b2d21f43 [INFO] Removing k8s docker containers d521d265a0f1 a322357b2c83 bd3d27b47494 716bfbe8bb98 d6733fc8c0c8 38026548c0ff 42d8e141666d e72514700216 6e6b78715ea2 c6cd07f5a398 e1d796eedba5 c502f30abc12 182b1e787f9c b9b6f17f7863 d75d7985c9cb c1532a96240a c304758b5b38 f8c56b5c1fa7 a8dc360a32b0 77627ef9b638 2d65d6784fe3 b4e23f440bb8 d6992ab774f1 3a7704763668 5dddd5d239d2 81fb958e15a7 dc003edfe3d3 bfd16f47122e dde40b4f0e1d 65fb9d4482cb 2d3c317b0ed6 1d8f753e9c94 a9cc16804360 1d983358e82c f6d11da91b92 2d16f3644d5a f1e5a23575ee 3a70a24046da c45a880e2e1c 4345a1c76d30 ee0c021f829a 1f012b75146d e7a63f49c79e 04e9dc72186f 969f2eb09f44 b863032996eb 7ba0f9293494 3eb7755d5dc0 22d5c1f13afb 9a123175ed4d 3835b32ccb4d d6ec39ccfe04 5857158d5894 40ac9ee84d9e f1296a331416 30aff2582134 35ff3df4674a 701fe7aad8db dd4bd293787e 41fd8ef24e43 b8c8034c834e bc5b3a40344f fbe7b2d21f43 53a89ca51cc2 c02e7da6a85e 285e452f21c4 32bf9e0c2e07 [INFO] Exiting hack/../hack/test-end-to-end-docker.sh took 631 seconds make[1]: Leaving directory `/data/src/github.com/openshift/origin' Stderr from the command: go tool pprof: signal: broken pipe % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 100 87 100 87 0 0 560 0 --:--:-- --:--:-- --:--:-- 557 100 87 100 87 0 0 560 0 --:--:-- --:--:-- --:--:-- 557 /data/src/github.com/openshift/origin/hack/lib/log.sh: line 30: 6258 Terminated sar -A -o "${binary_logfile}" 1 86400 > /dev/null 2> "${stderr_logfile}" The connection to the server 127.0.0.1:12000 was refused - did you specify the right host or port? The connection to the server 127.0.0.1:12000 was refused - did you specify the right host or port? /data/src/github.com/openshift/origin/hack/util.sh: line 526: 6328 Terminated openshift start etcd --config=${MASTER_CONFIG_DIR}/master-config.yaml &>${LOG_DIR}/etcd.log info: password for stats user admin has been set to LxqlMjL6YQ The import completed successfully. Command "build-logs" is deprecated, use "oc logs build/<build-name>" instead. !!! Error in hack/../hack/../test/end-to-end/core.sh:433 'return 1' exited with status 1 Call stack: 1: hack/../hack/../test/end-to-end/core.sh:433 main(...) Exiting with status 1 % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 100 1306k 0 1306k 0 0 7860k 0 --:--:-- --:--:-- --:--:-- 7871k make[1]: *** [test-end-to-end] Error 1 make: *** [test] Error 2
Created attachment 1129773 [details] Test log 2
Origin PR has merged.