Description of problem: During docker run (e.g. docker run --rm <CONTAINER> <SCRIPT>) the docker client run sometimes crashes with the error message "Unrecognized input header" if SCRIPT is doing multithreaded outputs to stdout and stderr. This seems to be caused by broken multiplexing of stdout and stderr in the docker daemon. `docker version`: Docker version 1.9.1-fc23, build f7c1d52-dirty `uname -a`: Linux fedora23-1.novalocal 4.2.3-300.fc23.x86_64 #1 SMP Mon Oct 5 15:42:54 UTC 2015 x86_64 x86_64 x86_64 GNU/Linux How reproducible: This can be reproduced by doing a "docker run" with a executable (see below) which does multithreaded writes to stdout and stderr. Steps to Reproduce: 1. Create Dockerfile $ cat > Dockerfile <<EOF FROM fedora:23 RUN dnf install -y gcc-c++ && dnf clean all COPY main.cpp /main.cpp RUN g++ -std=c++11 -pthread /main.cpp -o /test ENTRYPOINT /test EOF 2. Create test program $ cat > main.cpp <<EOF #include <thread> #include <iostream> void print_stdout() { for (int i=0; i<10000; ++i) { std::cout << "tttttttttttttttttttttttttttttttttttt" "tttttttttttttttttttttttttttttttttttt" "tttttttttttttttttttttttttttttttttttt" "tttttttttttttttttttttttttttttttttttt" "tttttttttttttttttttttttttttttttttttt" ; } } void print_stderr() { for (int i=0; i<10000; i++) { std::cerr << "Error!"; } } int main() { std::thread t1(print_stdout); std::thread t2(print_stderr); t1.join(); t2.join(); } EOF 3. Create test script: $ cat > run-test.sh <<EOF #/bin/bash set -e set -o pipefail for i in \`seq 1 10\`; do docker run --rm dockertest 2>&1 | { grep "Unrecognized input header" || true; } done echo "No error" EOF 4. Make test script executable: chmod +x run-test.sh 5. Build container: docker build -t dockertest . 6. Execute test: ./run-test.sh Actual Results: Script prints "Unrecognized input header" which comes from the crashed docker client. Expected Results: Script prints "No error" Additional Notes: This bug does not occur when I build and run upstream docker 1.9.1 (following https://docs.docker.com/v1.5/contributing/devenvironment/).
Missed workaround: If it's not necessary to distinguish between stdout and stderr the error can be avoided by redirecting stderr to stdout inside the container.
Additional Note: This bug also appeared after upgrading docker from 1.7.0 to 1.8.2 on F22.
Looking, actually tried everything (including your c++ code, C code with threads and go with goroutines) to reproduce this and I'm not able to get the error you're getting.
Hm that's strange. I get this on all Fedora machines I tested: - Laptop with F23 and docker 1.8.2-fc23 - Workstation with F22 and docker 1.8.2-fc22 - VM with F22 (cloud image) and docker 1.8.2-fc22 - VM with F23 (cloud image) and docker 1.9.1-fc23 Maybe worth to note: On all machines I have the following differences to the default config: - Different networking settings (--bip=172.17.42.1/24) - btrfs storage backend My docker info on the F23 VM: $ docker info Containers: 1 Images: 174 Server Version: 1.9.1-fc23 Storage Driver: btrfs Build Version: Btrfs v4.2.2 Library Version: 101 Execution Driver: native-0.2 Logging Driver: json-file Kernel Version: 4.2.3-300.fc23.x86_64 Operating System: Fedora 23 (Cloud Edition) CPUs: 8 Total Memory: 7.799 GiB Name: fedora23-1.novalocal ID: 2FR6:4FVL:MLE4:XRTV:ENO3:2NFI:NZF5:V4TA:MCN6:MD7D:673B:AL7P
Thanks Tobias, my setup is entirely different, I'll try to setup my vms as you mentioned in hope to reproduce it.
Any update on this?
I still cannot reproduce the original issue - I'd say, Tobias can you try out if this is still happening with docker 1.10.3? Thanks
Well we will claim it is fixed in docker-1.10.3 :^)
This message is a reminder that Fedora 23 is nearing its end of life. Approximately 4 (four) weeks from now Fedora will stop maintaining and issuing updates for Fedora 23. It is Fedora's policy to close all bug reports from releases that are no longer maintained. At that time this bug will be closed as EOL if it remains open with a Fedora 'version' of '23'. Package Maintainer: If you wish for this bug to remain open because you plan to fix it in a currently maintained version, simply change the 'version' to a later Fedora version. Thank you for reporting this issue and we are sorry that we were not able to fix it before Fedora 23 is end of life. If you would still like to see this bug fixed and are able to reproduce it against a later version of Fedora, you are encouraged change the 'version' to a later Fedora version prior this bug is closed as described in the policy above. Although we aim to fix as many bugs as possible during every release's lifetime, sometimes those efforts are overtaken by events. Often a more recent Fedora release includes newer upstream software that fixes bugs or makes them obsolete.
Fedora 23 changed to end-of-life (EOL) status on 2016-12-20. Fedora 23 is no longer maintained, which means that it will not receive any further security or bug fix updates. As a result we are closing this bug. If you can reproduce this bug against a currently maintained version of Fedora please feel free to reopen this bug against that version. If you are unable to reopen this bug, please file a new report against the current release. If you experience problems, please add a comment to this bug. Thank you for reporting this bug and we are sorry it could not be fixed.