Bug 1532391

This appears to be race condition:

~~~
+ drake -j4 V=1 --trace

... snip

** Invoke apps:tools:ebml_validator 
g++ -Wall -Wno-comment -Wfatal-errors -D_FILE_OFFSET_BITS=64 -DMTX_LOCALE_DIR=\"/usr/share/locale\" -DMTX_PKG_DATA_DIR=\"/usr/share/mkvtoolnix\" -DMTX_DOC_DIR=\"/usr/share/doc/mkvtoolnix\" -fstack-protector-strong -fPIC -std=gnu++14 -Wnon-virtual-dtor -Woverloaded-virtual -Wextra -Wno-missing-field-initializers -Wno-maybe-uninitialized -I/usr/include/qt5/QtCore -I/usr/include/qt5 -I/usr/include/qt5/QtGui -I/usr/include/qt5/QtWidgets -I/usr/include/qt5/QtMultimedia -I/usr/include/qt5/QtNetwork -I/usr/include/qt5/QtConcurrent -I/usr/include -O2 -g -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -m64 -mtune=generic -I. -Ilib -Ilib/avilib-0.6.10 -Isrc -c -MMD -MF ./rake.d/dependency.d/tmp/src_common_common_pch_h_gch.d -o src/common/common_pch.h.gch -x c++-header src/common/common_pch.h
g++ -Wall -Wno-comment -Wfatal-errors -D_FILE_OFFSET_BITS=64 -DMTX_LOCALE_DIR=\"/usr/share/locale\" -DMTX_PKG_DATA_DIR=\"/usr/share/mkvtoolnix\" -DMTX_DOC_DIR=\"/usr/share/doc/mkvtoolnix\" -fstack-protector-strong -fPIC -std=gnu++14 -Wnon-virtual-dtor -Woverloaded-virtual -Wextra -Wno-missing-field-initializers -Wno-maybe-uninitialized -I/usr/include/qt5/QtCore -I/usr/include/qt5 -I/usr/include/qt5/QtGui -I/usr/include/qt5/QtWidgets -I/usr/include/qt5/QtMultimedia -I/usr/include/qt5/QtNetwork -I/usr/include/qt5/QtConcurrent -I/usr/include -O2 -g -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -m64 -mtune=generic -I. -Ilib -Ilib/avilib-0.6.10 -Isrc -c -MMD -MF ./rake.d/dependency.d/tmp/src_common_terminal_o.d -o src/common/terminal.o -x c++ src/common/terminal.cpp
g++ -Wall -Wno-comment -Wfatal-errors -D_FILE_OFFSET_BITS=64 -DMTX_LOCALE_DIR=\"/usr/share/locale\" -DMTX_PKG_DATA_DIR=\"/usr/share/mkvtoolnix\" -DMTX_DOC_DIR=\"/usr/share/doc/mkvtoolnix\" -fstack-protector-strong -fPIC -std=gnu++14 -Wnon-virtual-dtor -Woverloaded-virtual -Wextra -Wno-missing-field-initializers -Wno-maybe-uninitialized -I/usr/include/qt5/QtCore -I/usr/include/qt5 -I/usr/include/qt5/QtGui -I/usr/include/qt5/QtWidgets -I/usr/include/qt5/QtMultimedia -I/usr/include/qt5/QtNetwork -I/usr/include/qt5/QtConcurrent -I/usr/include -O2 -g -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -m64 -mtune=generic -I. -Ilib -Ilib/avilib-0.6.10 -Isrc -c -MMD -MF ./rake.d/dependency.d/tmp/src_common_utf8_codecvt_facet_o.d -o src/common/utf8_codecvt_facet.o -x c++ src/common/utf8_codecvt_facet.cpp
** Execute src/common/utf8_codecvt_facet.o
** Execute src/common/win_itaskbarlist3.o
** Execute src/common/common_pch.h.gch
** Execute src/common/terminal.o
rake aborted!
File exists @ dir_s_mkdir - ./rake.d/dependency.d
/builddir/build/BUILD/mkvtoolnix-19.0.0/rake.d/helpers.rb:95:in `mkdir'
/builddir/build/BUILD/mkvtoolnix-19.0.0/rake.d/helpers.rb:95:in `ensure_dir'
/builddir/build/BUILD/mkvtoolnix-19.0.0/rake.d/helpers.rb:99:in `block in create_dependency_dirs'
/builddir/build/BUILD/mkvtoolnix-19.0.0/rake.d/helpers.rb:99:in `each'
/builddir/build/BUILD/mkvtoolnix-19.0.0/rake.d/helpers.rb:99:in `create_dependency_dirs'
/builddir/build/BUILD/mkvtoolnix-19.0.0/Rakefile:261:in `block in <top (required)>'
/usr/share/gems/gems/drake-0.9.2.0.3.1/lib/rake/task.rb:229:in `block in execute'
/usr/share/gems/gems/drake-0.9.2.0.3.1/lib/rake/task.rb:224:in `each'
/usr/share/gems/gems/drake-0.9.2.0.3.1/lib/rake/task.rb:224:in `execute'
/usr/share/gems/gems/drake-0.9.2.0.3.1/lib/rake/parallel.rb:76:in `block (3 levels) in compute'
/usr/share/gems/gems/comp_tree-1.1.3/lib/comp_tree/node.rb:89:in `compute'
/usr/share/gems/gems/comp_tree-1.1.3/lib/comp_tree/algorithm.rb:25:in `block in new_worker'
~~~

So there are 4 tasks running in parallel, and they are trying to execute this method:

~~~
def ensure_dir dir
  File.unlink(dir) if  FileTest.exist?(dir) && !FileTest.directory?(dir)
  Dir.mkdir(dir)   if !FileTest.exist?(dir)
end
~~~

Although there are 'exists?' checks, if the directory is created in parallel running task meanwhile, then the re-creation must fail.

Not really sure what is the best option here. For you, the simplest option is either not running in parallel (i.e. drop the %{?_smp_mflags} or user rake instead of rake) or pre-create the directories:

~~~
diff --git a/mkvtoolnix.spec b/mkvtoolnix.spec
index 4a9eec4..5c8bddf 100644
--- a/mkvtoolnix.spec
+++ b/mkvtoolnix.spec
@@ -70,6 +70,9 @@ rm -rf rake.d/vendor drake
   --with-boost-libdir=%{_libdir} \
   --with-tools \
   || cat config.log
+
+mkdir -p rake.d/dependency.d/tmp
+
 drake %{?_smp_mflags} V=1
 
 %install
~~~

And for upstream? Simply make the ensure_dir thread safe. May be something like:

~~~
sed -i 's/Dir\.mkdir.*$/FileUtils.mkdir_p(dir)/' rake.d/helpers.rb
~~~

could be enough, but not sure how safe/atomic the FileUtils.mkdir_p is.

Thanks for the analysis. I'll forward it upstream. Any idea why it always worked before Ruby 2.5 update?

(In reply to Dominik 'Rathann' Mierzejewski from comment #2)
> Any idea why it always worked before Ruby 2.5 update?

I knew you are going to ask this question ;) Short answer could be "not really", but I think that Ruby usptream is trying to make Ruby more concurrent/parallel. So if they can, the remove the GIL from various places of Ruby.

Actually, I read the upstream ticket and due to my curiosity, I found this upstream commit which is likely the root cause of this issue:

https://github.com/ruby/ruby/commit/f5f6232399b1e4bf6b702b51bef4d9e36b42df1b

Sounds plausible. Thanks!

Also, upstream has fixed that already, so the fix will be present in the next release.

Fixed across all Fedora branches, but really relevant only for rawhide.