Bug 1532391 - mkvtoolnix: FTBFS in Fedora rawhide
Summary: mkvtoolnix: FTBFS in Fedora rawhide
Keywords:
Status: CLOSED RAWHIDE
Alias: None
Product: Fedora
Classification: Fedora
Component: mkvtoolnix
Version: rawhide
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: ---
Assignee: Dominik 'Rathann' Mierzejewski
QA Contact: Fedora Extras Quality Assurance
URL: http://apps.fedoraproject.org/koschei...
Whiteboard:
Depends On:
Blocks: 1515518
TreeView+ depends on / blocked
 
Reported: 2018-01-08 21:39 UTC by Dominik 'Rathann' Mierzejewski
Modified: 2018-01-19 08:11 UTC (History)
6 users (show)

See Also:
Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2018-01-19 08:11:59 UTC
Type: Bug
Embargoed:


Attachments (Terms of Use)

Description Dominik 'Rathann' Mierzejewski 2018-01-08 21:39:45 UTC
Description of problem:
Package mkvtoolnix fails to build from source in Fedora rawhide.

Version-Release number of selected component (if applicable):
19.0.0-1.fc28

Steps to Reproduce:
koji build --scratch f28 mkvtoolnix-19.0.0-1.fc28.src.rpm

Additional info:
This package is tracked by Koschei. See:
http://apps.fedoraproject.org/koschei/package/mkvtoolnix

It looks like this was caused by https://fedoraproject.org/wiki/Changes/Ruby_2.5.

The following can be seen in build log:
[...]
rake aborted!
File exists @ dir_s_mkdir - ./rake.d/dependency.d

Comment 1 Vít Ondruch 2018-01-10 17:59:56 UTC
This appears to be race condition:

~~~
+ drake -j4 V=1 --trace

... snip

** Invoke apps:tools:ebml_validator 
g++ -Wall -Wno-comment -Wfatal-errors -D_FILE_OFFSET_BITS=64 -DMTX_LOCALE_DIR=\"/usr/share/locale\" -DMTX_PKG_DATA_DIR=\"/usr/share/mkvtoolnix\" -DMTX_DOC_DIR=\"/usr/share/doc/mkvtoolnix\" -fstack-protector-strong -fPIC -std=gnu++14 -Wnon-virtual-dtor -Woverloaded-virtual -Wextra -Wno-missing-field-initializers -Wno-maybe-uninitialized -I/usr/include/qt5/QtCore -I/usr/include/qt5 -I/usr/include/qt5/QtGui -I/usr/include/qt5/QtWidgets -I/usr/include/qt5/QtMultimedia -I/usr/include/qt5/QtNetwork -I/usr/include/qt5/QtConcurrent -I/usr/include -O2 -g -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -m64 -mtune=generic -I. -Ilib -Ilib/avilib-0.6.10 -Isrc -c -MMD -MF ./rake.d/dependency.d/tmp/src_common_common_pch_h_gch.d -o src/common/common_pch.h.gch -x c++-header src/common/common_pch.h
g++ -Wall -Wno-comment -Wfatal-errors -D_FILE_OFFSET_BITS=64 -DMTX_LOCALE_DIR=\"/usr/share/locale\" -DMTX_PKG_DATA_DIR=\"/usr/share/mkvtoolnix\" -DMTX_DOC_DIR=\"/usr/share/doc/mkvtoolnix\" -fstack-protector-strong -fPIC -std=gnu++14 -Wnon-virtual-dtor -Woverloaded-virtual -Wextra -Wno-missing-field-initializers -Wno-maybe-uninitialized -I/usr/include/qt5/QtCore -I/usr/include/qt5 -I/usr/include/qt5/QtGui -I/usr/include/qt5/QtWidgets -I/usr/include/qt5/QtMultimedia -I/usr/include/qt5/QtNetwork -I/usr/include/qt5/QtConcurrent -I/usr/include -O2 -g -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -m64 -mtune=generic -I. -Ilib -Ilib/avilib-0.6.10 -Isrc -c -MMD -MF ./rake.d/dependency.d/tmp/src_common_terminal_o.d -o src/common/terminal.o -x c++ src/common/terminal.cpp
g++ -Wall -Wno-comment -Wfatal-errors -D_FILE_OFFSET_BITS=64 -DMTX_LOCALE_DIR=\"/usr/share/locale\" -DMTX_PKG_DATA_DIR=\"/usr/share/mkvtoolnix\" -DMTX_DOC_DIR=\"/usr/share/doc/mkvtoolnix\" -fstack-protector-strong -fPIC -std=gnu++14 -Wnon-virtual-dtor -Woverloaded-virtual -Wextra -Wno-missing-field-initializers -Wno-maybe-uninitialized -I/usr/include/qt5/QtCore -I/usr/include/qt5 -I/usr/include/qt5/QtGui -I/usr/include/qt5/QtWidgets -I/usr/include/qt5/QtMultimedia -I/usr/include/qt5/QtNetwork -I/usr/include/qt5/QtConcurrent -I/usr/include -O2 -g -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -m64 -mtune=generic -I. -Ilib -Ilib/avilib-0.6.10 -Isrc -c -MMD -MF ./rake.d/dependency.d/tmp/src_common_utf8_codecvt_facet_o.d -o src/common/utf8_codecvt_facet.o -x c++ src/common/utf8_codecvt_facet.cpp
** Execute src/common/utf8_codecvt_facet.o
** Execute src/common/win_itaskbarlist3.o
** Execute src/common/common_pch.h.gch
** Execute src/common/terminal.o
rake aborted!
File exists @ dir_s_mkdir - ./rake.d/dependency.d
/builddir/build/BUILD/mkvtoolnix-19.0.0/rake.d/helpers.rb:95:in `mkdir'
/builddir/build/BUILD/mkvtoolnix-19.0.0/rake.d/helpers.rb:95:in `ensure_dir'
/builddir/build/BUILD/mkvtoolnix-19.0.0/rake.d/helpers.rb:99:in `block in create_dependency_dirs'
/builddir/build/BUILD/mkvtoolnix-19.0.0/rake.d/helpers.rb:99:in `each'
/builddir/build/BUILD/mkvtoolnix-19.0.0/rake.d/helpers.rb:99:in `create_dependency_dirs'
/builddir/build/BUILD/mkvtoolnix-19.0.0/Rakefile:261:in `block in <top (required)>'
/usr/share/gems/gems/drake-0.9.2.0.3.1/lib/rake/task.rb:229:in `block in execute'
/usr/share/gems/gems/drake-0.9.2.0.3.1/lib/rake/task.rb:224:in `each'
/usr/share/gems/gems/drake-0.9.2.0.3.1/lib/rake/task.rb:224:in `execute'
/usr/share/gems/gems/drake-0.9.2.0.3.1/lib/rake/parallel.rb:76:in `block (3 levels) in compute'
/usr/share/gems/gems/comp_tree-1.1.3/lib/comp_tree/node.rb:89:in `compute'
/usr/share/gems/gems/comp_tree-1.1.3/lib/comp_tree/algorithm.rb:25:in `block in new_worker'
~~~

So there are 4 tasks running in parallel, and they are trying to execute this method:

~~~
def ensure_dir dir
  File.unlink(dir) if  FileTest.exist?(dir) && !FileTest.directory?(dir)
  Dir.mkdir(dir)   if !FileTest.exist?(dir)
end
~~~

Although there are 'exists?' checks, if the directory is created in parallel running task meanwhile, then the re-creation must fail.

Not really sure what is the best option here. For you, the simplest option is either not running in parallel (i.e. drop the %{?_smp_mflags} or user rake instead of rake) or pre-create the directories:

~~~
diff --git a/mkvtoolnix.spec b/mkvtoolnix.spec
index 4a9eec4..5c8bddf 100644
--- a/mkvtoolnix.spec
+++ b/mkvtoolnix.spec
@@ -70,6 +70,9 @@ rm -rf rake.d/vendor drake
   --with-boost-libdir=%{_libdir} \
   --with-tools \
   || cat config.log
+
+mkdir -p rake.d/dependency.d/tmp
+
 drake %{?_smp_mflags} V=1
 
 %install
~~~

And for upstream? Simply make the ensure_dir thread safe. May be something like:

~~~
sed -i 's/Dir\.mkdir.*$/FileUtils.mkdir_p(dir)/' rake.d/helpers.rb
~~~

could be enough, but not sure how safe/atomic the FileUtils.mkdir_p is.

Comment 2 Dominik 'Rathann' Mierzejewski 2018-01-10 21:31:34 UTC
Thanks for the analysis. I'll forward it upstream. Any idea why it always worked before Ruby 2.5 update?

Comment 3 Vít Ondruch 2018-01-11 09:35:17 UTC
(In reply to Dominik 'Rathann' Mierzejewski from comment #2)
> Any idea why it always worked before Ruby 2.5 update?

I knew you are going to ask this question ;) Short answer could be "not really", but I think that Ruby usptream is trying to make Ruby more concurrent/parallel. So if they can, the remove the GIL from various places of Ruby.

Actually, I read the upstream ticket and due to my curiosity, I found this upstream commit which is likely the root cause of this issue:

https://github.com/ruby/ruby/commit/f5f6232399b1e4bf6b702b51bef4d9e36b42df1b

Comment 4 Dominik 'Rathann' Mierzejewski 2018-01-11 10:03:46 UTC
Sounds plausible. Thanks!

Also, upstream has fixed that already, so the fix will be present in the next release.

Comment 5 Dominik 'Rathann' Mierzejewski 2018-01-19 08:11:59 UTC
Fixed across all Fedora branches, but really relevant only for rawhide.


Note You need to log in before you can comment on or make changes to this bug.