Hide Forgot
Description of problem: Parallel build of webkitgtk4 fails with cmake 3.2. I wonder if it's a WebKit bug or a CMake bug. Version-Release number of selected component (if applicable): cmake-3.2.0-0.1.rc1.fc22 How reproducible: ...eh... always? Steps to Reproduce: 1. Revert the latest commit to webkitgtk4 which disables parallel build 2. fedpkg build webkitgtk4 Actual results: https://kojipkgs.fedoraproject.org//work/tasks/2915/9132915/build.log The interesting part is this: [ 15%] cd /builddir/build/BUILD/webkitgtk-2.7.91/x86_64-redhat-linux-gnu/DerivedSources/WebCore && /usr/bin/perl -I/builddir/build/BUILD/webkitgtk-2.7.91/Source/WebCore/bindings/scripts /builddir/build/BUILD/webkitgtk-2.7.91/Source/WebCore/css/makeprop.pl --defines " ENABLE_3D_RENDERING ENABLE_ACCELERATED_2D_CANVAS ENABLE_ACCESSIBILITY ENABLE_CHANNEL_MESSAGING ENABLE_CONTEXT_MENUS ENABLE_CSS_BOX_DECORATION_BREAK ENABLE_CSS_IMAGE_SET ENABLE_CSS_REGIONS ENABLE_CSS_SELECTORS_LEVEL4 ENABLE_DETAILS_ELEMENT ENABLE_DRAG_SUPPORT ENABLE_FULLSCREEN_API ENABLE_GEOLOCATION ENABLE_ICONDATABASE ENABLE_INPUT_TYPE_COLOR ENABLE_JIT ENABLE_LEGACY_VENDOR_PREFIXES ENABLE_MATHML ENABLE_MEDIA_CONTROLS_SCRIPT ENABLE_MEMORY_SAMPLER ENABLE_METER_ELEMENT ENABLE_MHTML ENABLE_NAVIGATOR_HWCONCURRENCY ENABLE_NETSCAPE_PLUGIN_API ENABLE_NETWORK_PROCESS ENABLE_NOTIFICATIONS ENABLE_PICTURE_SIZES ENABLE_PROMISES ENABLE_REQUEST_ANIMATION_FRAME ENABLE_SATURATED_LAYOUT_ARITHMETIC ENABLE_SPELLCHECK ENABLE_STREAMS_API ENABLE_SVG_FONTS ENABLE_TEMPLATE_ELEMENT ENABLE_TOUCH_EVENTS ENABLE_USERSELECT_ALL ENABLE_USER_MESSAGE_HANDLERS ENABLE_VIDEO ENABLE_VIDEO_TRACK ENABLE_VIEW_MODE_CSS_MEDIA ENABLE_WEB_AUDIO ENABLE_WEB_SOCKETS ENABLE_WEB_TIMING ENABLE_WEBGL ENABLE_XHR_TIMEOUT ENABLE_XSLT ENABLE_PLUGIN_PROCESS_GTK2" --preprocessor "/usr/bin/c++ -E -P -x c++" /usr/bin/cmake -E cmake_progress_report /builddir/build/BUILD/webkitgtk-2.7.91/x86_64-redhat-linux-gnu/CMakeFiles /usr/bin/cmake -E cmake_progress_report /builddir/build/BUILD/webkitgtk-2.7.91/x86_64-redhat-linux-gnu/CMakeFiles /usr/bin/cmake -E cmake_progress_report /builddir/build/BUILD/webkitgtk-2.7.91/x86_64-redhat-linux-gnu/CMakeFiles /usr/bin/cmake -E cmake_progress_report /builddir/build/BUILD/webkitgtk-2.7.91/x86_64-redhat-linux-gnu/CMakeFiles /usr/bin/cmake -E cmake_progress_report /builddir/build/BUILD/webkitgtk-2.7.91/x86_64-redhat-linux-gnu/CMakeFiles /usr/bin/cmake -E cmake_progress_report /builddir/build/BUILD/webkitgtk-2.7.91/x86_64-redhat-linux-gnu/CMakeFiles [ 15%] [ 15%] [ 15%] [ 15%] [ 15%] Generating ../../DerivedSources/WebCore/SelectorPseudoElementTypeMap.gperf, ../../DerivedSources/WebCore/SelectorPseudoElementTypeMap.cpp [ 15%] CSSPropertyNames.gperf: The input file is empty! calling gperf failed: 256 at /builddir/build/BUILD/webkitgtk-2.7.91/Source/WebCore/css/makeprop.pl line 1032. Source/WebCore/CMakeFiles/WebCore.dir/build.make:155: recipe for target 'DerivedSources/WebCore/CSSPropertyNames.h' failed make[2]: *** [DerivedSources/WebCore/CSSPropertyNames.h] Error 25 make[2]: *** Waiting for unfinished jobs... Line 1032 of makeprop.pl: system("\"$gperf\" --key-positions=\"*\" -D -n -s 2 CSSPropertyNames.gperf --output-file=CSSPropertyNames.cpp") == 0 || die "calling gperf failed: $?"; So I guess CSSPropertyNames.gperf has not been built yet. Oh, but that file is built by makeprop.pl, beginning with line 113: open GPERF, ">CSSPropertyNames.gperf" || die "Could not open CSSPropertyNames.gperf for writing"; So it doesn't SEEM like a CMake dependency issue... but the script is a mess to read, and I can imagine it not working if CSSPropertyNames.in was not properly generated. Now, in WebKit/Source/WebCore/CMakeLists.txt we have this: # Generate CSS property names add_custom_command( OUTPUT ${DERIVED_SOURCES_WEBCORE_DIR}/CSSPropertyNames.in ${DERIVED_SOURCES_WEBCORE_DIR}/CSSPropertyNames.h ${DERIVED_SOURCES_WEBCORE_DIR}/CSSPropertyNames.cpp ${DERIVED_SOURCES_WEBCORE_DIR}/CSSPropertyNames.gperf ${DERIVED_SOURCES_WEBCORE_DIR}/StyleBuilder.cpp ${DERIVED_SOURCES_WEBCORE_DIR}/StylePropertyShorthandFunctions.h ${DERIVED_SOURCES_WEBCORE_DIR}/StylePropertyShorthandFunctions.cpp MAIN_DEPENDENCY ${WEBCORE_DIR}/css/makeprop.pl DEPENDS ${WebCore_CSS_PROPERTY_NAMES} WORKING_DIRECTORY ${DERIVED_SOURCES_WEBCORE_DIR} COMMAND ${PERL_EXECUTABLE} -ne "print" ${WebCore_CSS_PROPERTY_NAMES} > ${DERIVED_SOURCES_WEBCORE_DIR}/CSSPropertyNames.in COMMAND ${PERL_EXECUTABLE} -I${WEBCORE_DIR}/bindings/scripts ${WEBCORE_DIR}/css/makeprop.pl --defines "${FEATURE_DEFINES_WITH_SPACE_SEPARATOR}" --preprocessor "${CODE_GENERATOR_PREPROCESSOR}" VERBATIM) list(APPEND WebCore_SOURCES ${DERIVED_SOURCES_WEBCORE_DIR}/CSSPropertyNames.cpp) list(APPEND WebCore_SOURCES ${DERIVED_SOURCES_WEBCORE_DIR}/StyleBuilder.cpp) list(APPEND WebCore_SOURCES ${DERIVED_SOURCES_WEBCORE_DIR}/StylePropertyShorthandFunctions.cpp) ADD_SOURCE_WEBCORE_DERIVED_DEPENDENCIES(${WEBCORE_DIR}/css/CSSParser.cpp CSSValueKeywords.h) ADD_SOURCE_WEBCORE_DERIVED_DEPENDENCIES(${DERIVED_SOURCES_WEBCORE_DIR}/CSSGrammar.cpp CSSPropertyNames.h) Simple, right? :p And also: set(WebCore_CSS_PROPERTY_NAMES ${WEBCORE_DIR}/css/CSSPropertyNames.in ) ...looks right? At least, I don't notice anything wrong there. But I do notice higher up in the build log: [ 15%] [ 15%] [ 15%] Generating ../../DerivedSources/WebCore/CSSPropertyNames.in, ../../DerivedSources/WebCore/CSSPropertyNames.h, ../../DerivedSources/WebCore/CSSPropertyNames.cpp, ../../DerivedSources/WebCore/CSSPropertyNames.gperf, ../../DerivedSources/WebCore/StyleBuilder.cpp, ../../DerivedSources/WebCore/StylePropertyShorthandFunctions.h, ../../DerivedSources/WebCore/StylePropertyShorthandFunctions.cpp cd /builddir/build/BUILD/webkitgtk-2.7.91/x86_64-redhat-linux-gnu/DerivedSources/WebCore && /usr/bin/perl -ne print /builddir/build/BUILD/webkitgtk-2.7.91/Source/WebCore/css/CSSPropertyNames.in > /builddir/build/BUILD/webkitgtk-2.7.91/x86_64-redhat-linux-gnu/DerivedSources/WebCore/CSSPropertyNames.in Generating ../../DerivedSources/WebCore/CSSPropertyNames.in, ../../DerivedSources/WebCore/CSSPropertyNames.h, ../../DerivedSources/WebCore/CSSPropertyNames.cpp, ../../DerivedSources/WebCore/CSSPropertyNames.gperf, ../../DerivedSources/WebCore/StyleBuilder.cpp, ../../DerivedSources/WebCore/StylePropertyShorthandFunctions.h, ../../DerivedSources/WebCore/StylePropertyShorthandFunctions.cpp cd /builddir/build/BUILD/webkitgtk-2.7.91/x86_64-redhat-linux-gnu/DerivedSources/WebCore && /usr/bin/perl -ne print /builddir/build/BUILD/webkitgtk-2.7.91/Source/WebCore/css/CSSPropertyNames.in > /builddir/build/BUILD/webkitgtk-2.7.91/x86_64-redhat-linux-gnu/DerivedSources/WebCore/CSSPropertyNames.in Generating ../../DerivedSources/WebCore/CSSPropertyNames.in, ../../DerivedSources/WebCore/CSSPropertyNames.h, ../../DerivedSources/WebCore/CSSPropertyNames.cpp, ../../DerivedSources/WebCore/CSSPropertyNames.gperf, ../../DerivedSources/WebCore/StyleBuilder.cpp, ../../DerivedSources/WebCore/StylePropertyShorthandFunctions.h, ../../DerivedSources/WebCore/StylePropertyShorthandFunctions.cpp cd /builddir/build/BUILD/webkitgtk-2.7.91/x86_64-redhat-linux-gnu/DerivedSources/WebCore && /usr/bin/perl -ne print /builddir/build/BUILD/webkitgtk-2.7.91/Source/WebCore/css/CSSPropertyNames.in > /builddir/build/BUILD/webkitgtk-2.7.91/x86_64-redhat-linux-gnu/DerivedSources/WebCore/CSSPropertyNames.in /usr/bin/cmake -E cmake_progress_report /builddir/build/BUILD/webkitgtk-2.7.91/x86_64-redhat-linux-gnu/CMakeFiles Generating ../../DerivedSources/WebCore/CSSPropertyNames.in, ../../DerivedSources/WebCore/CSSPropertyNames.h, ../../DerivedSources/WebCore/CSSPropertyNames.cpp, ../../DerivedSources/WebCore/CSSPropertyNames.gperf, ../../DerivedSources/WebCore/StyleBuilder.cpp, ../../DerivedSources/WebCore/StylePropertyShorthandFunctions.h, ../../DerivedSources/WebCore/StylePropertyShorthandFunctions.cpp /usr/bin/cmake -E cmake_progress_report /builddir/build/BUILD/webkitgtk-2.7.91/x86_64-redhat-linux-gnu/CMakeFiles cd /builddir/build/BUILD/webkitgtk-2.7.91/x86_64-redhat-linux-gnu/DerivedSources/WebCore && /usr/bin/perl -ne print /builddir/build/BUILD/webkitgtk-2.7.91/Source/WebCore/css/CSSPropertyNames.in > /builddir/build/BUILD/webkitgtk-2.7.91/x86_64-redhat-linux-gnu/DerivedSources/WebCore/CSSPropertyNames.in Um, that command is run four times instead of once! And that is as far as I've gotten, but I guess that must be a CMake bug.
I've heard of this before (but have not seen it myself). The fix before was to not use the same MAIN_DEPENDENCY for multiple add_custom_commands. Looking at the source tree of WebKit though doesn't seem to indicate that makeprop.pl is being used in that way though. Is it possible to get the debug output from make for why it is running the command? Also, how does the Ninja generator fare?
Bug reported upstream: http://www.cmake.org/Bug/view.php?id=15434
Thanks for your help Ben! As noted in your upstream bug, it's probably the same issue as http://www.cmake.org/Bug/view.php?id=15116#c38168 which was hopefully fixed earlier today, so I'd rather see if that works before spending more time on this.
Well, if you get a chance, it might be worth trying the Ninja generator to help get build times down as well (I'd also be interested in a %cmake_ninja macro for use in specfiles). You might need to set some pool limits to keep the memory usage down during link steps though. See: http://www.cmake.org/cmake/help/v3.1/prop_gbl/JOB_POOLS.html
Can we have cmake updated to 3.2.1, will fix this bug (hopefully): http://www.kitware.com/blog/home/post/862
Building - http://koji.fedoraproject.org/koji/taskinfo?taskID=9205195
Thanks Orion!
By the way, I've been using the Ninja generator pretty much exclusively for local development of WebKitGTK+ since last summer. It works great.
(In reply to Orion Poplawski from comment #6) > Building - http://koji.fedoraproject.org/koji/taskinfo?taskID=9205195 Orion, can you please build it for F22 as well?
cmake-3.2.1-1.fc22 has been submitted as an update for Fedora 22. https://admin.fedoraproject.org/updates/cmake-3.2.1-1.fc22
Package cmake-3.2.1-1.fc22: * should fix your issue, * was pushed to the Fedora 22 testing repository, * should be available at your local mirror within two days. Update it with: # su -c 'yum update --enablerepo=updates-testing cmake-3.2.1-1.fc22' as soon as you are able to. Please go to the following url: https://admin.fedoraproject.org/updates/FEDORA-2015-3892/cmake-3.2.1-1.fc22 then log in and leave karma (feedback).
cmake-3.2.1-1.fc22 has been pushed to the Fedora 22 stable repository. If problems still persist, please make note of it in this bug report.