Bug 1526768 - Review Request: clickhouse - a distributed column-oriented DBMS
Summary: Review Request: clickhouse - a distributed column-oriented DBMS
Keywords:
Status: CLOSED NOTABUG
Alias: None
Product: Fedora
Classification: Fedora
Component: Package Review
Version: rawhide
Hardware: All
OS: Linux
medium
medium
Target Milestone: ---
Assignee: Nobody's working on this, feel free to take it
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On:
Blocks: FE-DEADREVIEW
TreeView+ depends on / blocked
 
Reported: 2017-12-17 11:39 UTC by Roman Tsisyk
Modified: 2021-03-14 19:48 UTC (History)
5 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2020-08-10 00:57:32 UTC
Type: ---
Embargoed:


Attachments (Terms of Use)
/tmp/cci1BNri.out (1.46 MB, text/x-csrc)
2019-06-16 18:42 UTC, Marius Andreiana
no flags Details

Description Roman Tsisyk 2017-12-17 11:39:54 UTC
Spec URL: https://gist.githubusercontent.com/rtsisyk/1a45e69293f3e2c91126fe04a8374af8/raw/c5758620cc9ad95772c53b3910ba18e512e432dd/clickhouse.spec
SRPM URL: https://gist.github.com/rtsisyk/1a45e69293f3e2c91126fe04a8374af8/raw/c5758620cc9ad95772c53b3910ba18e512e432dd/clickhouse-1.1.54326-1.fc26.src.rpm
Koji URL: https://koji.fedoraproject.org/koji/taskinfo?taskID=23753966
Description: ClickHouse is a distributed column-oriented DBMS
Homepage: https://clickhouse.yandex/
SCM: https://github.com/yandex/clickhouse
Fedora Account System Username: rtsisyk

ClickHouse is an open source column-oriented database management system capable of real time generation of analytical data reports using SQL queries.

 - Blazing Fast
 - Linearly Scalable
 - Hardware Efficient
 - Fault Tolerant
 - Feature Rich
 - Highly Reliable
 - Simple and Handy

ClickHouse is popular (3.3k likes on GitHub, 1.5k users in Telegram Chat) and used by multiple companies in production. I want to add add this software to Fedora Package Collection. I'm already in packager group and sponsorship is not required.



I already have access to `packager' group and don't need sponsorship.

Comment 1 Roman Tsisyk 2017-12-17 13:23:16 UTC
Minor changes:
- Fix paths in /etc/clickhouse-server/config.xml
- Fix permissions for /etc/clickhouse-server/
- Get clickhouse-server to work out of the box

Spec URL: https://gist.githubusercontent.com/rtsisyk/1a45e69293f3e2c91126fe04a8374af8/raw/1780b8b0548841957b09cabdcfc1dc8b47e0dced/clickhouse.spec
SRPM URL: https://gist.github.com/rtsisyk/1a45e69293f3e2c91126fe04a8374af8/raw/1780b8b0548841957b09cabdcfc1dc8b47e0dced/clickhouse-1.1.54326-1.fc26.src.rpm

Comment 2 Michael Cullen 2017-12-17 22:03:38 UTC
Ok... a few things I note:

* The BuildRequires are at the top - I'd expect the name block at the top.
* This bundles cctz without any comments as to why
* Have you tried unbundling boost? Although it looks for 1.60 the bundled version is 1.65 so I'd guess the version isn't actually that critical - a small patch to remove the version number would most likely be all that's required
* The CMake file seems to support ARM builds - any reason to make this exclusivearch other than the docs claiming it is?
* Groups aren't needed or particularly wanted [1]

[1] https://fedoraproject.org/wiki/RPMGroups

Comment 3 Michael Cullen 2017-12-18 02:51:59 UTC
I tried a mock build of this in a rawhide container and got this error:

CMake Error at dbms/cmake_install.cmake:61 (file):
  file INSTALL cannot copy file
  "/builddir/build/BUILD/ClickHouse-1.1.54326-testing/dbms/libclickhouse.so.1.1.54326"
  to
  "/builddir/build/BUILDROOT/clickhouse-1.1.54326-1.fc28.x86_64/usr/lib64/clickhouse/libclickhouse.so.1.1.54326".

Comment 4 Michael Cullen 2017-12-18 02:53:19 UTC
actually ignore me that's probably a disk space issue on my end!

Comment 5 Roman Tsisyk 2017-12-20 07:38:01 UTC
Hi Michael,

Thanks for the fast response.

* The BuildRequires are at the top - I'd expect the name block at the top.

BuildRequires have been moved to a proper place.

Fixed.

* This bundles cctz without any comments as to why

According to Github [1], the latest released version of google/cctz is 2.1, but
ClickHouse's scripts needs src/zone_info_source.cc which was added after 2.1 release. I added a comment about that to spec file. This package is very tricky in terms of dependencies, as you can see in the BuildRequires list.

[1] https://github.com/google/cctz/releases

* Have you tried unbundling boost? Although it looks for 1.60 the bundled version is 1.65 so I'd guess the version isn't actually that critical - a small patch to remove the version number would most likely be all that's required

Switched to boost-devel. Initially it didn't compile, but it seems that my combination of CMake flags has fixed this problem.
Fixed.

* The CMake file seems to support ARM builds - any reason to make this exclusivearch other than the docs claiming it is?

Upstream claims that "Only x86_64 with SSE 4.2 is supported. Support for AArch64 is experimental." [2]. I tried to compile aarch64 on my test host and realized that some code under #ifdef..#else..#endif is completely broken. I fixed this problem and send a pull request to upstream [2], hope they will merge it soon. Using this fix I was able to reach 99% completeness in CMake. Unfortunately, linking of C++ code requires a lot of memory and fails with OOM on my coin-size aarch64 mini-PC. 

[2] https://clickhouse.yandex/docs/en/development/build.html
[3] https://github.com/yandex/ClickHouse/pull/1674

Plan:

- Wait until pull request is accepted or include this fix as a patch
- Re-try on an aarch64 host with more RAM
- Enable or disable aarch64 in ExclusiveArch depending on result

I don't think that aarch64 supports blocks review. I'll make the best efforts to fix aarch64 both in upstream and this package. 

* Groups aren't needed or particularly wanted [1]

I removed `Group:` tag.
Fixed.

Full changelog:

    - Re-order BuildRequires
    - Add `sed` to BuildRequires
    - Add a comment about google/cctz
    - Remove `Group`
    - Use `boost-devel` instead of bundled boost
    - Enable experimental aarch64 support


Spec URL: https://gist.githubusercontent.com/rtsisyk/1a45e69293f3e2c91126fe04a8374af8/raw/f43fc9f556998507bfbbdfbee50410f1b6c8c39b/clickhouse.spec
SRPM URL: https://gist.github.com/rtsisyk/1a45e69293f3e2c91126fe04a8374af8/raw/f43fc9f556998507bfbbdfbee50410f1b6c8c39b/clickhouse-1.1.54326-1.fc26.src.rpm

Comment 7 Michael Cullen 2018-01-22 11:03:12 UTC
Looking much better in terms of comments!

You might want to consider packaging cctz from that Git release (it doesn't seem to be in Fedora at all right now) but it isn't something I'm going to moan about too much.

Just one thing about the bundling though: the guidelines [0] require any bundled libraries to be called out in the spec file as provides - something like:

Provides: bundled(cctz) = 20171031.4f9776a

should do the trick. 

[0] https://fedoraproject.org/wiki/Packaging:Guidelines#Bundling_and_Duplication_of_system_libraries
[1] https://fedoraproject.org/wiki/Packaging:Versioning#Snapshots

Comment 8 Elliott Sales de Andrade 2018-03-17 02:08:02 UTC
cctz is now packaged for Fedora 28+ [1] if you want to depend on it.

[1] https://bugzilla.redhat.com/show_bug.cgi?id=1556661

Comment 9 Marius Andreiana 2019-06-16 11:14:41 UTC
ping?

Comment 10 Marius Andreiana 2019-06-16 18:41:16 UTC
I've tried to build this on F30 

dnf -y install cctz
dnf builddep clickhouse-1.1.54330-1.fc28.src.rpm
rpmbuild --rebuild clickhouse-1.1.54330-1.fc28.src.rpm

and it gives an error:


*** WARNING *** there are active plugins, do not report this as a bug unless you can reproduce it without enabling any plugins.
Event                            | Plugins
PLUGIN_FINISH_UNIT               | annobin: Generate final annotations
PLUGIN_START_UNIT                | annobin: Generate global annotations
PLUGIN_ALL_PASSES_START          | annobin: Generate per-function annotations
PLUGIN_ALL_PASSES_END            | annobin: Register per-function end symbol
/root/rpmbuild/BUILD/ClickHouse-1.1.54330-testing/dbms/src/Storages/ColumnDefault.cpp: In function 'std::string DB::toString(DB::ColumnDefaultType)':
/root/rpmbuild/BUILD/ClickHouse-1.1.54330-testing/dbms/src/Storages/ColumnDefault.cpp:31:90: internal compiler error: in ocp_convert, at cp/cvt.c:766
   31 |     return it != std::end(map) ? it->second : throw Exception{"Invalid ColumnDefaultType"};
      |                                                                                          ^
Please submit a full bug report,
with preprocessed source if appropriate.
See <http://bugzilla.redhat.com/bugzilla> for instructions.
[ 65%] Building CXX object dbms/CMakeFiles/dbms.dir/src/Storages/StorageBuffer.cpp.o

...

Preprocessed source stored into /tmp/cci1BNri.out file, please attach this to your bugreport.
make[2]: *** [dbms/CMakeFiles/dbms.dir/build.make:3069: dbms/CMakeFiles/dbms.dir/src/Storages/ColumnDefault.cpp.o] Error 1
make[2]: *** Waiting for unfinished jobs....
In file included from /root/rpmbuild/BUILD/ClickHouse-1.1.54330-testing/dbms/src/Dictionaries/CacheDictionary.h:9,
                 from /root/rpmbuild/BUILD/ClickHouse-1.1.54330-testing/dbms/src/Storages/StorageDictionary.cpp:7:
/root/rpmbuild/BUILD/ClickHouse-1.1.54330-testing/libs/libcommon/include/ext/bit_cast.h: In instantiation of 'std::decay_t<_Tp> ext::bit_cast(const From&) [with To = std::chrono::time_point<std::chrono::_V2::system_clock, std::chrono::duration<long int, std::ratio<1, 1000000000> > >; From = long unsigned int; std::decay_t<_Tp> = std::chrono::time_point<std::chrono::_V2::system_clock, std::chrono::duration<long int, std::ratio<1, 1000000000> > >]':
/root/rpmbuild/BUILD/ClickHouse-1.1.54330-testing/libs/libcommon/include/ext/bit_cast.h:28:34:   required from 'std::decay_t<_Tp> ext::safe_bit_cast(const From&) [with To = std::chrono::time_point<std::chrono::_V2::system_clock, std::chrono::duration<long int, std::ratio<1, 1000000000> > >; From = long unsigned int; std::decay_t<_Tp> = std::chrono::time_point<std::chrono::_V2::system_clock, std::chrono::duration<long int, std::ratio<1, 1000000000> > >]'
/root/rpmbuild/BUILD/ClickHouse-1.1.54330-testing/dbms/src/Dictionaries/CacheDictionary.h:163:104:   required from here
/root/rpmbuild/BUILD/ClickHouse-1.1.54330-testing/libs/libcommon/include/ext/bit_cast.h:17:15: warning: 'void* memcpy(void*, const void*, size_t)' copying an object of non-trivial type 'struct std::chrono::time_point<std::chrono::_V2::system_clock, std::chrono::duration<long int, std::ratio<1, 1000000000> > >' from an array of 'const long unsigned int' [-Wclass-memaccess]
   17 |         memcpy(&res, &from, std::min(sizeof(res), sizeof(from)));
      |         ~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
In file included from /usr/include/c++/9/mutex:39,
                 from /root/rpmbuild/BUILD/ClickHouse-1.1.54330-testing/libs/libcommon/include/common/DateLUT.h:8,
                 from /root/rpmbuild/BUILD/ClickHouse-1.1.54330-testing/dbms/src/Common/FieldVisitors.h:5,
                 from /root/rpmbuild/BUILD/ClickHouse-1.1.54330-testing/dbms/src/Parsers/ASTSetQuery.h:4,
                 from /root/rpmbuild/BUILD/ClickHouse-1.1.54330-testing/dbms/src/Parsers/ASTCreateQuery.h:5,
                 from /root/rpmbuild/BUILD/ClickHouse-1.1.54330-testing/dbms/src/Storages/StorageDictionary.cpp:2:
/usr/include/c++/9/chrono:626:14: note: 'struct std::chrono::time_point<std::chrono::_V2::system_clock, std::chrono::duration<long int, std::ratio<1, 1000000000> > >' declared here
  626 |       struct time_point
      |              ^~~~~~~~~~
make[2]: Leaving directory '/root/rpmbuild/BUILD/ClickHouse-1.1.54330-testing'
make[1]: *** [CMakeFiles/Makefile2:1910: dbms/CMakeFiles/dbms.dir/all] Error 2
make[1]: Leaving directory '/root/rpmbuild/BUILD/ClickHouse-1.1.54330-testing'
make: *** [Makefile:133: all] Error 2
error: Bad exit status from /var/tmp/rpm-tmp.QQhrtI (%build)


RPM build errors:
    user roman does not exist - using root
    group roman does not exist - using root
    user roman does not exist - using root
    group roman does not exist - using root
    user roman does not exist - using root
    group roman does not exist - using root
    user roman does not exist - using root
    group roman does not exist - using root
    Bad exit status from /var/tmp/rpm-tmp.QQhrtI (%build)



Would also be great to upgrade to latest version.

Comment 11 Marius Andreiana 2019-06-16 18:42:34 UTC
Created attachment 1581180 [details]
/tmp/cci1BNri.out

Comment 12 Package Review 2020-07-10 00:56:22 UTC
This is an automatic check from review-stats script.

This review request ticket hasn't been updated for some time. We're sorry
it is taking so long. If you're still interested in packaging this software
into Fedora repositories, please respond to this comment clearing the
NEEDINFO flag.

You may want to update the specfile and the src.rpm to the latest version
available and to propose a review swap on Fedora devel mailing list to increase
chances to have your package reviewed. If this is your first package and you
need a sponsor, you may want to post some informal reviews. Read more at
https://fedoraproject.org/wiki/How_to_get_sponsored_into_the_packager_group.

Without any reply, this request will shortly be considered abandoned
and will be closed.
Thank you for your patience.

Comment 13 Package Review 2020-08-10 00:57:32 UTC
This is an automatic action taken by review-stats script.

The ticket submitter failed to clear the NEEDINFO flag in a month.
As per https://fedoraproject.org/wiki/Policy_for_stalled_package_reviews
we consider this ticket as DEADREVIEW and proceed to close it.

Comment 14 Kfir Itzhak 2021-03-14 12:22:31 UTC
Hi,

I rewrote an old SPEC file, and i'm using it to build ClickHouse for CentOS8 on x86 and aarch64, and currently working on getting it built on ppc64le.
I would like to become a maintainer for this package on Fedora 32+ & RHEL8+ if possible.


Kfir

Comment 15 Elliott Sales de Andrade 2021-03-14 19:48:46 UTC
This review died; you should probably just open a new one.


Note You need to log in before you can comment on or make changes to this bug.