ClickHouse源码阅读环境之Ubuntu编译，主要介绍如何在Ubuntu 17版本成功构建ClickHouse，生成可部署的二进制文件。

基于Ubuntu 17编译ClickHouse

获取Ubuntu 17.10版本，并且运行此版本镜像，把宿主机/data/gitlab/jdp目录挂载到容器/opt目录。 通过docker exec命令进入容器，执行相关编译操作。

1
2
3
docker run -itd -v /data/gitlab/jdp:/opt --workdir /opt ubuntu:17.10 /bin/bash

docker exec -it angry_edison /bin/bash

接下来就是容器内的操作，为了方便，一般我在我的Ubuntu和macos机器上都是通过docker来跑各种系统和测试的，基本当做虚拟机用，关于docker知识查阅资料吧。

1
2
3
4
5
# cat /etc/issue
Ubuntu 17.10

# uname -ar
Linux 88fb00182673 4.9.60-linuxkit-aufs #1 SMP Mon Nov 6 16:00:12 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux

在Ubuntu 17.10容器中安装相关编译环境软件包。

1
2
3
apt update -y 

apt install -y cmake libssl-dev libcrypto++-dev libglib2.0-dev libltdl-dev libicu-dev libmysql++-dev libreadline-dev libmysqlclient-dev unixodbc-dev gcc-7 g++-7 unixodbc-dev devscripts dupload fakeroot debhelper liblld-5.0-dev libclang-5.0-dev liblld-5.0

进入宿主机映射容器目录，为了持久化保存编译目录数据。

1
2
3
4
cd /opt
git clone -b stable --recursive https://github.com/yandex/ClickHouse.git

cd ClickHouse

开始编译ClickHouse

1
2
3
4
5
6
7
mkdir -p build

cd build

cmake .. -DENABLE_EMBEDDED_COMPILER=1 -DENABLE_TESTS=0

make -j $(nproc || grep -c ^processor /proc/cpuinfo)

编译成功 - 尾部信息

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
[100%] Linking CXX static library libclickhouse-benchmark-lib.a
[100%] Built target clickhouse-benchmark-lib
Scanning dependencies of target clickhouse-local-lib
[100%] Building CXX object dbms/src/Server/CMakeFiles/clickhouse-local-lib.dir/LocalServer.cpp.o
[100%] Linking CXX static library libclickhouse-copier-lib.a
[100%] Built target clickhouse-copier-lib
[100%] Linking CXX static library libclickhouse-local-lib.a
[100%] Built target clickhouse-local-lib
Scanning dependencies of target clickhouse
[100%] Building CXX object dbms/src/Server/CMakeFiles/clickhouse.dir/main.cpp.o
[100%] Linking CXX executable clickhouse
[100%] Built target clickhouse
Scanning dependencies of target clickhouse-lld
Scanning dependencies of target clickhouse-extract-from-config
[100%] Built target clickhouse-lld
[100%] Built target clickhouse-extract-from-config
Scanning dependencies of target clickhouse-clang
Scanning dependencies of target clickhouse-copier
[100%] Built target clickhouse-copier
[100%] Built target clickhouse-clang
Scanning dependencies of target clickhouse-format
Scanning dependencies of target clickhouse-compressor
[100%] Built target clickhouse-format
[100%] Built target clickhouse-compressor
Scanning dependencies of target clickhouse-benchmark
Scanning dependencies of target clickhouse-server
[100%] Built target clickhouse-benchmark
[100%] Built target clickhouse-server
Scanning dependencies of target clickhouse-client
Scanning dependencies of target clickhouse-local
[100%] Built target clickhouse-client
[100%] Built target clickhouse-local
Scanning dependencies of target clickhouse-performance-test
[100%] Built target clickhouse-performance-test
Scanning dependencies of target clickhouse-bundle
[100%] Built target clickhouse-bundle

为了便于使用，我把上述编译环境变成镜像已经上传到jikelab仓库，你可以通过如下命令获取。

1
2
3
4
5
docker pull jikelab/ubuntu-17.10-clickhouse:v1.1.54370-stable

docker run -itd -v /data/gitlab/jdp:/opt --workdir /opt jikelab/ubuntu-17.10-clickhouse:v1.1.54370-stable /bin/bash

docker exec -it 容器ID /bin/bash   # 进入容器后，clone源码，即可开始building...

我是如何制作此镜像呢？

Examples:

Push一个新的image到公共镜像仓库。

[1] 通过正在运行的容器ID快速保存成新的镜像。

1
$ docker commit c16378f943fe ubuntu-17.10-clickhouse:v1.1.54370-stable

[2] 现在通过image ID Push镜像到公共镜像仓库。

1
2
3
$ docker tag ubuntu-17.10-clickhouse:v1.1.54370-stable jikelab/ubuntu-17.10-clickhouse:v1.1.54370-stable

$ docker push jikelab/ubuntu-17.10-clickhouse:v1.1.54370-stable

[3] 通过运行如下命令检查镜像生成是否生效。

1
$ docker images

你应该能看到ubuntu-17.10-clickhouse:v1.1.54370-stable和jikelab/ubuntu-17.10-clickhouse:v1.1.54370-stable镜像列表。

关于ClickHouse源码阅读环境之Centos编译

正在整理相关编译环境和文档，会通过自动化的持续集成平台发布，可关注镜像仓库。

关于构建deb包

类似rm -f ../clickhouse.deb && ./release && ls -l ../clickhouse.deb命令，然后，愉快的部署和调试ClickHouse。

1
2
3
4
5
6
7
export CMAKE=cmake
export PREFIX=/data

DAEMONS="clickhouse clickhouse-test clickhouse-compressor clickhouse-client clickhouse-server"
for daemon in $DAEMONS; do \
        DESTDIR=$PREFIX $CMAKE -DCOMPONENT=$daemon -P cmake_install.cmake; \
done

小试一下CK

1
clickhouse server --config=/data/usr/local/etc/clickhouse-server/config.xml

启动报错

1
2
Logging trace to console
Poco::Exception. Code: 1000, e.code() = 0, e.displayText() = Exception: Could not determine local time zone: boost::filesystem::canonical: No such file or directory: "/usr/share/zoneinfo/", e.what() = Exception

解决

1
2
3
apt-get install --reinstall tzdata

dpkg-reconfigure tzdata   //根据提示选择时区，因为我是在docker编译和运行测试

clickhouse CLI

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
clickhouse-client -h 127.0.0.1 -d default

b5c8238c1618 :) select count(*) from system.clusters;

SELECT count(*)
FROM system.clusters

┌─count()─┐
│       2 │
└─────────┘

1 rows in set. Elapsed: 0.007 sec.


b5c8238c1618 :) use default;

Table测试

1
2
3
4
5
6
7
8
9
10
CREATE TABLE ontime_local (FlightDate Date,Year UInt16) ENGINE = MergeTree(FlightDate, (Year, FlightDate), 8192);


insert into ontime_local (FlightDate,Year)values('2001-10-12',2001);

insert into ontime_local (FlightDate,Year)values('2002-10-12',2002);

insert into ontime_local (FlightDate,Year)values('2003-10-12',2003);

select count(*) from ontime_all;

接下来，就愉快的进行代码开发，debug吧。

小结

介绍clickhouse在Ubuntu中进行源码编译和涉及到的Docker容器相关技术，对于想深入源码研究clickhouse的人来说，此步骤后就可以愉快的阅读代码，调试和改代码啦，简单看了一下clickhouse的代码量还是相当惊人的，相当于2个大型的C++项目，不得不佩服开发者，是在短短几年内完成的工作，而且还能保证测试覆盖率和高性能，那是相当厉害。

参考：

[1] https://docs.docker.com/engine/reference/commandline/push/#examples
[2] https://clickhouse.yandex/docs/en/development/build/