Skip to content

Building CockroachDB

Jump to bottom
aborkar-ibm edited this page Aug 9, 2022 · 38 revisions

Building CockroachDB

The instructions provided below specify the steps to build CockroachDB version 21.2.6 on Linux on IBM Z for the following distributions:

  • Ubuntu (18.04, 20.04)
  • SLES (12 SP5, 15 SP3)
  • RHEL (7.8, 7.9, 8.4)

General Notes:

  • When following the steps below please use a standard permission user unless otherwise specified.
  • A directory /<source_root>/ will be referred to in these instructions, this is a temporary writable directory anywhere you'd like to place it.

Step 1: Build and Install CockroachDB

1.1) Build using script

If you'd like to build CockroachDB using the manual steps, please go to STEP 1.2.

Use the following commands to build CockroachDB using the build script. Please make sure you have wget installed.

wget -q https://raw.githubusercontent.com/linux-on-ibm-z/scripts/master/CockroachDB/21.2.6/build_crdb.sh

# Build CockroachDB
bash build_crdb.sh   [Provide -t option for executing build with tests]

In case of any errors, check logs for more details or go to STEP 1.2 to follow the manual build steps.

1.2) Install the Dependencies

export SOURCE_ROOT=/<source_root>/
  • Ubuntu (18.04, 20.04)
sudo apt-get update && sudo apt-get -y install autoconf automake wget make libssl-dev libncurses5-dev bison xz-utils patch g++ curl git ccache python
  • SLES 12 SP5
sudo zypper install -y libopenssl-devel autoconf automake wget ncurses-devel bison patch tar gzip cmake gawk xz python gcc7-c++ zlib-devel bzip2 curl git ccache which

sudo update-alternatives --install /usr/bin/gcc gcc /usr/bin/gcc-7 100
sudo update-alternatives --install /usr/bin/g++ g++ /usr/bin/g++-7 100
sudo update-alternatives --install /usr/bin/cpp cpp /usr/bin/cpp-7 100
sudo ln -f -s /usr/bin/gcc /usr/bin/cc
  • SLES 15 SP3
sudo zypper install -y libopenssl-devel autoconf automake wget ncurses-devel bison patch tar gzip cmake gawk xz gcc-c++ git python curl ccache
  • If applicable, link /usr/bin/s390x-linux-gnu-gcc to /usr/bin/gcc on SLES
sudo ln -s /usr/bin/gcc /usr/bin/s390x-linux-gnu-gcc
  • RHEL (7.8, 7.9)
sudo subscription-manager repos --enable=rhel-7-server-for-system-z-rhscl-rpms
sudo yum install -y git autoconf automake openssl-devel wget ncurses-devel bison patch tar gzip xz make bzip2 zlib-devel devtoolset-7-gcc-c++ devtoolset-7-gcc curl diffutils gettext-devel python
  • RHEL 8.4
sudo yum install -y gcc-c++ autoconf libarchive cmake git wget make ncurses-devel curl xz diffutils bison ccache python2
sudo ln /usr/bin/gcc /usr/bin/s390x-linux-gnu-gcc

1.3) Enable Software Collections: (Only for RHEL 7.x)

CockroachDB requires GCC 7. It is available on RHEL 7.x via the 'Software Collections' packaging system. You can find out more about Software Collections here: https://www.softwarecollections.org/en/.

Following command will enable GCC 7.

source /opt/rh/devtoolset-7/enable

sudo ln -f -s /opt/rh/devtoolset-7/root/usr/bin/s390x-redhat-linux-gcc /opt/rh/devtoolset-7/root/usr/bin/s390x-linux-gnu-gcc
sudo ln -f -s /opt/rh/devtoolset-7/root/usr/bin/s390x-redhat-linux-c++ /opt/rh/devtoolset-7/root/usr/bin/s390x-linux-gnu-c++
sudo ln -f -s /opt/rh/devtoolset-7/root/usr/bin/s390x-redhat-linux-g++ /opt/rh/devtoolset-7/root/usr/bin/s390x-linux-gnu-g++

1.4) Build and install CMake (Only for RHEL 7.x, SLES 12 SP5, Ubuntu 18.04 and Ubuntu 20.04)

cd $SOURCE_ROOT
wget https://github.com/Kitware/CMake/releases/download/v3.20.3/cmake-3.20.3.tar.gz
tar -xvzf cmake-3.20.3.tar.gz
cd cmake-3.20.3
./bootstrap
make
sudo make install
cmake --version

1.5) Build and install Git (Only for RHEL 7.x)

cd $SOURCE_ROOT
wget https://github.com/git/git/archive/refs/tags/v2.27.1.tar.gz
tar -xvzf v2.27.1.tar.gz
cd git-2.27.1
make configure
./configure --prefix=/usr
make
sudo make install
git --version

1.6) Install Go 1.16.6

cd $SOURCE_ROOT
wget https://storage.googleapis.com/golang/go1.16.6.linux-s390x.tar.gz
sudo tar -C /usr/local -xzf go1.16.6.linux-s390x.tar.gz
export PATH=/usr/local/go/bin:$PATH

1.7) Install Nodejs and yarn

cd $SOURCE_ROOT
wget https://nodejs.org/dist/v12.20.1/node-v12.20.1-linux-s390x.tar.xz
chmod ugo+r node-v12.20.1-linux-s390x.tar.xz
sudo tar -C /usr/local -xf node-v12.20.1-linux-s390x.tar.xz
export PATH=$PATH:/usr/local/node-v12.20.1-linux-s390x/bin
node -v

# For Ubuntu and RHEL
sudo env PATH=$PATH npm install -g yarn

# For SLES
sudo chmod ugo+w -R /usr/local/node-v12.20.1-linux-s390x
env PATH=$PATH npm install -g yarn

1.8) Change the ownership of .config (Only for Ubuntu)

cd
mkdir -p .config
sudo chown -R $(whoami):$(whoami) .config

1.9) Build and Install CockroachDB

  • Download source code

    export GOPATH=$SOURCE_ROOT
cd $SOURCE_ROOT
mkdir -p $(go env GOPATH)/src/github.com/cockroachdb
cd $(go env GOPATH)/src/github.com/cockroachdb
git clone https://github.com/cockroachdb/cockroach
cd cockroach
git checkout v21.2.6
git submodule update --init --recursive # Fetch the submodules

  • Apply patches to code using below commands:

    cd $SOURCE_ROOT/src/github.com/cockroachdb/cockroach
curl -sSL https://github.com/apache/arrow/commit/aca707086160afd92da62aa2f9537a284528e48a.patch | git apply --directory=vendor/github.com/apache/arrow --exclude '**/array/bufferbuilder_numeric_test.go' -

  • Build and install

    cd $(go env GOPATH)/src/github.com/cockroachdb/cockroach
make build
sudo env GOPATH=$GOPATH PATH=$PATH make install
sudo mkdir -p /usr/local/lib/cockroach
sudo cp lib/libgeos.so /usr/local/lib/cockroach/
sudo cp lib/libgeos_c.so /usr/local/lib/cockroach/

Note: The warning 'bazel' not found (brew install bazel for macs) is issued if bazel is not installed on the system. It is observed that bazel is not a strict requirement and doesn't affect any functionality for the mentioned release.

Step 2: Testing (Optional)

cd $(go env GOPATH)/src/github.com/cockroachdb/cockroach
make test

There are currently known test failures on s390x.

Many tests check for exact floating point values however due to optimizations on the s390x platform that cause small differences, tests in these packages will fail.

  • github.com/cockroachdb/cockroach/pkg/ccl/logictestccl
  • github.com/cockroachdb/cockroach/pkg/geo
  • github.com/cockroachdb/cockroach/pkg/geo/geogfn
  • github.com/cockroachdb/cockroach/pkg/geo/geographiclib
  • github.com/cockroachdb/cockroach/pkg/geo/geoindex
  • github.com/cockroachdb/cockroach/pkg/geo/geomfn
  • github.com/cockroachdb/cockroach/pkg/sql/logictest
  • github.com/cockroachdb/cockroach/pkg/sql/opt/exec/execbuilder
  • github.com/cockroachdb/cockroach/pkg/sql/opt/memo
  • github.com/cockroachdb/cockroach/pkg/sql/sem/tree
  • github.com/cockroachdb/cockroach/pkg/sql/sem/tree/eval_test

Tests in these packages fail because of platform dependent filenames in the stack traces.

  • github.com/cockroachdb/cockroach/pkg/sql
  • github.com/cockroachdb/cockroach/pkg/util/log/logcrash

Tests in this package fail due to platform dependent hash function values.

  • github.com/cockroachdb/cockroach/pkg/sql/colflow

These specific tests in the github.com/cockroachdb/cockroach/pkg/sql/logictest package fail because the binary encoding of the sql plan is platform dependent but the plan is correct and is decoded correctly.

  • github.com/cockroachdb/cockroach/pkg/sql/logictest
    • TestLogic/5node/dist_vectorize
    • TestLogic/5node/explain_analyze_plans
    • TestLogic/5node/distsql_stats
    • TestLogic/5node-disk/dist_vectorize
    • TestLogic/5node-metadata/distsql_stats

Note:

1. Some test cases might experience time out problem. They should pass after increasing the timeout and run the test again. To run individual test with an increased time out. For example, github.com/cockroachdb/cockroach/pkg/sql/logictest could fail due to timeout, we do the following:

cd $(go env GOPATH)/src/github.com/cockroachdb/cockroach/pkg/sql/logictest
go test . -timeout 1h

Step 3: Verify

cockroach version

Output should look like:

Build Tag:        v21.2.6-dirty
Build Time:       2022/03/25 09:36:51
Distribution:     CCL
Platform:         linux s390x (s390x-linux-gnu)
Go Version:       go1.16.6
C Compiler:       gcc 7.5.0
Build Commit ID:  c7cfac57476f06e3154b2fc38e77982a8bbc64c2
Build Type:       development

Please visit this link to start a local cluster and CockroachDB SQL.

References:

The information provided in this article is accurate at the time of writing, but on-going development in the open-source projects involved may make the information incorrect or obsolete. Please open issue or contact us on IBM Z Community if you have any questions or feedback.

Clone this wiki locally