Building Apache Storm

Apache Storm

The instructions provided below specify the steps to build Apache Storm version 2.1.0 on Linux on IBM Z for the following distributions:

• RHEL (7.5, 7.6, 7.7, 8.0)
• SLES (12 SP4, 15 SP1)
• Ubuntu (16.04, 18.04, 19.10)

General Notes:

• When following the steps below please use a standard permission user unless otherwise specified.
• A directory /<source_root>/ will be referred to in these instructions, this is a temporary writable directory anywhere you'd like to place it.

Step 1: Building and installing Apache Storm

1.1) Install Dependencies

export SOURCE_ROOT=/<source_root>/

• RHEL (7.5, 7.6, 7.7, 8.0)

  sudo yum install git tar java-1.8.0-openjdk-devel.s390x wget which unzip xz procps

• SLES (12 SP4, 15 SP1)

  sudo zypper install git java-1_8_0-openjdk-devel tar wget curl which unzip xz gzip

• Ubuntu (16.04)

  sudo apt update
  sudo apt-get install git tar ant wget openjdk-8-jdk curl gpgv nodejs gnupg2 python nodejs-legacy

• Ubuntu (18.04, 19.10)

  sudo apt update
  sudo apt-get install git tar ant wget openjdk-8-jdk curl gpgv nodejs gnupg2 python

1.2) Install Maven 3.5.4

cd $SOURCE_ROOT
wget https://repo.maven.apache.org/maven2/org/apache/maven/apache-maven/3.5.4/apache-maven-3.5.4-bin.tar.gz
tar -xzf apache-maven-3.5.4-bin.tar.gz
export PATH=$PWD/apache-maven-3.5.4/bin:$PATH

1.3) Install NodeJS SDK (For RHEL and SLES only)

Install nodejs sdk from https://nodejs.org/en/download/ for Linux on System z

Add the nodejs executable to the path:

cd $SOURCE_ROOT
wget https://nodejs.org/dist/v12.13.1/node-v12.13.1-linux-s390x.tar.xz
tar xvf node-v12.13.1-linux-s390x.tar.xz
export PATH=$PATH:$PWD/node-v12.13.1-linux-s390x/bin

1.4) Install rvm and nvm

command curl -sSL https://rvm.io/mpapis.asc | gpg2 --import -
command curl -sSL https://rvm.io/pkuczynski.asc | gpg2 --import -
curl -L https://get.rvm.io | bash -s stable --autolibs=enabled && source $HOME/.profile
wget -qO- https://raw.githubusercontent.com/creationix/nvm/v0.26.1/install.sh | bash && source $HOME/.bashrc

1.5) Set environment variables

Setup up Maven and JAVA:

export JAVA_HOME=<path to java>          
export PATH=$JAVA_HOME/bin:$PATH
export _JAVA_OPTIONS=-Xmx4096m
export JVM_ARGS="-Xms4096m -Xmx4096m"
export MAVEN_OPTS="-Xms4096m -Xmx8192m -Xss4096k"

1.6) Download the source code

cd $SOURCE_ROOT
git clone https://github.com/apache/storm.git  
cd storm  
git checkout tags/v2.1.0

1.7) Build Apache Storm

mvn clean install  

Note: If you wish to skip the unit tests you can do this by adding -DskipTests to the command line.

Step 2: Testing(Optional)

2.1) Run unit test cases

To run Clojure and Java unit tests but no integration tests execute the command

cd $SOURCE_ROOT/storm
mvn test  

Note:

There are some failed test cases that are also observed on x86_64 platform.

Step 3: Create a Storm distribution (packaging)

cd $SOURCE_ROOT/storm/storm-dist/binary
mvn package

Note:

• After running mvn package you may be asked to enter your GPG/PGP credentials (once for each binary file, in fact). This happens because the packaging step will create *.asc digital signatures for all the binaries, and in the workflow above your GPG private key will be used to create those signatures.
• After running mvn package you may get error while building Storm Binary Distribution(final package). This happens because of failure in GPG signing.
• You can disbale GPG signing by adding -Dgpg.skip=true
• The binaries can be found at /<source_root>/storm/storm-dist/binary/target/.

References:

• http://storm.apache.org/releases/current/index.html
• https://github.com/apache/storm/blob/v2.1.0/DEVELOPER.md