-
Notifications
You must be signed in to change notification settings - Fork 56
Building Apache Mesos
Building Apache Mesos The instructions provided below specify the steps to build Apache Mesos version 1.8.0 on Linux on IBM Z for the following distributions:
- RHEL (7.4, 7.5, 7.6)
- SLES (12 SP4, 15)
- Ubuntu (16.04, 18.04, 19.04)
General Notes:
-
When following the steps below please use a standard permission user unless otherwise specified
-
A directory /<source_root>/ will be referred to in these instructions, this is a temporary writable directory anywhere you'd like to place it
If you want to build mesos using manual steps, go to STEP 1.2.
Use the following commands to build mesos using the build script. Please make sure you have wget installed.
wget -q https://raw.githubusercontent.com/linux-on-ibm-z/scripts/master/ApacheMesos/1.8.0/build_mesos.sh
# Build mesos
bash build_mesos.sh
If the build completes successfully, go to STEP 3. In case of error, check logs
for more details or go to STEP 1.2 to follow manual build steps.
-
RHEL (7.4, 7.5, 7.6)
-
With IBM SDK
sudo yum install make git tar wget java-1.8.0-ibm-devel gcc gcc-c++ patch libzip-devel zlib-devel libcurl-devel apr apr-util apr-devel subversion subversion-devel cyrus-sasl-md5 cyrus-sasl-devel python-devel which autoconf automake libtool bzip2 unzip openssl openssl-devel gperftools-devel net-tools
-
With OpenJDK
sudo yum install make git tar wget java-1.8.0-openjdk-devel gcc gcc-c++ patch libzip-devel zlib-devel libcurl-devel apr apr-util apr-devel subversion subversion-devel cyrus-sasl-md5 cyrus-sasl-devel python-devel which autoconf automake libtool bzip2 unzip openssl openssl-devel gperftools-devel net-tools
-
-
SLES 12 SP4
-
With IBM SDK
sudo zypper install wget tar gcc gcc-c++ git patch java-1_8_0-ibm java-1_8_0-ibm-devel libzypp-devel libapr1 libapr1-devel subversion subversion-devel cyrus-sasl-devel cyrus-sasl-crammd5 python-devel libclang autoconf automake libtool bzip2 unzip make
-
With OpenJDK
sudo zypper install wget tar gcc gcc-c++ git patch java-1_8_0-openjdk-devel libzypp-devel libapr1 libapr1-devel subversion subversion-devel cyrus-sasl-devel cyrus-sasl-crammd5 python-devel libclang autoconf automake libtool bzip2 unzip make
-
-
SLES 15
-
With IBM SDK
sudo zypper install wget tar gcc gcc-c++ git patch java-1_8_0-ibm java-1_8_0-ibm-devel libzypp-devel apr-devel libapr1 subversion subversion-devel cyrus-sasl-devel cyrus-sasl-crammd5 python-devel libclang5 autoconf automake libtool bzip2 unzip python-xml
-
With OpenJDK
sudo zypper install wget tar gcc gcc-c++ git patch java-1_8_0-openjdk-devel libzypp-devel apr-devel libapr1 subversion subversion-devel cyrus-sasl-devel cyrus-sasl-crammd5 python-devel libclang5 autoconf automake libtool bzip2 unzip python-xml
-
-
Ubuntu (16.04, 18.04, 19.04)
-
With IBM SDK
sudo apt-get update sudo apt-get install tar wget git build-essential python-dev python-six python-virtualenv libcurl4-nss-dev libsasl2-dev libsasl2-modules maven libapr1-dev libsvn-dev zlib1g-dev libssl-dev autoconf automake libtool bzip2 unzip libgflags-dev libgtest-dev pkg-config clang libc++-dev
Download IBM Java 8 SDK binary from IBM Java 8 and follow the instructions as per given in the link. Update
JAVA_HOME
andPATH
accordingly. -
With OpenJDK
sudo apt-get update sudo apt-get install tar wget git build-essential python-dev python-six python-virtualenv openjdk-8-jdk libcurl4-nss-dev libsasl2-dev libsasl2-modules maven libapr1-dev libsvn-dev zlib1g-dev libssl-dev autoconf automake libtool bzip2 unzip libgflags-dev libgtest-dev pkg-config clang libc++-dev
-
-
Install Maven 3.3.9 (on RHEL and SLES only)
export SOURCE_ROOT=/<source_root> cd $SOURCE_ROOT wget http://www-us.apache.org/dist/maven/maven-3/3.3.9/binaries/apache-maven-3.3.9-bin.tar.gz tar zxf apache-maven-3.3.9-bin.tar.gz export M2_HOME=$SOURCE_ROOT/apache-maven-3.3.9 export PATH=$M2_HOME/bin:$PATH
-
Build and install Python 3.x (on RHEL 7)
See instructions available here.
-
Build and install curl 7.64 (on Ubuntu 18.04)
cd $SOURCE_ROOT git clone https://github.com/curl/curl.git cd curl git checkout curl-7_64_0 ./buildconf ./configure --prefix=/opt/curl --disable-libcurl-option --disable-shared make sudo make install export PATH=/opt/curl/bin:$PATH export LD_LIBRARY_PATH=/opt/curl/lib:$LD_LIBRARY_PATH export CPATH=/opt/curl/include:$CPATH
-
Docker is a prerequisite for running the Apache Mesos test cases. Docker packages are provided for SLES and Ubuntu in there respective repositories. Instructions for installing Docker on RHEL can be found here.
-
Set environment variables only for OpenJDK:
export JAVA_HOME=/usr/lib/jvm/java-1.8.0 # Only for RHEL export JAVA_HOME=/usr/lib64/jvm/java-1.8.0 # Only for SLES export JAVA_HOME=/usr/lib/jvm/java-1.8.0-openjdk-s390x # Only for Ubuntu export JAVA_TOOL_OPTIONS='-Xmx2048M' export PATH=$JAVA_HOME/bin:$PATH
-
Set environment variables only for IBM SDK:
export JAVA_HOME=/usr/lib/jvm/java-1.8.0-ibm # Only for RHEL export JAVA_HOME=/usr/lib64/jvm/java-1.8.0-ibm # Only for SLES export JVM_DIR=$JAVA_HOME/jre/lib/s390x/default export JAVA_TEST_LDFLAGS="-L$JVM_DIR -R$JVM_DIR -Wl,-ljvm -ldl" export JAVA_JVM_LIBRARY=$JAVA_HOME/jre/lib/s390x/default/libjvm.so export PATH=$JAVA_HOME/bin:$PATH
export SOURCE_ROOT=/<source_root>
cd $SOURCE_ROOT
git clone https://github.com/apache/mesos
cd mesos/
git checkout 1.8.0
Bundling gRPC 1.11.0
cd $SOURCE_ROOT/mesos/3rdparty/
git clone -b v1.11.0 https://github.com/grpc/grpc.git grpc-1.11.0
cd grpc-1.11.0/
git submodule update --init third_party/cares
cd ..
tar zcf grpc-1.11.0.tar.gz --exclude .git grpc-1.11.0
rm -rf grpc-1.11.0
Modify $SOURCE_ROOT/mesos/3rdparty/versions.am
:
@@ -27,7 +27,7 @@ ELFIO_VERSION = 3.2
GLOG_VERSION = 0.3.3
GOOGLETEST_VERSION = 1.8.0
GPERFTOOLS_VERSION = 2.5
-GRPC_VERSION = 1.10.0
+GRPC_VERSION = 1.11.0
HTTP_PARSER_VERSION = 2.6.2
JEMALLOC_VERSION = 5.0.1
LEVELDB_VERSION = 1.19
Modify $SOURCE_ROOT/mesos/src/python/native_common/ext_modules.py.in
:
@@ -151,7 +151,7 @@ def _create_module(module_name):
# We link different grpc library variants based on whether SSL is enabled.
- grpc = os.path.join('3rdparty', 'grpc-1.10.0')
+ grpc = os.path.join('3rdparty', 'grpc-1.11.0')
grpc_variant = '_unsecure' if '@ENABLE_SSL_TRUE@' == '#' else ''
libgrpcpp = os.path.join(abs_top_builddir, grpc, 'libs', 'opt', 'libgrpc++%s.a' % grpc_variant)
libgrpc = os.path.join(abs_top_builddir, grpc, 'libs', 'opt', 'libgrpc%s.a' % grpc_variant)
Append the following lines to file $SOURCE_ROOT/mesos/3rdparty/protobuf-3.5.0.patch
:
diff --git a/src/google/protobuf/stubs/atomicops_internals_generic_gcc.h b/src/google/protobuf/stubs/atomicops_internals_generic_gcc.h
index 0b0b06c..075c406 100644
--- a/src/google/protobuf/stubs/atomicops_internals_generic_gcc.h
+++ b/src/google/protobuf/stubs/atomicops_internals_generic_gcc.h
@@ -146,6 +146,14 @@ inline Atomic64 NoBarrier_Load(volatile const Atomic64* ptr) {
return __atomic_load_n(ptr, __ATOMIC_RELAXED);
}
+inline Atomic64 Release_CompareAndSwap(volatile Atomic64* ptr,
+ Atomic64 old_value,
+ Atomic64 new_value) {
+ __atomic_compare_exchange_n(ptr, &old_value, new_value, false,
+ __ATOMIC_RELEASE, __ATOMIC_ACQUIRE);
+ return old_value;
+}
+
#endif // defined(__LP64__)
} // namespace internal
Append the following lines to file $SOURCE_ROOT/mesos/3rdparty/boost-1.65.0.patch
: (see https://github.com/boostorg/mpl/pull/34)
diff --git a/boost/mpl/assert.hpp b/boost/mpl/assert.hpp
index 1af1b05..e41b583 100644
--- a/boost/mpl/assert.hpp
+++ b/boost/mpl/assert.hpp
@@ -184,16 +184,27 @@ template< typename P > struct assert_arg_pred_not
typedef typename assert_arg_pred_impl<p>::type type;
};
+#if defined(BOOST_GCC) && BOOST_GCC >= 80000
+#define BOOST_MPL_IGNORE_PARENTHESES_WARNING
+#pragma GCC diagnostic push
+#pragma GCC diagnostic ignored "-Wparentheses"
+#endif
+
template< typename Pred >
-failed ************ (Pred::************
+failed ************ (Pred::************
assert_arg( void (*)(Pred), typename assert_arg_pred<Pred>::type )
);
template< typename Pred >
-failed ************ (boost::mpl::not_<Pred>::************
+failed ************ (boost::mpl::not_<Pred>::************
assert_not_arg( void (*)(Pred), typename assert_arg_pred_not<Pred>::type )
);
+#ifdef BOOST_MPL_IGNORE_PARENTHESES_WARNING
+#undef BOOST_MPL_IGNORE_PARENTHESES_WARNING
+#pragma GCC diagnostic pop
+#endif
+
template< typename Pred >
AUX778076_ASSERT_ARG(assert<false>)
assert_arg( void (*)(Pred), typename assert_arg_pred_not<Pred>::type );
cd $SOURCE_ROOT/mesos
./bootstrap
mkdir build
cd build
../configure
make
# Install (Optional)
sudo make install
NOTE: In case of Java out of Memory issue while building, do the following modification to file $SOURCE_ROOT/mesos/src/java/mesos.pom.in
and run make command again.
@@ -84,6 +84,7 @@
<plugin>
<artifactId>maven-javadoc-plugin</artifactId>
<configuration>
+ <maxmemory>512m</maxmemory>
<sourcepath>@abs_top_srcdir@/src/java/src:@abs_top_builddir@/src/java/generated</sourcepath>
<subpackages>org.apache.mesos</subpackages>
</configuration>
NOTE: Test cases need to be executed as root user.
Test cases of Apache Mesos uses the following images: alpine
, mesosphere/alpine-expect
, mesosphere/inky
, mesosphere/test-executor
, tnachen/test-executor
, haosdent/https-server
, zhq527725/https-server
, zhq527725/whiteout
and chhsiao/overwrite
. Dockerfiles for each of these images is given below. Make sure all the images are present on your machine before you start testing.
-
Dockerfile for image mesosphere/alpine-expect
FROM s390x/debian RUN apt-get update RUN apt-get install -y expect
-
Dockerfile for image mesosphere/inky
FROM s390x/busybox CMD ["inky"] ENTRYPOINT ["echo"]
-
Dockerfile for both the images mesosphere/test-executor and tnachen/test-executor
FROM s390x/debian RUN apt-get update RUN apt-get install -y wget tar git make libnss-myhostname WORKDIR /root RUN wget https://storage.googleapis.com/golang/go1.7.1.linux-s390x.tar.gz RUN tar -xzf go1.7.1.linux-s390x.tar.gz ENV PATH $PATH:/root/go/bin ENV GOROOT /root/go ENV GOPATH /root ENV CC gcc RUN mkdir -p src/github.com/mesos/mesos-go WORKDIR /root/src RUN git clone https://github.com/tnachen/go-mesos.git github.com/mesos/mesos-go WORKDIR /root/src/github.com/mesos/mesos-go/examples/ RUN go get github.com/tools/godep ENV PATH $PATH:/root/bin RUN godep restore && go install ./... RUN sed -i 's/RACE := -race//g' Makefile RUN sed -i 's/$(RACE)//g' Makefile RUN make RUN cp /root/src/github.com/mesos/mesos-go/examples/_output/executor /bin/test-executor
-
Dockerfile for image zhq527725/whiteout
FROM s390x/busybox RUN mkdir -p /dir1/dir2 && touch /dir1/file1 && touch /dir1/dir2/file2 RUN rm -rf /dir1/file1 && rm -rf /dir1/dir2 && mkdir /dir1/dir2 && touch /dir1/dir2/file3
-
Dockerfile for image chhsiao/overwrite
FROM s390x/alpine RUN mkdir /merged /replaced1 /replaced2 && touch /merged/m1 /replaced1/r1 /replaced2/r2 /replaced3 baz && ln -s /merged /replaced4 && ln -s baz foo && ln -s bar bar && ln -s ../../../../../../../abc xyz RUN rm -rf /replaced1 /replaced2 /replaced3 /replaced4 foo bar baz xyz && mkdir /replaced3 /replaced4 && touch /merged/m2 /replaced1 bar xyz && ln -s /merged /replaced2 && ln -s bar foo && ln -s baz baz
-
Steps for building both haosdent/https-server and zhq527725/https-server images
-
For haosdent/https-server, download files as given below
git clone https://github.com/haosdent/https-server cd https-server
-
For zhq527725/https-server, download files as given below
git clone https://github.com/qianzhangxa/https-server.git cd https-server
-
Replace the contents of
Dockerfile
in each of thehttps-server
folders with the followingFROM s390x/debian RUN apt-get update RUN apt-get install -y wget tar xz-utils gcc make patch tcl-tls libssl-dev RUN wget https://www.python.org/ftp/python/2.7.13/Python-2.7.13.tar.xz RUN tar -xvf Python-2.7.13.tar.xz WORKDIR /Python-2.7.13 RUN ./configure --prefix=/usr/ --exec-prefix=/usr/ RUN make RUN make install WORKDIR /tmp ADD https_server.py ./ ADD server.pem ./
-
-
Command to create image from
Dockerfile
isdocker build -t <name_of_image> .
-
Steps to save images to
/opt/docker/images
mkdir -p /opt/docker/images docker pull s390x/debian docker tag s390x/debian debian docker save s390x/debian >> /opt/docker/images/debian.tar docker pull s390x/alpine docker tag s390x/alpine alpine docker save alpine >> /opt/docker/images/alpine.tar docker pull s390x/hello-world docker tag s390x/hello-world hello-world docker save hello-world >> /opt/docker/images/hello-world.tar # Save the mesosphere/inky image created in previous step mkdir -p /opt/docker/images/mesosphere docker save mesosphere/inky >> /opt/docker/images/mesosphere/inky.tar mkdir -p /opt/docker/images/haosdent/ docker save haosdent/https-server >> /opt/docker/images/haosdent/https-server.tar mkdir -p /opt/docker/images/zhq527725/ docker save zhq527725/https-server >> /opt/docker/images/zhq527725/https-server.tar docker save zhq527725/whiteout >> /opt/docker/images/zhq527725/whiteout.tar docker save chhsiao/overwrite >> /opt/docker/images/chhsiao/overwrite.tar
-
For alpine image, find its digest ID:
docker images --digests //Copy the digest ID of the s390x/alpine image in the form of "sha256:####...""
Copy the digest ID for the step below. Create symlink of the image tarball with digest ID in the name:
ln -s /opt/docker/images/alpine.tar /opt/docker/images/alpine@sha256:<digest_id>.tar
- Modify
$SOURCE_ROOT/mesos/src/slave/flags.cpp
@@ -211,7 +211,7 @@ mesos::internal::slave::Flags::Flags() "path (e.g., `/tmp/docker/images`), or as an HDFS URI (*experimental*)\n" "(e.g., `hdfs://localhost:8020/archives/`). Note that this option won't\n" "change the default registry server for Docker containerizer.", - "https://registry-1.docker.io"); + "/opt/docker/images"); add(&Flags::docker_store_dir, "docker_store_dir",
- Modify
$SOURCE_ROOT/mesos/src/tests/containerizer/provisioner_docker_tests.cpp
, the SHA is the digest ID noted in the previous step.@@ -643,13 +643,13 @@ INSTANTIATE_TEST_CASE_P( ProvisionerDockerTest, ::testing::ValuesIn(vector<string>({ "alpine", // Verifies the normalization of the Docker repository name. - "library/alpine", - "gcr.io/google-containers/busybox:1.24", // manifest.v1+prettyjws - "gcr.io/google-containers/busybox:1.27", // manifest.v2+json + //"library/alpine", + //"gcr.io/google-containers/busybox:1.24", // manifest.v1+prettyjws + //"gcr.io/google-containers/busybox:1.27", // manifest.v2+json // TODO(alexr): The registry below is unreliable and hence disabled. // Consider re-enabling shall it become more stable. // "registry.cn-hangzhou.aliyuncs.com/acs-sample/alpine", - "quay.io/coreos/alpine-sh" // manifest.v1+prettyjws + //"quay.io/coreos/alpine-sh" // manifest.v1+prettyjws }))); @@ -1345,11 +1345,11 @@ TEST_F(ProvisionerDockerTest, ROOT_INTERNET_CURL_ImageDigest) // NOTE: We use the digest of the 'alpine:2.7' image because it has a // Schema 1 manifest (the only manifest schema that we currently support). const string digest = - "sha256:9f08005dff552038f0ad2f46b8e65ff3d25641747d3912e3ea8da6785046561a"; + "sha256:<digest_id>"; Image image; image.set_type(Image::DOCKER); - image.mutable_docker()->set_name("library/alpine@" + digest); + image.mutable_docker()->set_name("alpine@" + digest); ContainerInfo* container = task.mutable_container(); container->set_type(ContainerInfo::MESOS);
- Modify
$SOURCE_ROOT/mesos/src/tests/containerizer/runtime_isolator_tests.cpp
@@ -411,7 +411,7 @@ TEST_F(DockerRuntimeIsolatorTest, ROOT_INTERNET_CURL_NestedSimpleCommand) v1::createCommandInfo("/bin/ls", {"ls", "-al", "/"})); taskInfo.mutable_container()->CopyFrom( - v1::createContainerInfo("library/alpine")); + v1::createContainerInfo("debian")); Future<Event::Update> updateStarting; Future<Event::Update> updateRunning;
- Modify
$SOURCE_ROOT/mesos/src/tests/health_check_tests.cpp
@@ -1444,7 +1444,7 @@ TEST_F_TEMP_DISABLED_ON_WINDOWS( // Use Netcat to launch a HTTP server. const string command = strings::format( - "nc -lk -p %u -e echo -e \"HTTP/1.1 200 OK\r\nContent-Length: 0\r\n\"", + "nc -ll -p %u -e echo -e \"HTTP/1.1 200 OK\r\nContent-Length: 0\r\n\"", testPort).get(); TaskInfo task = createTask(offers.get()[0], command); @@ -1638,7 +1638,7 @@ TEST_F_TEMP_DISABLED_ON_WINDOWS( // Use Netcat to launch a HTTP server. const string command = strings::format( - "nc -lk -p %u -e echo -e \"HTTP/1.1 200 OK\r\nContent-Length: 0\r\n\"", + "nc -ll -p %u -e echo -e \"HTTP/1.1 200 OK\r\nContent-Length: 0\r\n\"", testPort).get(); TaskInfo task = createTask(offers.get()[0], command);
- Modify
$SOURCE_ROOT/mesos/src/tests/containerizer/docker_containerizer_tests.cpp
@@ -3965,7 +3965,7 @@ TEST_F(DockerContainerizerTest, ROOT_DOCKER_NoTransitionFromKillingToRunning) // Launch a HTTP server until SIGTERM is received, then sleep for // 15 seconds to let the health check fail. const string command = strings::format( - "trap \"sleep 15\" SIGTERM && nc -lk -p %u -e echo", + "trap \"sleep 15\" SIGTERM && nc -ll -p %u -e echo", testPort).get(); #endif // __WINDOWS__
- Modify
$SOURCE_ROOT/mesos/src/tests/containerizer/cgroups_isolator_tests.cpp
(Ubuntu 19.04 only)diff --git a/src/tests/containerizer/cgroups_isolator_tests.cpp b/src/tests/containerizer/cgroups_isolator_tests.cpp index 957f72d78..a92e2cdba 100644 --- a/src/tests/containerizer/cgroups_isolator_tests.cpp +++ b/src/tests/containerizer/cgroups_isolator_tests.cpp @@ -1763,7 +1763,7 @@ TEST_F(CgroupsIsolatorTest, ROOT_CGROUPS_MemoryBackward) // This test verifies the cgroups blkio statistics // of the container can be successfully retrieved. -TEST_F(CgroupsIsolatorTest, ROOT_CGROUPS_BlkioUsage) +TEST_F(CgroupsIsolatorTest, DISABLED_ROOT_CGROUPS_BlkioUsage) { Try<Owned<cluster::Master>> master = StartMaster(); ASSERT_SOME(master);
cd $SOURCE_ROOT/mesos/build
make check -k
NOTE:
1. In case of any test case failures, execute the failed test case individually as given below
cd $SOURCE_ROOT/mesos/build
./bin/mesos-tests.sh --gtest_filter=<test_case>
2. If the docker containers on your machine takes longer time to release the resources after exiting, there are chances of following 2 test case failures: DockerContainerizerTest.ROOT_DOCKER_NC_PortMapping, DockerTest.ROOT_DOCKER_CheckPortResource
3. In case of the following test case failures: CgroupsAnyHierarchyWithCpuMemoryTest.ROOT_CGROUPS_Listen
, execute swapoff -a
command on your machine.
4. Make sure /etc/hosts
file have entry with following details:
<ipaddress> <Fully Qualified Domain Name> <hostname>
5.ContainerizerTest.ROOT_CGROUPS_BalloonFramework
test case fails if 5432 port is already in use. Port can be changed in file $SOURCE_ROOT/mesos/src/tests/balloon_framework_test.sh
6. Test cases CniIsolatorTest.ROOT_*
, CniIsolatorPortMapperTest.ROOT_IPTABLES_NC_PortMapper
, NetworkParam/DefaultExecutorCniTest.ROOT_VerifyContainerIP
and DefaultContainerDNSInfo/DefaultContainerDNSCniTest.ROOT_VerifyDefaultDNS
has been known to fail in SLES for both s390x and x86. https://issues.apache.org/jira/browse/MESOS-8364
cd $SOURCE_ROOT/mesos/build
sudo ./bin/mesos-master.sh --ip=<ip_address> --work_dir=/var/lib/mesos
NOTE: Ensure work directory exists and has required permissions
cd $SOURCE_ROOT/mesos/build
sudo ./bin/mesos-agent.sh --master=<ip_address>:5050 --work_dir=/var/lib/mesos
Open http://\<ip_address\>:5050
in your browser.
The information provided in this article is accurate at the time of writing, but on-going development in the open-source projects involved may make the information incorrect or obsolete. Please open issue or contact us on IBM Z Community if you have any questions or feedback.