Separate IN PR (#124)

* finishing the in expression. adding more tests and null support. need confirmation on null behavior and also I wonder why integer field is sufficient for string

* adding additional test

* adding additional test

* saving concat implementation and it's passing basic functionality tests

* adding type aware comparison and better error message for IN operator

* adding null checking for the concat operator and adding one additional test

* cleaning up IN&Concat PR

* deleting concat and preping the in branch for in pr

* fixing null bahavior

now it's only null when there's no match and there's null input

* Build failed

Co-authored-by: Ubuntu <[email protected]>
Co-authored-by: Wenting Zheng <[email protected]>
Co-authored-by: Wenting Zheng <[email protected]>

Separate Concat PR  (#125)

Implementation of the CONCAT expression.

Co-authored-by: Ubuntu <[email protected]>
Co-authored-by: Wenting Zheng <[email protected]>

Removed calls to toSet in TPC-H tests (#140)

* removed calls to toSet

* added calls to toSet back where queries are unordered

Documentation update (#148)

Cluster Remote Attestation Fix (#146)

The existing code only had RA working when run locally. This PR adds a sleep for 5 seconds to make sure that all executors are spun up successfully before attestation begins.

Closes #147

upgrade to 3.0.1 (#144)

Update two TPC-H queries (#149)

Tests for TPC-H 12 and 19 pass.

TPC-H 20 Fix (#142)

* string to stringtype error

* tpch 20 passes

* cleanup

* implemented changes

* decimal.tofloat

Co-authored-by: Wenting Zheng <[email protected]>

Join update (#145)

Migrate from Travis CI to Github Actions (#156)

matching in strategies.scala

set up class thing

cleanup

added test cases for non-equi left anti join

rename to serializeEquiJoinExpression

added isEncrypted condition

set up keys

JoinExpr now has condition

rename

serialization does not throw compile error for BNLJ

split up

added condition in ExpressionEvaluation.h

zipPartitions

cpp put in place

typo

added func to header

two loops in place

update tests

condition

fixed scala loop

interchange rows

added tags

ensure cached

== match working

comparison decoupling in ExpressionEvalulation

save

compiles and condition works

is printing

fix swap outer/inner

o_i_match

show() has the same result

tests pass

test cleanup

added test cases for different condition

BuildLeft works

optional keys in scala

started C++

passes the operator tests

comments, cleanup

attemping to do it the ~right~ way

comments to distinguish between primary/secondary, operator tests pass

cleanup comments, about to begin implementation for distinct agg ops

is_distinct

added test case

serializing with isDistinct

is_distinct in ExpressionEvaluation.h

removed unused code from join implementation

remove RowWriter/Reader in condition evaluation (join)

easier test

serialization done

correct checking in Scala

set is set up

spaghetti but it finally works

function for clearing values

condition_eval isntead of condition

goto

comment

remove explain from test, need to fix distinct aggregation for >1 partitions

started impl of multiple partitions fix

added rangepartitionexec that runs

partitioning cleanup

serialization properly

comments, generalization for > 1 distinct function

comments

about to refactor into logical.Aggregation

the new case has distinct in result expressions

need to match on distinct

removed new case (doesn't make difference?)

works

remove traces of distinct

more cleanup

Upgrade to OE 0.12 (#153)

Update README.md

Support for scalar subquery (#157)

This PR implements the scalar subquery expression, which is triggered whenever a subquery returns a scalar value. There were two main problems that needed to be solved.

First, support for matching the scalar subquery expression is necessary. Spark implements this by wrapping a SparkPlan within the expression and calls executeCollect. Then it constructs a literal with that value. However, this is problematic for us because that value should not be decrypted by the driver and serialized into an expression, since it's an intermediate value.

Therefore, the second issue to be addressed here is supporting an encrypted literal. This is implemented in this PR by serializing an encrypted ciphertext into a base64 encoded string, and wrapping a Decrypt expression on top of it. This expression is then evaluated in the enclave and returns a literal. Note that, in order to test our implementation, we also implement a Decrypt expression in Scala. However, this should never be evaluated on the driver side and serialized into a plaintext literal. This is because Decrypt is designated as a Nondeterministic expression, and therefore will always evaluate on the workers.

Add TPC-H Benchmarks (#139)

* logic decoupling in TPCH.scala for easier benchmarking

* added TPCHBenchmark.scala

* Benchmark.scala rewrite

* done adding all support TPC-H query benchmarks

* changed commandline arguments that benchmark takes

* TPCHBenchmark takes in parameters

* fixed issue with spark conf

* size error handling, --help flag

* add Utils.force, break cluster mode

* comment out logistic regression benchmark

* ensureCached right before temp view created/replaced

* upgrade to 3.0.1

* upgrade to 3.0.1

* 10 scale factor

* persistData

* almost done refactor

* more cleanup

* compiles

* 9 passes

* cleanup

* collect instead of force, sf_none

* remove sf_none

* defaultParallelism

* no removing trailing/leading whitespace

* add sf_med

* hdfs works in local case

* cleanup, added new CLI argument

* added newly supported tpch queries

* function for running all supported tests

address comments

added one test case

non-null case working

rename equi join

split Join.cpp into two files

outer and default joins split up

not handling nulls at all

first test case works

force_null to all appends

test, matching in scala

non-nulls working

it works for anti and outer

cleanup

test cases added

one row is not being added in the sort merge implementation

tpc-h 13 passes

comments

outer/inner swap, breaks a bunch of things

Update App.cpp

fixed swap issues

for loop instead of flatten

concatEncryptedBlocks

tpch 13 test passes

one more swap

stream/broadcast

concatEncryptedBlocks, remove import iostream

comment for for loop

added comments explaining constraints with broadcast side

comments

Author: Chenyu-Shi

.github/scripts/build.sh

@@ -0,0 +1,25 @@
+# Install OpenEnclave 0.9.0
+echo 'deb [arch=amd64] https://download.01.org/intel-sgx/sgx_repo/ubuntu bionic main' | sudo tee /etc/apt/sources.list.d/intel-sgx.list
+wget -qO - https://download.01.org/intel-sgx/sgx_repo/ubuntu/intel-sgx-deb.key | sudo apt-key add -
+echo "deb http://apt.llvm.org/bionic/ llvm-toolchain-bionic-7 main" | sudo tee /etc/apt/sources.list.d/llvm-toolchain-bionic-7.list
+wget -qO - https://apt.llvm.org/llvm-snapshot.gpg.key | sudo apt-key add -
+echo "deb [arch=amd64] https://packages.microsoft.com/ubuntu/18.04/prod bionic main" | sudo tee /etc/apt/sources.list.d/msprod.list
+wget -qO - https://packages.microsoft.com/keys/microsoft.asc | sudo apt-key add -
+
+sudo apt update
+sudo apt -y install clang-7 libssl-dev gdb libsgx-enclave-common libsgx-enclave-common-dev libprotobuf10 libsgx-dcap-ql libsgx-dcap-ql-dev az-dcap-client open-enclave=0.12.0
+
+# Install Opaque Dependencies
+sudo apt -y install wget build-essential openjdk-8-jdk python libssl-dev
+
+wget https://github.com/Kitware/CMake/releases/download/v3.15.6/cmake-3.15.6-Linux-x86_64.sh
+sudo bash cmake-3.15.6-Linux-x86_64.sh --skip-license --prefix=/usr/local
+
+# Generate keypair for attestation
+openssl genrsa -out ./private_key.pem -3 3072
+
+source opaqueenv
+source /opt/openenclave/share/openenclave/openenclaverc
+export MODE=SIMULATE
+
+build/sbt test

.github/workflows/main.yml

@@ -0,0 +1,40 @@
+name: CI
+
+# Controls when the action will run. 
+on:
+  # Triggers the workflow on push or pull request events but only for the master branch
+  push:
+    branches: [ master ]
+  pull_request:
+    branches: [ master ]
+
+  # Allows you to run this workflow manually from the Actions tab
+  workflow_dispatch:
+
+# A workflow run is made up of one or more jobs that can run sequentially or in parallel
+jobs:
+  build:
+    # Define the OS to run on
+    runs-on: ubuntu-18.04
+    # Steps represent a sequence of tasks that will be executed as part of the job
+    steps:
+      # Checks-out your repository under $GITHUB_WORKSPACE, so your job can access it
+      - uses: actions/checkout@v2
+      # Specify the version of Java that is installed
+      - uses: actions/setup-java@v1
+        with:
+          java-version: '8'
+      # Caching (from https://www.scala-sbt.org/1.x/docs/GitHub-Actions-with-sbt.html)
+      - uses: coursier/cache-action@v5
+      # Run the test
+      - name: Install dependencies, set environment variables, and run sbt tests
+        run: |
+          ./.github/scripts/build.sh
+
+          rm -rf "$HOME/.ivy2/local" || true
+          find $HOME/Library/Caches/Coursier/v1        -name "ivydata-*.properties" -delete || true
+          find $HOME/.ivy2/cache                       -name "ivydata-*.properties" -delete || true
+          find $HOME/.cache/coursier/v1                -name "ivydata-*.properties" -delete || true
+          find $HOME/.sbt                              -name "*.lock"               -delete || true
+        shell: bash
+

.travis.yml

@@ -16,7 +16,7 @@ before_install:
   - sudo apt update
   - sudo apt -y install clang-7 libssl-dev gdb libsgx-enclave-common libsgx-enclave-common-dev libprotobuf10 libsgx-dcap-ql libsgx-dcap-ql-dev
   - sudo apt-get -y install wget build-essential openjdk-8-jdk python libssl-dev
-  - sudo apt-get -y install open-enclave=0.9.0
+  - sudo apt-get -y install open-enclave=0.12.0
   - wget https://github.com/Kitware/CMake/releases/download/v3.15.6/cmake-3.15.6-Linux-x86_64.sh
   - sudo bash cmake-3.15.6-Linux-x86_64.sh --skip-license --prefix=/usr/local
   - export PATH=/usr/local/bin:"$PATH"

README.md

@@ -8,11 +8,12 @@ Opaque is a package for Apache Spark SQL that enables encryption for DataFrames
 
 This project is based on the following NSDI 2017 paper [1]. The oblivious execution mode is not included in this release.
 
-This is an alpha preview of Opaque, which means the software is still in development (not production-ready!). It currently has the following limitations:
+This is an alpha preview of Opaque, but the software is still in active development. It currently has the following limitations:
 
 - Unlike the Spark cluster, the master must be run within a trusted environment (e.g., on the client).
 
-- Not all Spark SQL operations are supported. UDFs must be [implemented in C++](#user-defined-functions-udfs).
+- Not all Spark SQL operations are supported (see the [list of supported operations](#supported-functionalities)).
+UDFs must be [implemented in C++](#user-defined-functions-udfs).
 
 - Computation integrity verification (section 4.2 of the NSDI paper) is currently work in progress.
 
@@ -23,7 +24,7 @@ This is an alpha preview of Opaque, which means the software is still in develop
 
 After downloading the Opaque codebase, build and test it as follows.
 
-1. Install dependencies and the [OpenEnclave SDK](https://github.com/openenclave/openenclave/blob/v0.9.x/docs/GettingStartedDocs/install_oe_sdk-Ubuntu_18.04.md). We currently support OE version 0.9.0 (so please install with `open-enclave=0.9.0`) and Ubuntu 18.04.
+1. Install dependencies and the [OpenEnclave SDK](https://github.com/openenclave/openenclave/blob/v0.12.0/docs/GettingStartedDocs/install_oe_sdk-Ubuntu_18.04.md). We currently support OE version 0.12.0 (so please install with `open-enclave=0.12.0`) and Ubuntu 18.04.
 
     ```sh
     # For Ubuntu 18.04:
@@ -59,7 +60,9 @@ After downloading the Opaque codebase, build and test it as follows.
 
 ## Usage
 
-Next, run Apache Spark SQL queries with Opaque as follows, assuming [Spark 3.0](https://www.apache.org/dyn/closer.lua/spark/spark-3.0.1/spark-3.0.1-bin-hadoop2.7.tgz) (`wget http://apache.mirrors.pair.com/spark/spark-3.0.1/spark-3.0.1-bin-hadoop2.7.tgz`) is already installed:
+Next, run Apache Spark SQL queries with Opaque as follows, assuming [Spark 3.0.1](https://www.apache.org/dyn/closer.lua/spark/spark-3.0.1/spark-3.0.1-bin-hadoop2.7.tgz) (`wget http://apache.mirrors.pair.com/spark/spark-3.0.1/spark-3.0.1-bin-hadoop2.7.tgz`) is already installed:
+
+\* Opaque needs Spark's `'spark.executor.instances'` property to be set. This can be done in a custom config file, the default config file found at `/opt/spark/conf/spark-defaults.conf`, or as a `spark-submit` or `spark-shell` argument: `--conf 'spark.executor.instances=<value>`.
 
 1. Package Opaque into a JAR:
 
@@ -136,6 +139,41 @@ Next, run Apache Spark SQL queries with Opaque as follows, assuming [Spark 3.0](
     // | baz|    5|
     // +----+-----+
     ```
+
+## Supported functionalities
+
+This section lists Opaque's supported functionalities, which is a subset of that of Spark SQL. Note that the syntax for these functionalities is the same as Spark SQL -- Opaque simply replaces the execution to work with encrypted data.
+
+### Data types
+Out of the existing [Spark SQL types](https://spark.apache.org/docs/latest/sql-ref-datatypes.html), Opaque supports
+
+- All numeric types except `DecimalType`, which is currently converted into `FloatType`
+- `StringType`
+- `BinaryType`
+- `BooleanType`
+- `TimestampTime`, `DateType`
+- `ArrayType`, `MapType`
+
+### Functions
+We currently support a subset of the Spark SQL functions, including both scalar and aggregate-like functions.
+
+- Scalar functions: `case`, `cast`, `concat`, `contains`, `if`, `in`, `like`, `substring`, `upper`
+- Aggregate functions: `average`, `count`, `first`, `last`, `max`, `min`, `sum`
+
+UDFs are not supported directly, but one can [extend Opaque with additional functions](#user-defined-functions-udfs) by writing it in C++.
+
+
+### Operators
+
+Opaque supports the core SQL operators:
+
+- Projection
+- Filter
+- Global aggregation and grouping aggregation
+- Order by, sort by
+- Inner join
+- Limit
+
     
 ## User-Defined Functions (UDFs)
 
@@ -168,7 +206,3 @@ Now we can port this UDF to Opaque as follows:
     ```
 
 3. Finally, implement the UDF in C++. In [`FlatbuffersExpressionEvaluator#eval_helper`](src/enclave/Enclave/ExpressionEvaluation.h), add a case for `tuix::ExprUnion_DotProduct`. Within that case, cast the expression to a `tuix::DotProduct`, recursively evaluate the left and right children, perform the dot product computation on them, and construct a `DoubleField` containing the result.
-
-## Contact
-
-If you want to know more about our project or have questions, please contact Wenting ([email protected]) and/or Ankur ([email protected]).

build.sbt

@@ -8,7 +8,7 @@ scalaVersion := "2.12.10"
 
 spName := "amplab/opaque"
 
-sparkVersion := "3.0.0"
+sparkVersion := "3.0.1"
 
 sparkComponents ++= Seq("core", "sql", "catalyst")
 

src/enclave/App/App.cpp

@@ -519,7 +519,7 @@ JNIEXPORT jbyteArray JNICALL Java_edu_berkeley_cs_rise_opaque_execution_SGXEncla
 }
 
 JNIEXPORT jbyteArray JNICALL
-Java_edu_berkeley_cs_rise_opaque_execution_SGXEnclave_ScanCollectLastPrimary(
+Java_edu_berkeley_cs_rise_opaque_execution_SGXEnclave_NonObliviousSortMergeJoin(
   JNIEnv *env, jobject obj, jlong eid, jbyteArray join_expr, jbyteArray input_rows) {
   (void)obj;
 
@@ -535,16 +535,16 @@ Java_edu_berkeley_cs_rise_opaque_execution_SGXEnclave_ScanCollectLastPrimary(
   size_t output_rows_length = 0;
 
   if (input_rows_ptr == nullptr) {
-    ocall_throw("ScanCollectLastPrimary: JNI failed to get input byte array.");
+    ocall_throw("NonObliviousSortMergeJoin: JNI failed to get input byte array.");
   } else {
-    oe_check_and_time("Scan Collect Last Primary",
-                       ecall_scan_collect_last_primary(
+    oe_check_and_time("Non-Oblivious Sort-Merge Join",
+                       ecall_non_oblivious_sort_merge_join(
                          (oe_enclave_t*)eid,
                          join_expr_ptr, join_expr_length,
                          input_rows_ptr, input_rows_length,
                          &output_rows, &output_rows_length));
   }
-
+  
   jbyteArray ret = env->NewByteArray(output_rows_length);
   env->SetByteArrayRegion(ret, 0, output_rows_length, (jbyte *) output_rows);
   free(output_rows);
@@ -556,44 +556,45 @@ Java_edu_berkeley_cs_rise_opaque_execution_SGXEnclave_ScanCollectLastPrimary(
 }
 
 JNIEXPORT jbyteArray JNICALL
-Java_edu_berkeley_cs_rise_opaque_execution_SGXEnclave_NonObliviousSortMergeJoin(
-  JNIEnv *env, jobject obj, jlong eid, jbyteArray join_expr, jbyteArray input_rows,
-  jbyteArray join_row) {
+Java_edu_berkeley_cs_rise_opaque_execution_SGXEnclave_BroadcastNestedLoopJoin(
+  JNIEnv *env, jobject obj, jlong eid, jbyteArray join_expr, jbyteArray outer_rows, jbyteArray inner_rows) {
   (void)obj;
 
   jboolean if_copy;
 
   uint32_t join_expr_length = (uint32_t) env->GetArrayLength(join_expr);
   uint8_t *join_expr_ptr = (uint8_t *) env->GetByteArrayElements(join_expr, &if_copy);
 
-  uint32_t input_rows_length = (uint32_t) env->GetArrayLength(input_rows);
-  uint8_t *input_rows_ptr = (uint8_t *) env->GetByteArrayElements(input_rows, &if_copy);
+  uint32_t outer_rows_length = (uint32_t) env->GetArrayLength(outer_rows);
+  uint8_t *outer_rows_ptr = (uint8_t *) env->GetByteArrayElements(outer_rows, &if_copy);
 
-  uint32_t join_row_length = (uint32_t) env->GetArrayLength(join_row);
-  uint8_t *join_row_ptr = (uint8_t *) env->GetByteArrayElements(join_row, &if_copy);
+  uint32_t inner_rows_length = (uint32_t) env->GetArrayLength(inner_rows);
+  uint8_t *inner_rows_ptr = (uint8_t *) env->GetByteArrayElements(inner_rows, &if_copy);
 
   uint8_t *output_rows = nullptr;
   size_t output_rows_length = 0;
 
-  if (input_rows_ptr == nullptr) {
-    ocall_throw("NonObliviousSortMergeJoin: JNI failed to get input byte array.");
+  if (outer_rows_ptr == nullptr) {
+    ocall_throw("BroadcastNestedLoopJoin: JNI failed to get inner byte array.");
+  } else if (inner_rows_ptr == nullptr) {
+    ocall_throw("BroadcastNestedLoopJoin: JNI failed to get outer byte array.");
   } else {
-    oe_check_and_time("Non-Oblivious Sort-Merge Join",
-                       ecall_non_oblivious_sort_merge_join(
+    oe_check_and_time("Broadcast Nested Loop Join",
+                       ecall_broadcast_nested_loop_join(
                          (oe_enclave_t*)eid,
                          join_expr_ptr, join_expr_length,
-                         input_rows_ptr, input_rows_length,
-                         join_row_ptr, join_row_length,
+                         outer_rows_ptr, outer_rows_length,
+                         inner_rows_ptr, inner_rows_length,
                          &output_rows, &output_rows_length));
   }
-  
+
   jbyteArray ret = env->NewByteArray(output_rows_length);
   env->SetByteArrayRegion(ret, 0, output_rows_length, (jbyte *) output_rows);
   free(output_rows);
 
   env->ReleaseByteArrayElements(join_expr, (jbyte *) join_expr_ptr, 0);
-  env->ReleaseByteArrayElements(input_rows, (jbyte *) input_rows_ptr, 0);
-  env->ReleaseByteArrayElements(join_row, (jbyte *) join_row_ptr, 0);
+  env->ReleaseByteArrayElements(outer_rows, (jbyte *) outer_rows_ptr, 0);
+  env->ReleaseByteArrayElements(inner_rows, (jbyte *) inner_rows_ptr, 0);
 
   return ret;
 }

src/enclave/App/CMakeLists.txt

@@ -7,7 +7,10 @@ set(SOURCES
   ${CMAKE_CURRENT_BINARY_DIR}/Enclave_u.c)
 
 add_custom_command(
-  COMMAND oeedger8r --untrusted ${CMAKE_SOURCE_DIR}/Enclave/Enclave.edl --search-path ${CMAKE_SOURCE_DIR}/Enclave
+  COMMAND oeedger8r --untrusted ${CMAKE_SOURCE_DIR}/Enclave/Enclave.edl
+          --search-path ${CMAKE_SOURCE_DIR}/Enclave
+          --search-path ${OE_INCLUDEDIR}
+          --search-path ${OE_INCLUDEDIR}/openenclave/edl/sgx
   DEPENDS ${CMAKE_SOURCE_DIR}/Enclave/Enclave.edl
   OUTPUT ${CMAKE_CURRENT_BINARY_DIR}/Enclave_u.h ${CMAKE_CURRENT_BINARY_DIR}/Enclave_u.c ${CMAKE_CURRENT_BINARY_DIR}/Enclave_args.h)
 
@@ -22,6 +25,6 @@ if ("$ENV{MODE}" STREQUAL "SIMULATE")
   target_compile_definitions(enclave_jni PUBLIC -DSIMULATE)
 endif()
 
-target_link_libraries(enclave_jni openenclave::oehost openenclave::oehostverify)
+target_link_libraries(enclave_jni openenclave::oehost)
 
 install(TARGETS enclave_jni DESTINATION lib)

src/enclave/App/SGXEnclave.h

@@ -38,11 +38,11 @@ extern "C" {
     JNIEnv *, jobject, jlong, jbyteArray, jbyteArray);
 
   JNIEXPORT jbyteArray JNICALL
-  Java_edu_berkeley_cs_rise_opaque_execution_SGXEnclave_ScanCollectLastPrimary(
+  Java_edu_berkeley_cs_rise_opaque_execution_SGXEnclave_NonObliviousSortMergeJoin(
     JNIEnv *, jobject, jlong, jbyteArray, jbyteArray);
 
   JNIEXPORT jbyteArray JNICALL
-  Java_edu_berkeley_cs_rise_opaque_execution_SGXEnclave_NonObliviousSortMergeJoin(
+  Java_edu_berkeley_cs_rise_opaque_execution_SGXEnclave_BroadcastNestedLoopJoin(
     JNIEnv *, jobject, jlong, jbyteArray, jbyteArray, jbyteArray);
 
   JNIEXPORT jobject JNICALL

src/enclave/CMakeLists.txt

@@ -1,13 +1,17 @@
 cmake_minimum_required(VERSION 3.13)
 
 project(OpaqueEnclave)
-
 enable_language(ASM)
 
 option(FLATBUFFERS_LIB_DIR "Location of Flatbuffers library headers.")
 option(FLATBUFFERS_GEN_CPP_DIR "Location of Flatbuffers generated C++ files.")
 
-find_package(OpenEnclave CONFIG REQUIRED)
+set(OE_MIN_VERSION 0.12.0)
+find_package(OpenEnclave ${OE_MIN_VERSION} CONFIG REQUIRED)
+
+set(OE_CRYPTO_LIB
+    mbed
+    CACHE STRING "Crypto library used by enclaves.")
 
 include_directories(App)
 include_directories(${CMAKE_BINARY_DIR}/App)
@@ -18,7 +22,7 @@ include_directories(${CMAKE_BINARY_DIR}/Enclave)
 include_directories(ServiceProvider)
 include_directories(${FLATBUFFERS_LIB_DIR})
 include_directories(${FLATBUFFERS_GEN_CPP_DIR})
-include_directories("/opt/openenclave/include")
+include_directories(${OE_INCLUDEDIR})
 
 if(CMAKE_SIZEOF_VOID_P EQUAL 4)
   set(CMAKE_C_FLAGS "${CMAKE_C_FLAGS} -m32")
@@ -31,14 +35,11 @@ set(CMAKE_CXX_FLAGS_DEBUG "${CMAKE_CXX_FLAGS_DEBUG} -O0 -g -DDEBUG -UNDEBUG -UED
 set(CMAKE_CXX_FLAGS_RELEASE "${CMAKE_CXX_FLAGS_RELEASE} -O2 -DNDEBUG -DEDEBUG -UDEBUG")
 set(CMAKE_CXX_FLAGS_PROFILE "${CMAKE_CXX_FLAGS_PROFILE} -O2 -DNDEBUG -DEDEBUG -UDEBUG -DPERF")
 
-message("openssl rsa -in $ENV{OPAQUE_HOME}/private_key.pem -pubout -out $ENV{OPAQUE_HOME}/public_key.pub")
-message("$ENV{OPAQUE_HOME}/public_key.pub")
-
 add_custom_target(run ALL
     DEPENDS $ENV{OPAQUE_HOME}/public_key.pub)
 
 add_custom_command(
-  COMMAND openssl rsa -in $ENV{OPAQUE_HOME}/private_key.pem -pubout -out $ENV{OPAQUE_HOME}/public_key.pub
+  COMMAND openssl rsa -in $ENV{PRIVATE_KEY_PATH} -pubout -out $ENV{OPAQUE_HOME}/public_key.pub
   OUTPUT $ENV{OPAQUE_HOME}/public_key.pub)
 
 add_subdirectory(App)