Skip to content

Commit f98f344

Browse files
andrewlawhhAndrew Laweric-feng-2011Eric Fengchester-leung
authored
Merge bnlj (#190)
* Support for multiple branched CaseWhen * Interval (#116) * add date_add, interval sql still running into issues * Add Interval SQL support * uncomment out the other tests * resolve comments * change interval equality Co-authored-by: Eric Feng <[email protected]> * Remove partition ID argument from enclaves * Fix comments * updates * Modifications to integrate crumb, log-mac, and all-outputs_mac, wip * Store log mac after each output buffer, add all-outputs-mac to each encryptedblocks wip * Add all_outputs_mac to all EncryptedBlocks once all log_macs have been generated * Almost builds * cpp builds * Use ubyte for all_outputs_mac * use Mac for all_outputs_mac * Hopefully this works for flatbuffers all_outputs_mac mutation, cpp builds * Scala builds now too, running into error with union * Stuff builds, error with all outputs mac serialization. this commit uses all_outputs_mac as Mac table * Fixed bug, basic encryption / show works * All single partition tests pass, multiple partiton passes until tpch-9 * All tests pass except tpch-9 and skew join * comment tpch back in * Check same number of ecalls per partition - exception for scanCollectLastPrimary(?) * First attempt at constructing executed DAG * Fix typos * Rework graph * Add log macs to graph nodes * Construct expected DAG and refactor JobNode. Refactor construction of executed DAG. * Implement 'paths to sink' for a DAG * add crumb for last ecall * Fix NULL handling for aggregation (#130) * Modify COUNT and SUM to correctly handle NULL values * Change average to support NULL values * Fix * Changing operator matching from logical to physical (#129) * WIP * Fix * Unapply change * Aggregation rewrite (#132) * updated build/sbt file (#135) * Travis update (#137) * update breeze (#138) * TPC-H test suite added (#136) * added tpch sql files * functions updated to save temp view * main function skeleton done * load and clear done * fix clear * performQuery done * import cleanup, use OPAQUE_HOME * TPC-H 9 refactored to use SQL rather than DF operations * removed : Unit, unused imports * added TestUtils.scala * moved all common initialization to TestUtils * update name * begin rewriting TPCH.scala to store persistent tables * invalid table name error * TPCH conversion to class started * compiles * added second case, cleared up names * added TPC-H 6 to check that persistent state has no issues * added functions for the last two tables * addressed most logic changes * DataFrame only loaded once * apply method in companion object * full test suite added * added testFunc parameter to testAgainstSpark * ignore #18 * Separate IN PR (#124) * finishing the in expression. adding more tests and null support. need confirmation on null behavior and also I wonder why integer field is sufficient for string * adding additional test * adding additional test * saving concat implementation and it's passing basic functionality tests * adding type aware comparison and better error message for IN operator * adding null checking for the concat operator and adding one additional test * cleaning up IN&Concat PR * deleting concat and preping the in branch for in pr * fixing null bahavior now it's only null when there's no match and there's null input * Build failed Co-authored-by: Ubuntu <[email protected]> Co-authored-by: Wenting Zheng <[email protected]> Co-authored-by: Wenting Zheng <[email protected]> * Merge new aggregate * Uncomment log_mac_lst clear * Clean up comments * Separate Concat PR (#125) Implementation of the CONCAT expression. Co-authored-by: Ubuntu <[email protected]> Co-authored-by: Wenting Zheng <[email protected]> * Clean up comments in other files * Update pathsEqual to be less conservative * Remove print statements from unit tests * Removed calls to toSet in TPC-H tests (#140) * removed calls to toSet * added calls to toSet back where queries are unordered * Documentation update (#148) * Cluster Remote Attestation Fix (#146) The existing code only had RA working when run locally. This PR adds a sleep for 5 seconds to make sure that all executors are spun up successfully before attestation begins. Closes #147 * upgrade to 3.0.1 (#144) * Update two TPC-H queries (#149) Tests for TPC-H 12 and 19 pass. * TPC-H 20 Fix (#142) * string to stringtype error * tpch 20 passes * cleanup * implemented changes * decimal.tofloat Co-authored-by: Wenting Zheng <[email protected]> * Add expected operator DAG generation from executedPlan string * Rebase * Join update (#145) * Merge join update * Integrate new join * Add expected operator for sortexec * Merge comp-integrity with join update * Remove some print statements * Migrate from Travis CI to Github Actions (#156) * Upgrade to OE 0.12 (#153) * Update README.md * Support for scalar subquery (#157) This PR implements the scalar subquery expression, which is triggered whenever a subquery returns a scalar value. There were two main problems that needed to be solved. First, support for matching the scalar subquery expression is necessary. Spark implements this by wrapping a SparkPlan within the expression and calls executeCollect. Then it constructs a literal with that value. However, this is problematic for us because that value should not be decrypted by the driver and serialized into an expression, since it's an intermediate value. Therefore, the second issue to be addressed here is supporting an encrypted literal. This is implemented in this PR by serializing an encrypted ciphertext into a base64 encoded string, and wrapping a Decrypt expression on top of it. This expression is then evaluated in the enclave and returns a literal. Note that, in order to test our implementation, we also implement a Decrypt expression in Scala. However, this should never be evaluated on the driver side and serialized into a plaintext literal. This is because Decrypt is designated as a Nondeterministic expression, and therefore will always evaluate on the workers. * Add TPC-H Benchmarks (#139) * logic decoupling in TPCH.scala for easier benchmarking * added TPCHBenchmark.scala * Benchmark.scala rewrite * done adding all support TPC-H query benchmarks * changed commandline arguments that benchmark takes * TPCHBenchmark takes in parameters * fixed issue with spark conf * size error handling, --help flag * add Utils.force, break cluster mode * comment out logistic regression benchmark * ensureCached right before temp view created/replaced * upgrade to 3.0.1 * upgrade to 3.0.1 * 10 scale factor * persistData * almost done refactor * more cleanup * compiles * 9 passes * cleanup * collect instead of force, sf_none * remove sf_none * defaultParallelism * no removing trailing/leading whitespace * add sf_med * hdfs works in local case * cleanup, added new CLI argument * added newly supported tpch queries * function for running all supported tests * Construct expected DAG from dataframe physical plan * Refactor collect and add integrity checking helper function to OpaqueOperatorTest * Float expressions (#160) This PR adds float normalization expressions [implemented in Spark](https://github.com/apache/spark/blob/master/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/NormalizeFloatingNumbers.scala#L170). TPC-H query 2 also passes. * Broadcast Nested Loop Join - Left Anti and Left Semi (#159) This PR is the first of two parts towards making TPC-H 16 work: the other will be implementing `is_distinct` for aggregate operations. `BroadcastNestedLoopJoin` is Spark's "catch all" for non-equi joins. It works by first picking a side to broadcast, then iterating through every possible row combination and checking the non-equi condition against the pair. * Remove addExpectedOperator from JobVerificationEngine, add comments * Implement expected DAG construction by doing graph manipulation on dataframe field instead of string parsing * Fix merge errors in the test cases Co-authored-by: Andrew Law <[email protected]> Co-authored-by: Eric Feng <[email protected]> Co-authored-by: Eric Feng <[email protected]> Co-authored-by: Chester Leung <[email protected]> Co-authored-by: Wenting Zheng <[email protected]> Co-authored-by: octaviansima <[email protected]> Co-authored-by: Chenyu Shi <[email protected]> Co-authored-by: Ubuntu <[email protected]> Co-authored-by: Wenting Zheng <[email protected]>
1 parent 6f978a7 commit f98f344

36 files changed

+1155
-232
lines changed

‎.github/scripts/build.sh

Lines changed: 25 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,25 @@
1+
# Install OpenEnclave 0.9.0
2+
echo 'deb [arch=amd64] https://download.01.org/intel-sgx/sgx_repo/ubuntu bionic main' | sudo tee /etc/apt/sources.list.d/intel-sgx.list
3+
wget -qO - https://download.01.org/intel-sgx/sgx_repo/ubuntu/intel-sgx-deb.key | sudo apt-key add -
4+
echo "deb http://apt.llvm.org/bionic/ llvm-toolchain-bionic-7 main" | sudo tee /etc/apt/sources.list.d/llvm-toolchain-bionic-7.list
5+
wget -qO - https://apt.llvm.org/llvm-snapshot.gpg.key | sudo apt-key add -
6+
echo "deb [arch=amd64] https://packages.microsoft.com/ubuntu/18.04/prod bionic main" | sudo tee /etc/apt/sources.list.d/msprod.list
7+
wget -qO - https://packages.microsoft.com/keys/microsoft.asc | sudo apt-key add -
8+
9+
sudo apt update
10+
sudo apt -y install clang-7 libssl-dev gdb libsgx-enclave-common libsgx-enclave-common-dev libprotobuf10 libsgx-dcap-ql libsgx-dcap-ql-dev az-dcap-client open-enclave=0.12.0
11+
12+
# Install Opaque Dependencies
13+
sudo apt -y install wget build-essential openjdk-8-jdk python libssl-dev
14+
15+
wget https://github.com/Kitware/CMake/releases/download/v3.15.6/cmake-3.15.6-Linux-x86_64.sh
16+
sudo bash cmake-3.15.6-Linux-x86_64.sh --skip-license --prefix=/usr/local
17+
18+
# Generate keypair for attestation
19+
openssl genrsa -out ./private_key.pem -3 3072
20+
21+
source opaqueenv
22+
source /opt/openenclave/share/openenclave/openenclaverc
23+
export MODE=SIMULATE
24+
25+
build/sbt test

‎.github/workflows/main.yml

Lines changed: 40 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,40 @@
1+
name: CI
2+
3+
# Controls when the action will run.
4+
on:
5+
# Triggers the workflow on push or pull request events but only for the master branch
6+
push:
7+
branches: [ master ]
8+
pull_request:
9+
branches: [ master ]
10+
11+
# Allows you to run this workflow manually from the Actions tab
12+
workflow_dispatch:
13+
14+
# A workflow run is made up of one or more jobs that can run sequentially or in parallel
15+
jobs:
16+
build:
17+
# Define the OS to run on
18+
runs-on: ubuntu-18.04
19+
# Steps represent a sequence of tasks that will be executed as part of the job
20+
steps:
21+
# Checks-out your repository under $GITHUB_WORKSPACE, so your job can access it
22+
- uses: actions/checkout@v2
23+
# Specify the version of Java that is installed
24+
- uses: actions/setup-java@v1
25+
with:
26+
java-version: '8'
27+
# Caching (from https://www.scala-sbt.org/1.x/docs/GitHub-Actions-with-sbt.html)
28+
- uses: coursier/cache-action@v5
29+
# Run the test
30+
- name: Install dependencies, set environment variables, and run sbt tests
31+
run: |
32+
./.github/scripts/build.sh
33+
34+
rm -rf "$HOME/.ivy2/local" || true
35+
find $HOME/Library/Caches/Coursier/v1 -name "ivydata-*.properties" -delete || true
36+
find $HOME/.ivy2/cache -name "ivydata-*.properties" -delete || true
37+
find $HOME/.cache/coursier/v1 -name "ivydata-*.properties" -delete || true
38+
find $HOME/.sbt -name "*.lock" -delete || true
39+
shell: bash
40+

‎.travis.yml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -16,7 +16,7 @@ before_install:
1616
- sudo apt update
1717
- sudo apt -y install clang-7 libssl-dev gdb libsgx-enclave-common libsgx-enclave-common-dev libprotobuf10 libsgx-dcap-ql libsgx-dcap-ql-dev
1818
- sudo apt-get -y install wget build-essential openjdk-8-jdk python libssl-dev
19-
- sudo apt-get -y install open-enclave=0.9.0
19+
- sudo apt-get -y install open-enclave=0.12.0
2020
- wget https://github.com/Kitware/CMake/releases/download/v3.15.6/cmake-3.15.6-Linux-x86_64.sh
2121
- sudo bash cmake-3.15.6-Linux-x86_64.sh --skip-license --prefix=/usr/local
2222
- export PATH=/usr/local/bin:"$PATH"

‎README.md

Lines changed: 1 addition & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -24,7 +24,7 @@ UDFs must be [implemented in C++](#user-defined-functions-udfs).
2424

2525
After downloading the Opaque codebase, build and test it as follows.
2626

27-
1. Install dependencies and the [OpenEnclave SDK](https://github.com/openenclave/openenclave/blob/v0.9.x/docs/GettingStartedDocs/install_oe_sdk-Ubuntu_18.04.md). We currently support OE version 0.9.0 (so please install with `open-enclave=0.9.0`) and Ubuntu 18.04.
27+
1. Install dependencies and the [OpenEnclave SDK](https://github.com/openenclave/openenclave/blob/v0.12.0/docs/GettingStartedDocs/install_oe_sdk-Ubuntu_18.04.md). We currently support OE version 0.12.0 (so please install with `open-enclave=0.12.0`) and Ubuntu 18.04.
2828

2929
```sh
3030
# For Ubuntu 18.04:
@@ -206,7 +206,3 @@ Now we can port this UDF to Opaque as follows:
206206
```
207207
208208
3. Finally, implement the UDF in C++. In [`FlatbuffersExpressionEvaluator#eval_helper`](src/enclave/Enclave/ExpressionEvaluation.h), add a case for `tuix::ExprUnion_DotProduct`. Within that case, cast the expression to a `tuix::DotProduct`, recursively evaluate the left and right children, perform the dot product computation on them, and construct a `DoubleField` containing the result.
209-
210-
## Contact
211-
212-
If you want to know more about our project or have questions, please contact Wenting ([email protected]) and/or Ankur ([email protected]).

‎src/enclave/App/App.cpp

Lines changed: 44 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -555,6 +555,50 @@ Java_edu_berkeley_cs_rise_opaque_execution_SGXEnclave_NonObliviousSortMergeJoin(
555555
return ret;
556556
}
557557

558+
JNIEXPORT jbyteArray JNICALL
559+
Java_edu_berkeley_cs_rise_opaque_execution_SGXEnclave_BroadcastNestedLoopJoin(
560+
JNIEnv *env, jobject obj, jlong eid, jbyteArray join_expr, jbyteArray outer_rows, jbyteArray inner_rows) {
561+
(void)obj;
562+
563+
jboolean if_copy;
564+
565+
uint32_t join_expr_length = (uint32_t) env->GetArrayLength(join_expr);
566+
uint8_t *join_expr_ptr = (uint8_t *) env->GetByteArrayElements(join_expr, &if_copy);
567+
568+
uint32_t outer_rows_length = (uint32_t) env->GetArrayLength(outer_rows);
569+
uint8_t *outer_rows_ptr = (uint8_t *) env->GetByteArrayElements(outer_rows, &if_copy);
570+
571+
uint32_t inner_rows_length = (uint32_t) env->GetArrayLength(inner_rows);
572+
uint8_t *inner_rows_ptr = (uint8_t *) env->GetByteArrayElements(inner_rows, &if_copy);
573+
574+
uint8_t *output_rows = nullptr;
575+
size_t output_rows_length = 0;
576+
577+
if (outer_rows_ptr == nullptr) {
578+
ocall_throw("BroadcastNestedLoopJoin: JNI failed to get inner byte array.");
579+
} else if (inner_rows_ptr == nullptr) {
580+
ocall_throw("BroadcastNestedLoopJoin: JNI failed to get outer byte array.");
581+
} else {
582+
oe_check_and_time("Broadcast Nested Loop Join",
583+
ecall_broadcast_nested_loop_join(
584+
(oe_enclave_t*)eid,
585+
join_expr_ptr, join_expr_length,
586+
outer_rows_ptr, outer_rows_length,
587+
inner_rows_ptr, inner_rows_length,
588+
&output_rows, &output_rows_length));
589+
}
590+
591+
jbyteArray ret = env->NewByteArray(output_rows_length);
592+
env->SetByteArrayRegion(ret, 0, output_rows_length, (jbyte *) output_rows);
593+
free(output_rows);
594+
595+
env->ReleaseByteArrayElements(join_expr, (jbyte *) join_expr_ptr, 0);
596+
env->ReleaseByteArrayElements(outer_rows, (jbyte *) outer_rows_ptr, 0);
597+
env->ReleaseByteArrayElements(inner_rows, (jbyte *) inner_rows_ptr, 0);
598+
599+
return ret;
600+
}
601+
558602
JNIEXPORT jobject JNICALL
559603
Java_edu_berkeley_cs_rise_opaque_execution_SGXEnclave_NonObliviousAggregate(
560604
JNIEnv *env, jobject obj, jlong eid, jbyteArray agg_op, jbyteArray input_rows, jboolean isPartial) {

‎src/enclave/App/CMakeLists.txt

Lines changed: 5 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -7,7 +7,10 @@ set(SOURCES
77
${CMAKE_CURRENT_BINARY_DIR}/Enclave_u.c)
88

99
add_custom_command(
10-
COMMAND oeedger8r --untrusted ${CMAKE_SOURCE_DIR}/Enclave/Enclave.edl --search-path ${CMAKE_SOURCE_DIR}/Enclave
10+
COMMAND oeedger8r --untrusted ${CMAKE_SOURCE_DIR}/Enclave/Enclave.edl
11+
--search-path ${CMAKE_SOURCE_DIR}/Enclave
12+
--search-path ${OE_INCLUDEDIR}
13+
--search-path ${OE_INCLUDEDIR}/openenclave/edl/sgx
1114
DEPENDS ${CMAKE_SOURCE_DIR}/Enclave/Enclave.edl
1215
OUTPUT ${CMAKE_CURRENT_BINARY_DIR}/Enclave_u.h ${CMAKE_CURRENT_BINARY_DIR}/Enclave_u.c ${CMAKE_CURRENT_BINARY_DIR}/Enclave_args.h)
1316

@@ -22,6 +25,6 @@ if ("$ENV{MODE}" STREQUAL "SIMULATE")
2225
target_compile_definitions(enclave_jni PUBLIC -DSIMULATE)
2326
endif()
2427

25-
target_link_libraries(enclave_jni openenclave::oehost openenclave::oehostverify)
28+
target_link_libraries(enclave_jni openenclave::oehost)
2629

2730
install(TARGETS enclave_jni DESTINATION lib)

‎src/enclave/App/SGXEnclave.h

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -41,6 +41,10 @@ extern "C" {
4141
Java_edu_berkeley_cs_rise_opaque_execution_SGXEnclave_NonObliviousSortMergeJoin(
4242
JNIEnv *, jobject, jlong, jbyteArray, jbyteArray);
4343

44+
JNIEXPORT jbyteArray JNICALL
45+
Java_edu_berkeley_cs_rise_opaque_execution_SGXEnclave_BroadcastNestedLoopJoin(
46+
JNIEnv *, jobject, jlong, jbyteArray, jbyteArray, jbyteArray);
47+
4448
JNIEXPORT jobject JNICALL
4549
Java_edu_berkeley_cs_rise_opaque_execution_SGXEnclave_NonObliviousAggregate(
4650
JNIEnv *, jobject, jlong, jbyteArray, jbyteArray, jboolean);

‎src/enclave/CMakeLists.txt

Lines changed: 8 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -1,13 +1,17 @@
11
cmake_minimum_required(VERSION 3.13)
22

33
project(OpaqueEnclave)
4-
54
enable_language(ASM)
65

76
option(FLATBUFFERS_LIB_DIR "Location of Flatbuffers library headers.")
87
option(FLATBUFFERS_GEN_CPP_DIR "Location of Flatbuffers generated C++ files.")
98

10-
find_package(OpenEnclave CONFIG REQUIRED)
9+
set(OE_MIN_VERSION 0.12.0)
10+
find_package(OpenEnclave ${OE_MIN_VERSION} CONFIG REQUIRED)
11+
12+
set(OE_CRYPTO_LIB
13+
mbed
14+
CACHE STRING "Crypto library used by enclaves.")
1115

1216
include_directories(App)
1317
include_directories(${CMAKE_BINARY_DIR}/App)
@@ -18,7 +22,7 @@ include_directories(${CMAKE_BINARY_DIR}/Enclave)
1822
include_directories(ServiceProvider)
1923
include_directories(${FLATBUFFERS_LIB_DIR})
2024
include_directories(${FLATBUFFERS_GEN_CPP_DIR})
21-
include_directories("/opt/openenclave/include")
25+
include_directories(${OE_INCLUDEDIR})
2226

2327
if(CMAKE_SIZEOF_VOID_P EQUAL 4)
2428
set(CMAKE_C_FLAGS "${CMAKE_C_FLAGS} -m32")
@@ -31,14 +35,11 @@ set(CMAKE_CXX_FLAGS_DEBUG "${CMAKE_CXX_FLAGS_DEBUG} -O0 -g -DDEBUG -UNDEBUG -UED
3135
set(CMAKE_CXX_FLAGS_RELEASE "${CMAKE_CXX_FLAGS_RELEASE} -O2 -DNDEBUG -DEDEBUG -UDEBUG")
3236
set(CMAKE_CXX_FLAGS_PROFILE "${CMAKE_CXX_FLAGS_PROFILE} -O2 -DNDEBUG -DEDEBUG -UDEBUG -DPERF")
3337

34-
message("openssl rsa -in $ENV{OPAQUE_HOME}/private_key.pem -pubout -out $ENV{OPAQUE_HOME}/public_key.pub")
35-
message("$ENV{OPAQUE_HOME}/public_key.pub")
36-
3738
add_custom_target(run ALL
3839
DEPENDS $ENV{OPAQUE_HOME}/public_key.pub)
3940

4041
add_custom_command(
41-
COMMAND openssl rsa -in $ENV{OPAQUE_HOME}/private_key.pem -pubout -out $ENV{OPAQUE_HOME}/public_key.pub
42+
COMMAND openssl rsa -in $ENV{PRIVATE_KEY_PATH} -pubout -out $ENV{OPAQUE_HOME}/public_key.pub
4243
OUTPUT $ENV{OPAQUE_HOME}/public_key.pub)
4344

4445
add_subdirectory(App)

‎src/enclave/Enclave/BroadcastNestedLoopJoin.cpp

Lines changed: 54 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,54 @@
1+
#include "BroadcastNestedLoopJoin.h"
2+
3+
#include "ExpressionEvaluation.h"
4+
#include "FlatbuffersReaders.h"
5+
#include "FlatbuffersWriters.h"
6+
#include "common.h"
7+
8+
/** C++ implementation of a broadcast nested loop join.
9+
* Assumes outer_rows is streamed and inner_rows is broadcast.
10+
* DOES NOT rely on rows to be tagged primary or secondary, and that
11+
* assumption will break the implementation.
12+
*/
13+
void broadcast_nested_loop_join(
14+
uint8_t *join_expr, size_t join_expr_length,
15+
uint8_t *outer_rows, size_t outer_rows_length,
16+
uint8_t *inner_rows, size_t inner_rows_length,
17+
uint8_t **output_rows, size_t *output_rows_length) {
18+
19+
FlatbuffersJoinExprEvaluator join_expr_eval(join_expr, join_expr_length);
20+
const tuix::JoinType join_type = join_expr_eval.get_join_type();
21+
22+
RowReader outer_r(BufferRefView<tuix::EncryptedBlocks>(outer_rows, outer_rows_length));
23+
RowWriter w;
24+
25+
while (outer_r.has_next()) {
26+
const tuix::Row *outer = outer_r.next();
27+
bool o_i_match = false;
28+
29+
RowReader inner_r(BufferRefView<tuix::EncryptedBlocks>(inner_rows, inner_rows_length));
30+
const tuix::Row *inner;
31+
while (inner_r.has_next()) {
32+
inner = inner_r.next();
33+
o_i_match |= join_expr_eval.eval_condition(outer, inner);
34+
}
35+
36+
switch(join_type) {
37+
case tuix::JoinType_LeftAnti:
38+
if (!o_i_match) {
39+
w.append(outer);
40+
}
41+
break;
42+
case tuix::JoinType_LeftSemi:
43+
if (o_i_match) {
44+
w.append(outer);
45+
}
46+
break;
47+
default:
48+
throw std::runtime_error(
49+
std::string("Join type not supported: ")
50+
+ std::string(to_string(join_type)));
51+
}
52+
}
53+
w.output_buffer(output_rows, output_rows_length, std::string("broadcastNestedLoopJoin"));
54+
}

‎src/enclave/Enclave/BroadcastNestedLoopJoin.h

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,8 @@
1+
#include <cstddef>
2+
#include <cstdint>
3+
4+
void broadcast_nested_loop_join(
5+
uint8_t *join_expr, size_t join_expr_length,
6+
uint8_t *outer_rows, size_t outer_rows_length,
7+
uint8_t *inner_rows, size_t inner_rows_length,
8+
uint8_t **output_rows, size_t *output_rows_length);

‎src/enclave/Enclave/CMakeLists.txt

Lines changed: 14 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -11,7 +11,8 @@ set(SOURCES
1111
FlatbuffersReaders.cpp
1212
FlatbuffersWriters.cpp
1313
IntegrityUtils.cpp
14-
Join.cpp
14+
NonObliviousSortMergeJoin.cpp
15+
BroadcastNestedLoopJoin.cpp
1516
Limit.cpp
1617
Project.cpp
1718
Sort.cpp
@@ -23,7 +24,10 @@ set(SOURCES
2324
${CMAKE_CURRENT_BINARY_DIR}/Enclave_t.c)
2425

2526
add_custom_command(
26-
COMMAND oeedger8r --trusted ${CMAKE_SOURCE_DIR}/Enclave/Enclave.edl --search-path ${CMAKE_SOURCE_DIR}/Enclave
27+
COMMAND oeedger8r --trusted ${CMAKE_SOURCE_DIR}/Enclave/Enclave.edl
28+
--search-path ${CMAKE_SOURCE_DIR}/Enclave
29+
--search-path ${OE_INCLUDEDIR}
30+
--search-path ${OE_INCLUDEDIR}/openenclave/edl/sgx
2731
DEPENDS ${CMAKE_SOURCE_DIR}/Enclave/Enclave.edl
2832
OUTPUT ${CMAKE_CURRENT_BINARY_DIR}/Enclave_t.h ${CMAKE_CURRENT_BINARY_DIR}/Enclave_t.c ${CMAKE_CURRENT_BINARY_DIR}/Enclave_args.h)
2933

@@ -42,22 +46,21 @@ endif()
4246
target_compile_definitions(enclave_trusted PUBLIC OE_API_VERSION=2)
4347

4448
# Need for the generated file Enclave_t.h
45-
target_include_directories(enclave_trusted PRIVATE ${CMAKE_CURRENT_BINARY_DIR})
49+
target_include_directories(enclave_trusted PRIVATE ${CMAKE_CURRENT_BINARY_DIR} ${OE_INCLUDEDIR}/openenclave/3rdparty)
4650

47-
target_link_libraries(enclave_trusted
48-
openenclave::oeenclave
49-
openenclave::oelibc
51+
link_directories(${OE_LIBDIR} ${OE_LIBDIR}/openenclave/enclave)
52+
target_link_libraries(enclave_trusted
53+
openenclave::oeenclave
54+
openenclave::oecrypto${OE_CRYPTO_LIB}
55+
openenclave::oelibc
5056
openenclave::oelibcxx
51-
openenclave::oehostsock
52-
openenclave::oehostresolver)
57+
openenclave::oecore)
5358

5459
add_custom_command(
55-
COMMAND oesign sign -e $<TARGET_FILE:enclave_trusted> -c ${CMAKE_CURRENT_SOURCE_DIR}/Enclave.conf -k $ENV{PRIVATE_KEY_PATH}
60+
COMMAND openenclave::oesign sign -e $<TARGET_FILE:enclave_trusted> -c ${CMAKE_CURRENT_SOURCE_DIR}/Enclave.conf -k $ENV{PRIVATE_KEY_PATH}
5661
DEPENDS enclave_trusted ${CMAKE_CURRENT_SOURCE_DIR}/Enclave.conf
5762
OUTPUT ${CMAKE_CURRENT_BINARY_DIR}/enclave_trusted.signed)
5863

59-
# TODO: Use the user-generated private key to sign the enclave code.
60-
# Currently we use the sample private key from the Intel SGX SDK.
6164
add_custom_command(
6265
COMMAND mv ${CMAKE_CURRENT_BINARY_DIR}/libenclave_trusted.so.signed ${CMAKE_CURRENT_BINARY_DIR}/libenclave_trusted_signed.so
6366
DEPENDS ${CMAKE_CURRENT_BINARY_DIR}/enclave_trusted.signed

‎src/enclave/Enclave/Enclave.cpp

Lines changed: 24 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -6,7 +6,8 @@
66
#include "Aggregate.h"
77
#include "Crypto.h"
88
#include "Filter.h"
9-
#include "Join.h"
9+
#include "NonObliviousSortMergeJoin.h"
10+
#include "BroadcastNestedLoopJoin.h"
1011
#include "Limit.h"
1112
#include "Project.h"
1213
#include "Sort.h"
@@ -196,7 +197,6 @@ void ecall_non_oblivious_sort_merge_join(uint8_t *join_expr, size_t join_expr_le
196197
__builtin_ia32_lfence();
197198

198199
try {
199-
debug("Ecall: NonObliviousSortMergJoin\n");
200200
non_oblivious_sort_merge_join(join_expr, join_expr_length,
201201
input_rows, input_rows_length,
202202
output_rows, output_rows_length);
@@ -208,6 +208,28 @@ void ecall_non_oblivious_sort_merge_join(uint8_t *join_expr, size_t join_expr_le
208208
}
209209
}
210210

211+
void ecall_broadcast_nested_loop_join(uint8_t *join_expr, size_t join_expr_length,
212+
uint8_t *outer_rows, size_t outer_rows_length,
213+
uint8_t *inner_rows, size_t inner_rows_length,
214+
uint8_t **output_rows, size_t *output_rows_length) {
215+
// Guard against operating on arbitrary enclave memory
216+
assert(oe_is_outside_enclave(outer_rows, outer_rows_length) == 1);
217+
assert(oe_is_outside_enclave(inner_rows, inner_rows_length) == 1);
218+
__builtin_ia32_lfence();
219+
220+
try {
221+
broadcast_nested_loop_join(join_expr, join_expr_length,
222+
outer_rows, outer_rows_length,
223+
inner_rows, inner_rows_length,
224+
output_rows, output_rows_length);
225+
complete_encrypted_blocks(*output_rows);
226+
EnclaveContext::getInstance().finish_ecall();
227+
} catch (const std::runtime_error &e) {
228+
EnclaveContext::getInstance().finish_ecall();
229+
ocall_throw(e.what());
230+
}
231+
}
232+
211233
void ecall_non_oblivious_aggregate(
212234
uint8_t *agg_op, size_t agg_op_length,
213235
uint8_t *input_rows, size_t input_rows_length,

0 commit comments

Comments
 (0)