Skip to content

Commit 0d69b7b

Browse files
authored
Documentation update (#148)
1 parent 0519def commit 0d69b7b

File tree

1 file changed

+39
-3
lines changed

1 file changed

+39
-3
lines changed

README.md

Lines changed: 39 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -8,11 +8,12 @@ Opaque is a package for Apache Spark SQL that enables encryption for DataFrames
88

99
This project is based on the following NSDI 2017 paper [1]. The oblivious execution mode is not included in this release.
1010

11-
This is an alpha preview of Opaque, which means the software is still in development (not production-ready!). It currently has the following limitations:
11+
This is an alpha preview of Opaque, but the software is still in active development. It currently has the following limitations:
1212

1313
- Unlike the Spark cluster, the master must be run within a trusted environment (e.g., on the client).
1414

15-
- Not all Spark SQL operations are supported. UDFs must be [implemented in C++](#user-defined-functions-udfs).
15+
- Not all Spark SQL operations are supported (see the [list of supported operations](#supported-functionalities)).
16+
UDFs must be [implemented in C++](#user-defined-functions-udfs).
1617

1718
- Computation integrity verification (section 4.2 of the NSDI paper) is currently work in progress.
1819

@@ -136,6 +137,41 @@ Next, run Apache Spark SQL queries with Opaque as follows, assuming [Spark 3.0](
136137
// | baz| 5|
137138
// +----+-----+
138139
```
140+
141+
## Supported functionalities
142+
143+
This section lists Opaque's supported functionalities, which is a subset of that of Spark SQL. Note that the syntax for these functionalities is the same as Spark SQL -- Opaque simply replaces the execution to work with encrypted data.
144+
145+
### Data types
146+
Out of the existing [Spark SQL types](https://spark.apache.org/docs/latest/sql-ref-datatypes.html), Opaque supports
147+
148+
- All numeric types except `DecimalType`, which is currently converted into `FloatType`
149+
- `StringType`
150+
- `BinaryType`
151+
- `BooleanType`
152+
- `TimestampTime`, `DateType`
153+
- `ArrayType`, `MapType`
154+
155+
### Functions
156+
We currently support a subset of the Spark SQL functions, including both scalar and aggregate-like functions.
157+
158+
- Scalar functions: `case`, `cast`, `concat`, `contains`, `if`, `in`, `like`, `substring`, `upper`
159+
- Aggregate functions: `average`, `count`, `first`, `last`, `max`, `min`, `sum`
160+
161+
UDFs are not supported directly, but one can [extend Opaque with additional functions](#user-defined-functions-udfs) by writing it in C++.
162+
163+
164+
### Operators
165+
166+
Opaque supports the core SQL operators:
167+
168+
- Projection
169+
- Filter
170+
- Global aggregation and grouping aggregation
171+
- Order by, sort by
172+
- Inner join
173+
- Limit
174+
139175
140176
## User-Defined Functions (UDFs)
141177
@@ -171,4 +207,4 @@ Now we can port this UDF to Opaque as follows:
171207
172208
## Contact
173209
174-
If you want to know more about our project or have questions, please contact Wenting ([email protected]) and/or Ankur ([email protected]).
210+
If you want to know more about our project or have questions, please contact Wenting ([email protected]) and/or Ankur ([email protected]).

0 commit comments

Comments
 (0)