Add more detail

Author: Andrew Law

docs/src/integrity/integrity.rst

@@ -11,37 +11,84 @@ shuffling data in an unexpected manner across data partitions, spoofing extra da
 
 Overview
 --------
-The main idea behind integrity support is to tag each step of computation with a MAC over individual enclave workers' encrypted output, attached by the enclave worker when it has completed its computation. 
-All MACs received by all previous enclave workers are logged. In the end during post verification, these MACs, which each represent an ecall at a data partition, are compared and reconstructed into a graph. 
+The main idea behind integrity support is to tag each step of computation with a log over individual enclave workers' encrypted output, attached by the enclave worker when it has completed its computation.
+During execution, each enclave worker checks its input, which contains logs of the previous ecall's output, to make sure that no rows were tampered with, dropped, or spoofed by the job driver.
+This is done using cryptographic MAC functions, whose output can only be computed by the enclave workers sharing a private key with the client. 
+The job driver or server is unable to tamper with the data without being detected, since they are unable to forge a well formed MAC without the private key.
+In the end during post verification, these log objects, called "Crumbs" which each represent an ecall at a data partition, are compared and used to reconstruct a graph representing the flow of information during query execution.
+Specifically, the ``input_macs`` field is matched to other ``all_outputs_mac`` fields of other ecalls to create edges between ecalls and their predecessors.
 This graph is compared to the DAG of the query plan computed by Catalyst. 
 If the graphs are isomorphic, then no tampering has occurred. 
 Else, the result of the query returned by the cloud is rejected.
 
+Logging
+-------
+Below are the flatbuffers schemas of the relevant logging objects used for integrity, which can be found under ``src/flatbuffers/EncryptedBlock.fbs``.
+
+::
+
+    table EncryptedBlocks {
+        blocks:[EncryptedBlock];
+        log:LogEntryChain;
+        log_mac:[Mac]; 
+        all_outputs_mac:[ubyte];
+    }
+
+    table LogEntry {
+        ecall:int; // ecall executed
+        num_macs:int; // Number of EncryptedBlock's in this EncryptedBlocks - checked during runtime
+        mac_lst:[ubyte]; // List of all MACs. one from each EncryptedBlocks - checked during runtime
+        mac_lst_mac:[ubyte]; // MAC(mac_lst) - checked during runtime
+        input_macs:[ubyte]; // List of input EncryptedBlocks' all_output_mac's
+        num_input_macs:int; // Number of input_macs
+    }
+
+    table LogEntryChain {
+        curr_entries:[LogEntry];
+        past_entries:[Crumb];
+    }
+
+    // Contains information about an ecall, which will be pieced together during post verfication to verify the DAG
+    // A crumb is created at an ecall for each previous ecall that sent some data to this ecall
+    table Crumb {
+        input_macs:[ubyte]; // List of EncryptedBlocks all_output_mac's, from LogEntry
+        num_input_macs:int; // Number of input_macs
+        all_outputs_mac:[ubyte]; // MAC over all outputs of ecall from which this EncryptedBlocks came from, of size OE_HMAC_SIZE
+        ecall:int; // Ecall executed
+        log_mac:[ubyte]; // MAC over the LogEntryChain from this EncryptedBlocks, of size OE_HMAC_SIZE
+    }
+
+The ``EncryptedBlocks`` object is what is produced from an enclave worker and passed to the next ecall.
+The ``LogEntry`` object contains information about the current ecall, including its unique integer identifier, MAC outputs over each ``EncryptedBlocks`` it produced, and the ``input_macs`` field, which is a list of the output macs of its predecessor ecall.
+The ``LogEntryChain`` contains a list of log entries for a single data partitions. There will be as many ``LogEntryChain`` objects for a given query as there are data partitions.
+The ``JobVerificationEngine`` has access to a list of ``LogEntryChain``\s. 
+The ``Crumb`` object contains the ``LogEntry`` information of previous ecalls, stored in the ``LogEntryChain``.
+
 Implementation
 --------------
 Two main extensions were made to support integrity - one in enclave code, and one in the Scala client application.
 
 Enclave Code
 ^^^^^^^^^^^^
 In the enclave code (C++), modifications were made to the ``FlatbuffersWriters.cpp`` file and ``FlatbuffersReaders.cpp`` file. 
-The "write" change attaches a MAC over the ``EncryptedBlocks`` object to the output.
-The "read" change checks whether all blocks that were output from the previous ecall were received by the subsequent ecall.
+The "write" change attaches a log to the ``EncryptedBlocks`` object, which contains the enclave worker's encrypted output.
+The "read" change is a runtime check verifying whether all blocks that were output from the previous ecall were received by the subsequent ecall.
 No further modifications need to be made to the application logic since this functionality hooks into how Opaque workers output their data.
 
 Scala/Application Code
 ^^^^^^^^^^^^^^^^^^^^^^
-The main extension supporting Integrity is the ```JobVerificationEngine`` which is a piece of Scala code that broadly carries out three tasks:
+The main extension supporting Integrity is the ``JobVerificationEngine`` which is a piece of Scala code that broadly carries out three tasks:
 
 1. Reconstruct the flow of information between enclave workers.
 
 2. Compute the corresponding DAG of ecalls for a given query.
 
 3. Compare the two DAGs and output "accept" or "reject."
 
-These happen in the "verify" function of the JobVerificationEngine class.
+These happen in the ``verify`` function of the JobVerificationEngine class.
 
-Reconstructing the executed DAG of ecalls involves iterating through the MACs attached by enclave workers, provided in the "LogEntryChain" object in the Job Verification Engine.
-This object is filled by Opaque when Spark's ``collect`` method is called when a query is executed.
+Reconstructing the executed DAG of ecalls involves iterating through the MACs attached by enclave workers, which are fields in the ``Crumb`` and ``LogEntry`` objects stored in each ``LogEntryChain`` in the Job Verification Engine.
+The list of ``LogEntryChain``\s is filled by Opaque when Spark's ``collect`` method is called when a query is executed.
 
 Output MACs of parents correspond to input MACs of their child. Using this information, the DAG is created.
 
@@ -52,13 +99,13 @@ Adding Integrity Support for New Operators
 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 To support new operators, if they are added, one should make changes to the Enclave code and the Job Verification Engine code.
 
-In the enclave, make sure that the enclave context's "finish_ecall" method is called before returning in ``Enclave.cpp```.
+In the enclave, make sure that the enclave context's ``finish_ecall`` method is called before returning in ``Enclave.cpp``.
 
 In the Job Verification Engine, add the logic to transform the operator into a list of ecalls that the operator uses in ``generateJobNodes``.
-This amounts to adding a case in the switch statement of this function.
+This amounts to adding a case in the cascading if/else statement of this function.
 
 Furthermore, add the logic to connect the ecalls together in ``linkEcalls``.
-As above, this amounts to adding a case in the switch statement of this function, but requires knowledge of how each ecall communicates the transfer of data partitions to its successor ecall
+As above, this amounts to adding a case in the cascading if/else statement of this function, but requires knowledge of how each ecall communicates the transfer of data partitions to its successor ecall
 (broadcast, all to one, one to all, etc.).
 
 Usage