updated readme

Author: octaviansima

README.md

@@ -61,14 +61,16 @@ After downloading the Opaque codebase, build and test it as follows.
 
 Next, run Apache Spark SQL queries with Opaque as follows, assuming [Spark 3.0](https://www.apache.org/dyn/closer.lua/spark/spark-3.0.1/spark-3.0.1-bin-hadoop2.7.tgz) (`wget http://apache.mirrors.pair.com/spark/spark-3.0.1/spark-3.0.1-bin-hadoop2.7.tgz`) is already installed:
 
-1. Package Opaque into a JAR:
+\* Opaque needs Spark's `'spark.executor.instances'` property to be set. This can be done in a custom config file, the default config file found at `/opt/spark/conf/spark-defaults.conf`, or as a `spark-submit` or `spark-shell` argument: `--conf 'spark.executor.instances=<value>`.
+
+2. Package Opaque into a JAR:
 
     ```sh
     cd ${OPAQUE_HOME}
     build/sbt package
     ```
 
-2. Launch the Spark shell with Opaque:
+3. Launch the Spark shell with Opaque:
 
     ```sh
     ${SPARK_HOME}/bin/spark-shell --jars ${OPAQUE_HOME}/target/scala-2.12/opaque_2.12-0.1.jar
@@ -81,23 +83,23 @@ Next, run Apache Spark SQL queries with Opaque as follows, assuming [Spark 3.0](
     JVM_OPTS="-Xmx4G" build/sbt console
     ```
 
-3. Inside the Spark shell, import Opaque's DataFrame methods and install Opaque's query planner rules:
+4. Inside the Spark shell, import Opaque's DataFrame methods and install Opaque's query planner rules:
 
     ```scala
     import edu.berkeley.cs.rise.opaque.implicits._
 
     edu.berkeley.cs.rise.opaque.Utils.initSQLContext(spark.sqlContext)
     ```
 
-4. Create an encrypted DataFrame:
+5. Create an encrypted DataFrame:
 
     ```scala
     val data = Seq(("foo", 4), ("bar", 1), ("baz", 5))
     val df = spark.createDataFrame(data).toDF("word", "count")
     val dfEncrypted = df.encrypted
     ```
 
-5. Query the DataFrames and explain the query plan to see the secure operators:
+6. Query the DataFrames and explain the query plan to see the secure operators:
 
 
     ```scala
@@ -117,7 +119,7 @@ Next, run Apache Spark SQL queries with Opaque as follows, assuming [Spark 3.0](
     // +----+-----+
     ```
 
-6. Save and load an encrypted DataFrame:
+7. Save and load an encrypted DataFrame:
 
     ```scala
     dfEncrypted.write.format("edu.berkeley.cs.rise.opaque.EncryptedSource").save("dfEncrypted")

src/main/scala/edu/berkeley/cs/rise/opaque/RA.scala

@@ -28,13 +28,13 @@ import edu.berkeley.cs.rise.opaque.execution.SP
 object RA extends Logging {
   def initRA(sc: SparkContext): Unit = {
 
-    // This is required in order for all executors
-    // to have time to start up. Otherwise, getExecutorMemoryStatus
-    // below will not be initialized to the correct value and
-    // all enclaves won't be attested successfully
-    Thread.sleep(5000)
+    // All executors need to be initialized before attestation can occur
+    var numExecutors = 1
+    if (!sc.isLocal) {
+      numExecutors = sc.getConf.getInt("spark.executor.instances", -1)
+      while (!sc.isLocal && sc.getExecutorMemoryStatus.size < numExecutors) {}
+    }
 
-    val numExecutors = sc.getExecutorMemoryStatus.size
     val rdd = sc.parallelize(Seq.fill(numExecutors) {()}, numExecutors)
     val intelCert = Utils.findResource("AttestationReportSigningCACert.pem")
     val sp = new SP()