# Package oracle.nosql.driver

Contains the public API for using the Oracle NoSQL Database as well as configuration and common parameter classes used in database operations. This package contains the major configuration and common operational classes and interfaces, as well as exception classes. Value classes used for data are in the values package. Request and result objects used for individual operations are in the ops package.

This and other packages in the system support both users of the Oracle NoSQL Database Cloud Service and the on-premise Oracle NoSQL Database. Some classes, methods, and parameters are specific to each environment. The documentation for affected classes and methods notes whether there are environment-specific considerations. Unless otherwise noted they are applicable to both environments. The differences mostly related to authentication models, encapsulated in `AuthorizationProvider` and resource constraints and limits in the Cloud Service that are not present on-premise.

The overall flow of a driver application is:

1. Configure the handle to use, including the endpoint for the server to use. The configuration object, `NoSQLHandleConfig`, has additional configuration options.
2. Use the configuration object to obtain a `NoSQLHandle` instance.
3. All data operations are methods on `NoSQLHandle`. They all have the same pattern of:
   - Create and configure Request instance for the operations
   - Call the appropriate method on `NoSQLHandle` for the request
   - Process results in the Result object
   Errors are thrown as exceptions. Many exceptions specific to the system are instances of `NoSQLException`, but common Java exceptions such as `IllegalArgumentException` are thrown directly.

Instances of `NoSQLHandle` are thread-safe and intended to be shared in a multi-threaded application. They are associated 1:1 with an identity so they cannot be shared by different users. While they are not extremely expensive, they have a connection pool and thread pool and are not intended to be disposable.

Configuration and Multi-threaded Applications

High performance multi-threaded applications may benefit from using some non-default configuration options related to how threads and connections are managed by the networking implementation (Netty).

There are 3 methods on `NoSQLHandleConfig` that relate to threads and connections. If your application isn't getting the performance expected these can be tuned. There is no single answer to what is best. It is best to experiment with different values and observe the behavior.

1. `NoSQLHandleConfig.setNumThreads(int)`. This is the number threads that the implementation uses to handle connections. By default it is set to the number of available CPUs * 2. This is the Netty default. Unless your application has high latency operations this should be sufficient.
2. `NoSQLHandleConfig.setConnectionPoolSize(int)`. This is the default size of the Netty connection pool. It also defaults to the number of available CPUs * 2. Again, unless you have high latency operations (long queries) this should be sufficient for most needs.
3. `NoSQLHandleConfig.setPoolMaxPending(int)`. This parameter is used by Netty to control the number of threads waiting for a connections from the pool. If your applications encounter exceptions of the nature, "too many outstanding acquires" this is the cause. The solution is usually to increase the size of the connection pool as this error means that your application has significantly more threads attempting to send requests than there are connections available.

When tuning multi-threaded applications it's important to remember that it's possible to have too many threads and that more threads does not equal more performance. The optimal number depends on request latency and other I/O performed. It is best to experiment.

Logging in the SDK

The SDK uses logging as provided by the java.util.logging package. By default the SDK will log to the console at level INFO. If nothing goes wrong there is little or no logging. There are 2 simple ways to configure logging if additional information is desired.

1. Create and use a logging configuration file and pass it to the application using a system property. E.g.

            $ java -Djava.util.logging.config.file=path.to.logging.properties ...
         or if using mvn:exec
            $ mvn exec:java -Djava.util.logging.config.file=path.to.logging.properties ...

   The content of the properties file is as documented by java.util.logging. See below for an example. This mechanism makes sense if the application doesn't have its own logging needs.
2. Creating an instance of `Logger` in the application and passing it to the SDK using `NoSQLHandleConfig.setLogger(java.util.logging.Logger)`. This option can be used if the application wants to create its own custom logging configuration. It can also use a logging configuration file.

Here is an example of a logging properties file that will log both the SDK and the underlying Netty networking at level FINE. It will log to both the console and a local file called "driver.log."

    handlers=java.util.logging.FileHandler, java.util.logging.ConsoleHandler

    # File config
    java.util.logging.FileHandler.level=ALL
    java.util.logging.FileHandler.formatter=java.util.logging.SimpleFormatter
    java.util.logging.FileHandler.pattern=driver.log
    java.util.logging.FileHandler.count=1
    java.util.logging.FileHandler.limit=50000

    # Console config
    java.util.logging.ConsoleHandler.level=ALL
    java.util.logging.ConsoleHandler.formatter=java.util.logging.SimpleFormatter

    # Use a non-default, single-line, simple format pattern
    java.util.logging.SimpleFormatter.format=%1$tF %1$tT %4$-7s %5$s %n

    # Level can be SEVERE, WARNING, INFO, FINE, ALL, OFF
    oracle.nosql.level=FINE
    io.netty.level=FINE

Logging internal SDK statistics

There are 4 profiles of statistics information that can be logged: none, regular, more, all. For none, which is the default, no statistics are collected and there are no messages logged.

For regular, more and all profiles, the statistics data is collected for an interval of time. At the end of the interval, the stats data is logged in a specified JSON format that can be filtered and parsed. At the end of each interval, after the logging, the counters are cleared and collection of data resumes.

Collection intervals are aligned to the top of the hour. This means first interval logs may contain stats for a shorter interval.

Collection of stats are controlled by the following system properties:

1. -Dcom.oracle.nosql.sdk.nosqldriver.stats.profile=[none|regular|more|all] Specifies the stats profile: none - disabled, regular - per request: counters, errors, latencies, delays, retries more - stats above with 95th and 99th percentile latencies all - stats above with per query information.
2. -Dcom.oracle.nosql.sdk.nosqldriver.stats.interval=600 Interval in seconds to log the stats, by default is 10 minutes.
3. -Dcom.oracle.nosql.sdk.nosqldriver.stats.pretty-print=true Option to enable pretty printing of the JSON data, default value is false
4. -Dcom.oracle.nosql.sdk.nosqldriver.stats.enable-log=false Option to turn on logging automatically if stats are enabled, default value is true

Statistics can also be enabled by using the API: `NoSQLHandleConfig.setStatsProfile(StatsControl.Profile)` or `StatsControl.setProfile(StatsControl.Profile)`. At runtime stats collection can be used selectively by using `StatsControl.start()` and `StatsControl.stop()`. The following example shows how to use a stats handler:

         NoSQLHandleConfig config = new NoSQLHandleConfig( endpoint );
         config.setStatsProfile(StatsControl.Profile.REGULAR);
         config.setStatsInterval(600);
         config.setStatsPrettyPrint(false);
         config.setStatsHandler(
             new StatsControl.StatsHandler() {
                 public void accept(FieldValue jsonStats) {
                     System.out.println("!!! Got a stat: " + jsonStats);
                 }
             });
         NoSQLHandle handle = NoSQLHandleFactory.createNoSQLHandle(config);

         StatsControl statsControl = handle.getStatsControl();

         //... application code without stats

         // enable observations
         statsControl.start();

         //... application code with REGULAR stats

         // For particular parts of code profile can be changed collect more stats.
         statsControl.setProfile(StatsControl.Profile.ALL)
         //... more sensitive code with ALL stats
         statsControl.setProfile(StatsControl.Profile.REGULAR)

         //... application code with REGULAR stats

         // disable observations
         statsControl.stop();

         // ... application code without stats
         handle.close();

The following is an example of stats log entry using the ALL profile:

1. A one time entry containing stats id and options:

       INFO: Client stats|{    // INFO log entry
           "sdkName" : "Oracle NoSQL SDK for Java",  // SDK name
           "sdkVersion" : "current",                 // SDK version
           "clientId" : "f595b333",                  // NoSQLHandle id
           "profile" : "ALL",                        // stats profile
           "intervalSec" : 600,                      // interval length in seconds
           "prettyPrint" : true,                     // JSON pretty print
           "rateLimitingEnabled" : false}            // if rate limiting is
           enabled

2. An entry at the end of each interval containing the stats values:

       INFO: Client stats|{
          "clientId" : "b7bc7734",              // id of NoSQLHandle object
          "startTime" : "2021-09-20T20:11:42Z", // UTC start interval time
          "endTime" : "2021-09-20T20:11:47Z",   // UTC end interval time
          "requests" : [{                       // array of types of requests
            "name" : "Get",                       // stats for GET request type
            "httpRequestCount" : 2,               // count of http requests
            "errors" : 0,                         // number of errors in interval
            "httpRequestLatencyMs" : {            // response time of http requests
              "min" : 4,                            // minimum value in interval
              "avg" : 4.5,                          // average value in interval
              "max" : 5,                            // maximum value in interval
              "95th" : 5,                           // 95th percentile value
              "99th" : 5                            // 99th percentile value
            },
            "requestSize" : {                     // http request size in bytes
              "min" : 42,                           // minimum value in interval
              "avg" : 42.5,                         // average value in interval
              "max" : 43                            // maximum value in interval
            },
            "resultSize" : {                      // http result size in bytes
              "min" : 193,                          // minimum value in interval
              "avg" : 206.5,                        // average value in interval
              "max" : 220                           // maximum value in interval
            },
            "rateLimitDelayMs" : 0,               // delay in milliseconds introduced by the rate limiter
            "retry" : {                           // retries
              "delayMs" : 0,                        // delay in milliseconds introduced by retries
              "authCount" : 0,                      // no of auth retries
              "throttleCount" : 0,                  // no of throttle retries
              "count" : 0                           // total number of retries
            }
          }, {
            "name" : "Query",                   // stats for all QUERY type requests
            "httpRequestCount" : 14,
            "errors" : 0,
            "httpRequestLatencyMs" : {
              "min" : 3,
              "avg" : 13.0,
              "max" : 32,
              "95th" : 32,
              "99th" : 32
            },
            "resultSize" : {
              "min" : 146,
              "avg" : 7379.71,
              "max" : 10989
            },
            "requestSize" : {
              "min" : 65,
              "avg" : 709.85,
              "max" : 799
            },
            "rateLimitDelayMs" : 0,
            "retry" : {
              "delayMs" : 0,
              "authCount" : 0,
              "throttleCount" : 0,
              "count" : 0
            }
          }, {
            "name" : "Put",                    // stats for PUT type requests
            "httpRequestCount" : 1002,
            "errors" : 0,
            "httpRequestLatencyMs" : {
              "min" : 1,
              "avg" : 4.41,
              "max" : 80,
              "95th" : 8,
              "99th" : 20
            },
            "requestSize" : {
              "min" : 90,
              "avg" : 90.16,
              "max" : 187
            },
            "resultSize" : {
              "min" : 58,
              "avg" : 58.0,
              "max" : 58
            },
            "rateLimitDelayMs" : 0,
            "retry" : {
              "delayMs" : 0,
              "authCount" : 0,
              "throttleCount" : 0,
              "count" : 0
            }
          }],
          "queries" : [{            // query stats aggregated by query statement
                                      // query statement
            "query" : "SELECT * FROM audienceData ORDER BY cookie_id",
                                      // query plan description
            "plan" : "SFW([6])\n[\n  FROM:\n  RECV([3])\n  [\n    DistributionKind :
            ALL_PARTITIONS,\n    Sort Fields : sort_gen,\n\n  ] as $from-0\n\n
            SELECT:\n  FIELD_STEP([6])\n  [\n    VAR_REF($from-0)([3]),\n
            audienceData\n  ]\n]",
            "doesWrites" : false,
            "httpRequestCount" : 12,  // number of http calls to the server
            "unprepared" : 1,         // number of query requests without prepare
            "simple" : false,         // type of query
            "count" : 20,             // number of handle.query() API calls
            "errors" : 0,             // number of calls throwing exception
            "httpRequestLatencyMs" : {// response time of http requests in milliseconds
              "min" : 8,                // minimum value in interval
              "avg" : 14.58,            // average value in interval
              "max" : 32,               // maximum value in interval
              "95th" : 32,              // 95th percentile value in interval
              "99th" : 32               // 99th percentile value in interval
            },
            "requestSize" : {         // http request size in bytes
              "min" : 65,               // minimum value in interval
              "avg" : 732.5,            // average value in interval
              "max" : 799               // maximum value in interval
            },
            "resultSize" : {          // http result size in bytes
              "min" : 914,              // minimum value in interval
              "avg" : 8585.33,          // average value in interval
              "max" : 10989             // maximum value in interval
            },
            "rateLimitDelayMs" : 0,   // total delay introduced by rate limiter in milliseconds
            "retry" : {               // automatic retries
              "delayMs" : 0,            // delay introduced by retries
              "authCount" : 0,          // count of auth related retries
              "throttleCount" : 0,      // count of throttle related retries
              "count" : 0               // total count of retries
            }
          }],
          "connections" : {           // concurrent opened connections
            "min" : 1,                  // minimum value in interval
            "avg" : 9.58,               // average value in interval
            "max" : 10                  // maximum value in interval
          }
        }

The log entries go to the logger configured in NoSQLHandlerConfig. For details on how to configure logger, for example to output to a file, see previous paragraph: "Logging in the SDK" . By default, if no logger is configured the statistics entries, if enabled, will be logged to the console.

Stats collection is not dependent of logging configuration, even if logging is disabled, collection of stats will still happen if stats profile other than none is used. In this case, the stats are available by using the stats handler.

Depending on the type of query, if client processing is required, for example in the case of ordered or aggregate queries, indicated by a false `simple` field, the `count` and `httpRequestsCount` numbers will differ. `count` represents the number of `handle.query()` API calls and `httpRequestCount` represents the number of internal http requests from server. For these type of queries, the driver executes several simpler queries, per shard or partition, and than combines the results locally.

- Interface Summary
  Interface | Description
  --- | ---
  AuthorizationProvider | A callback interface used by the driver to obtain an authorization string for a request.
  NoSQLHandle | NoSQLHandle is a handle that can be used to access Oracle NoSQL tables.
  RateLimiter | RateLimiter interface provides default methods that all rate limiters must implement.
  RetryHandler | RetryHandler is called by the request handling system when a `RetryableException` is thrown.
  StatsControl | This interface allows user to control the collection of driver statistics at runtime.
  StatsControl.StatsHandler | Handler interface that user can register to get access to stats at the end of the interval.

- Class Summary
  Class | Description
  --- | ---
  DefaultRetryHandler | Default retry handler.
  Durability | Defines the durability characteristics associated with a standalone write (put or update) operation.
  FieldRange | FieldRange defines a range of values to be used in a `NoSQLHandle.multiDelete(oracle.nosql.driver.ops.MultiDeleteRequest)` operation, as specified in `MultiDeleteRequest.setRange(oracle.nosql.driver.FieldRange)`.
  NoSQLHandleConfig | NoSQLHandleConfig groups parameters used to configure a `NoSQLHandle`.
  NoSQLHandleFactory | Factory class used to produce handles to operate on tables.
  Region | Cloud service only.
  TimeToLive | TimeToLive is a utility class that represents a period of time, similar to java.time.Duration in Java, but specialized to the needs of this driver.
  UserInfo | On-premise only.
  Version | Version is an opaque class that represents the version of a row in the database.

- Enum Summary
  Enum | Description
  --- | ---
  Consistency | Consistency is used to provide consistency guarantees for read operations.
  Durability.ReplicaAckPolicy | A replicated environment makes it possible to increase an application's transaction commit guarantees by committing changes to its replicas on the network.
  Durability.SyncPolicy | Defines the synchronization policy to be used when committing a transaction.
  StatsControl.Profile | The following semantics are attached to the Profile: - NONE: no stats are logged.

- Exception Summary
  Exception | Description
  --- | ---
  BatchOperationNumberLimitException | Cloud service only.
  DeploymentException | Cloud service only.
  EvolutionLimitException | Cloud service only.
  IndexExistsException | The operation attempted to create an index for a table but the named index already exists.
  IndexLimitException | Cloud service only.
  IndexNotFoundException | The operation attempted to access a index that does not exist or is not in a visible state.
  InvalidAuthorizationException | The exception is thrown if the application presents an invalid authorization string in a request.
  JsonParseException | An exception indicating a problem parsing JSON.
  KeySizeLimitException | Cloud service only.
  NoSQLException | A base exception for most exceptions thrown by the driver.
  OperationNotSupportedException | The operation attempted is not supported.
  OperationThrottlingException | Cloud service only.
  ReadThrottlingException | Cloud service only.
  RequestSizeLimitException | Cloud service only.
  RequestTimeoutException | Thrown when a request cannot be processed because the configured timeout interval is exceeded.
  ResourceExistsException | The operation attempted to create a resource but it already exists.
  ResourceLimitException | Cloud service only.
  ResourceNotFoundException | The operation attempted to access a resource that does not exist or is not in a visible state.
  RetryableException | A base exception for all exceptions that may be retried with a reasonable expectation that they may succeed on retry.
  RowSizeLimitException | Cloud service only.
  SecurityInfoNotReadyException | Cloud service only.
  SystemException | An exception that is thrown when there is an internal system problem.
  TableBusyException | Deprecated. this class is no longer used
  TableExistsException | The operation attempted to create a table but the named table already exists.
  TableLimitException | Cloud service only.
  TableNotFoundException | The operation attempted to access a table that does not exist or is not in a visible state.
  TableSizeException | An exception indicating a table size limit has been exceeded by writing more data than the table can support.
  ThrottlingException | Cloud service only.
  UnauthorizedException | The exception is thrown if an application does not have sufficient permission to perform a request.
  UnsupportedProtocolException | This exception is thrown if the server does not support the current driver protocol version.
  WriteThrottlingException | Cloud service only.