External Attributes

Since version 0.4.0 the couchbase-manager manages a new feature which is called external attributes. An external attribute is an attribute value that the manager manages in couchbase as an independent object (previously all the attributes were saved integrated in the session as a unique object). The idea behind this feature is managing lighter sessions, a small session in turn produces faster operations (couchbase operations, serialization, de-serialization, and so on).

Nevertheless externalizing all the attributes in the session is not a good idea (obviously only some attributes are valid candidates to be a separated object in couchbase). In summary this feature is quite complicated and the following points explain how it is implemented in the manager in both configurations (sticky and non-sticky):

• The manager decides if an attribute is externalized using only two parameters: its size and its usage ratio. It is quite clear that a big attribute which is rarely used by the application is a perfect candidate for externalization.

• Being more specific the condition for externalization is the following: any attribute whose size is bigger than a specified amount (manager property attrMaxSize) is being tracked for usage calculation; the usage ratio of a tracked attribute is the percentage of times the attribute is requested by the application; when the ratio is lower than a low limit the attribute is externalized; any externalized attribute can be re-integrated in the session if its ratio goes above another limit (the high limit); ratios are only used if there is a minimum amount of samples. The three commented parameters are specified in the manager using the property attrUsageCondition (this property is in the form {minimum-samples}-{low-limit}-{high-limit} an example is 10-20-40).

• All this calculation is locally done (this values are never persisted in the repository). So each instance performs its own calculations of ratios.

• Externalized attributes are removed from the local session in both configurations after the request (in non-sticky all attributes have been always removed but in sticky all the attributes remained with the local session, so for this configuration there is a change). An externalized attribute is got synchronously if it is requested by the application (in getAttribute method).

• Because of this feature the serialization / de-serialization of the session is completely different now. The version 0.4.0 does not use typical ObjectInputStream or ObjectOutputStream for the session itself, the session is managed by some methods which has been implemented ad hoc for this task. The different properties of the session are written to the byte array and then its attributes. This way if the attribute should be externalized the serialization process is never repeated. For this reason two special IO streams have been implemented (es.rickyepoderi.couchbasemanager.io).

• This feature has modified a lot the couchbase interaction package (es.rickyepoderi.couchbasemanager.couchbase). Now there is a BulkClientRequest class to perform several operations against couchbase at the same time (the operation for the external attributes and the final operation for the session itself). This class lets us launch a bulk operation in two ways: the final operation is sent after the previous ones finish and all at the same time. The wait and launch way is only used in non-sticky configuration when the session is saved (in the non-sticky setup the session is unlocked in the CAS operation, this means that all the previous attribute operations should be finished).

• The attribute operations can be a set, add, touch or delete. The touch is done only for expiration (the external objects follow the same idea of the manager, couchbase rules). In order to avoid touches there is another parameter attrTouchExtraTime, any external object is always set with the session expiration plus this time (during that time the attributes do not need to be touched again).

• Glassfish v4 gave a problem when changing the way the serialization / de-serialization was done. The problem is with CDI, CDI objects in that version check the class loader which stores the object and when the manager de-serialize the object the class loader is different and the following exception is thrown:

  java.lang.IllegalStateException: Singleton not set for org.glassfish.main.core.kernel [53] at org.glassfish.weld.ACLSingletonProvider$ACLSingleton.get(ACLSingletonProvider.java:110) at org.jboss.weld.Container.instance(Container.java:54) at org.jboss.weld.manager.BeanManagerImpl.readResolve(BeanManagerImpl.java:990) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at java.io.ObjectStreamClass.invokeReadResolve(ObjectStreamClass.java:1104)

• In order to fix this problem the workaround to set the application class loader is done (the current is backed up, the app one is set and finally the original loader is restored). It is quite tricky but it works.

Once the external attributes are explained, the properties of the manager related to this feature are going to be listed and described:

• attrMaxSize: Attributes with a size greater than this property value will be tracked to calculate its usage ratio. It is specified in bytes. Default 10240 (10K).
• attrUsageCondition: This property defines when a big attribute (see previous property) its externalized or re-integrated. The attribute is a three integer number: {minimum-samples}-{low-limit}-{high-limit}. The first one (minimum samples) defines the minimum samples taken to start checking if an attribute is externalized. The second number (low limit) is the ratio low limit, when an attribute ratio is below this limit it is externalized. The last one (high limit) is the upper limit, an externalized attribute with a ratio above this limit will be re-integrated in the session object. The default value is 10-10-25 for sticky configuration and 10-30-45 for non-sticky (the sticky configuration has lower limits).
• attrTouchExtraTime: The number of seconds of extra expiration time for external attributes, the external attributes expiration in couchbase is the session expiration plus this number of seconds. It is specified in seconds, default value 600 (10 minutes).