~pperalta

One of the major challenges in distributed computing is the maintenance of the application. You need the ability to make changes to the system without breaking the current users of the application. Anyone who uses Java serialization knows that this is a difficult task, especially since the use of java.io.Serializable results in a brittle binary format.

In a previous life when I was the designer and developer of a services-style distributed system, I took a brute force approach. I determined that each version of a service would include a set of serializable objects and service interfaces. If any of the serializable objects needed to change, that would trigger a new version of the services. This meant having to keep several versions running at a time, one to support new clients, and the older ones to support older clients.

I was able to get away with this because the objects in the system didn’t live for very long. The life cycle of an object tended to start either in the DAO layer (Hibernate) and end up being serialized to a client, or a client would send a serialized object to a service which would end up being written to a database using Hibernate. Upon completion of the transaction, there were no longer any references to these objects (L2 caching was not enabled.) As a result, running multiple copies of the application in a single container was feasible as the memory requirements didn’t exceed the capacity of the heap.

An object in a typical Coherence system has a much longer life cycle; many times the objects themselves will need to outlive the specific version of the application that is running. Coherence allows you to do this by performing a rolling restart. This means that instead of taking down the entire cluster to perform an upgrade, you can take down a node, update its classes (and maybe configuration, depending on what is being modified) and restart it. This process is repeated until all nodes have been updated.

However if the objects in the cache have a different serialization format than the updated version of their classes, you can run into problems when attempting to deserialize a new data format with an old class. Consider the following scenario:

Node 1 is shut down and updated with a new version of class A. This new version of class A holds a reference to class B, which is a new class. Upon restart, node 1 begins to insert new instances of class A that contain class B. In the meantime, node 2 has not been upgraded yet and it makes a request for an instance of class A that has been inserted by node 1. Since it doesn’t know anything about class B, it throws an exception while deserializing.

The good news is that Coherence makes this problem easy to solve. Using Portable Object Format (POF), you can make your cached objects forward and backward compatible in order to deal with the scenario above.

Before Coherence 3.4, POF was used exclusively for Coherence*Extend, in particular for .NET clients. Starting with Coherence 3.4, POF is natively supported inside of the cluster and is the preferred serialization format for Coherence.

In order to accomplish this, you must implement the EvolvablePortableObject interface.

• This means that you must write out the serialization routines (readExternal and writeExternal). Although somewhat tedious, this will pay off as POF serialized objects are smaller than their Serializable counterparts, and take less memory and CPU to serialize.
• EvolvablePortableObject extends the Evolvable interface, so these methods must also be implemented. The easiest way to do this is to extend AbstractEvolvable. The JavaDoc for Evolvable goes into detail about how it works, but the basics are that by implementing this interface, Coherence will automatically resolve differences between class and data versions.

Here is an example implementation:

import com.tangosol.io.AbstractEvolvable;
 
import com.tangosol.io.pof.EvolvablePortableObject;
import com.tangosol.io.pof.PofReader;
import com.tangosol.io.pof.PofWriter;
 
import java.io.IOException;
 
 
public class Person
        extends AbstractEvolvable
        implements EvolvablePortableObject
    {
    // ----- constructors ---------------------------------------------------
 
    public Person()
        {
        }
 
 
    // ----- accessors ------------------------------------------------------
 
 
    public long getId()
        {
        return m_id;
        }
 
    ...
 
 
    // ----- Evolvable interface ---------------------------------------
 
    public int getImplVersion()
        {
        return VERSION;
        }
 
 
    // ----- PortableObject interface ---------------------------------------
 
    public void readExternal(PofReader in)
            throws IOException
        {
        m_id         = in.readLong(0);
        m_sFirstName = in.readString(1);
        m_sLastName  = in.readString(2);
        m_sEmail     = in.readString(3);
        }
 
    public void writeExternal(PofWriter out)
            throws IOException
        {
        out.writeLong(0, m_id);
        out.writeString(1, m_sFirstName);
        out.writeString(2, m_sLastName);
        out.writeString(3, m_sEmail);
        }
 
 
    // ----- fields ---------------------------------------------------------
 
    private long   m_id;
    private String m_sFirstName;
    private String m_sLastName;
    private String m_sEmail;
 
    public static final int VERSION = 1;
    }

A POF configuration for such an object may look like this:

<!DOCTYPE pof-config SYSTEM "pof-config.dtd">
 
<pof-config>
  <user-type-list>
    <include>coherence-pof-config.xml</include>
 
    <user-type>
      <type-id>10000</type-id>
      <class-name>com.tangosol.examples.evolvable.Person</class-name>
    </user-type>
 
  </user-type-list>
</pof-config>

To configure Coherence to use POF, the easiest method is to configure the following system properties in your startup script:

-Dtangosol.pof.enabled=true 
-Dtangosol.pof.config=pof-config.xml

Now let’s say you have a running system and you want to add a new Address field to the Person class. The following modifications can be made to Person:

    // ----- PortableObject interface ---------------------------------------
 
    public void readExternal(PofReader in)
            throws IOException
        {
        m_id         = in.readLong(0);
        m_sFirstName = in.readString(1);
        m_sLastName  = in.readString(2);
        m_sEmail     = in.readString(3);
        m_address    = (Address) in.readObject(4); // <- new Address field
        }
 
    public void writeExternal(PofWriter out)
            throws IOException
        {
        out.writeLong(0, m_id);
        out.writeString(1, m_sFirstName);
        out.writeString(2, m_sLastName);
        out.writeString(3, m_sEmail);
        out.writeObject(4, m_address);  // <- new Address field
        }
 
 
    // ----- fields ---------------------------------------------------------
 
    private long   m_id;
    private String m_sFirstName;
    private String m_sLastName;
    private String m_sEmail;
    private Address m_address;  // <- new Address field
 
    public static final int VERSION = 2; // <- new version
    }

And this addition to the pof-config.xml file:

    <user-type>
      <type-id>10001</type-id>
      <class-name>com.tangosol.examples.evolvable.Address</class-name>
    </user-type>

There are the changes that I made:

1. Added the Address field as a member of the Person class and to the serialization routines (as index 4)
2. Incremented the version
3. Added the Address class to the POF configuration

That’s it! Now clients with an old version of the Person class can load new versions of Person data, modify it, and place it back into the cache while preserving all of the new data (for a hint on how this is done, take a look at the futureData property defined by Evolvable. The obvious caveat here is that new fields must always be added to a class, and the order must be preserved.