Paginate and Play Safe

Let's write a  simple application 'LicenseDB' that exposes an API to list the vehicle numbers in the United States.

public void printNumbers() {
    LicenseDB db = new LicenseDB();
    //print db.getVehicleNumbers();
}

Congratulations! We've just blown up someone's browser or mobile or a tablet or the caller is timed out before getting a response from us. Whenever we have to expose a list API, we should be careful about the amount of data we would end up dumping. We can get around this issue by paginating our result set.

How do we paginate ?

Lets rewrite our LicenseDB API so that when a client calls getVehicleNumbers(), it returns a continuation token along with a small list of numbers.

What is a continuation token ?

From a client's perspective, a continuation token is a unique identifier returned by getVehicleNumbers() which when passed to getVehicleNumbers() on a subsequent call, will return the next set of vehicle numbers.

In getVehicleNumbers() implementation, a continuation token is a bookmark that identifies the next list of vehicle numbers that gets returned to the caller. Let's give it a shot.

class ResultSet {
    private final List <String> results;
    private final String continuationToken;

    public ResultSet(List<String> results, String continuationToken) {
        this.results = result;
        this.continuationToken = continuationToken;
    }

    public List <String> getResults() {
        return results;
    }

    public String getContinuationToken() {
        return continuationToken;
    }
}

class LicenseDB {
    public ResultSet getVehicleNumbers(String continuationToken) {
        // parse the continuationToken to get the bookmark info.
        List <String> result = db.retrieve(bookmark);
        // construct a new continuation token. If there are no more results, return a null.
        String continuationToken = constructNewContinuationToken();
        return new ResultSet(result, continuationToken);
    }
}

A naive client side code would do something like,

public void printNumbers() {
    String continuationToken = null;
    do {
        ResultSet result =  db.getVehicleNumbers(continuationToken);
        continuationToken = result.getContinuationToken();
        // print result.getResults();
        // break if user doesn't want more results.
    } while (continuationToken != null);
}

We can also add the size of the result set along with continuation token on calls to getVehicleNumbers(), so that clients can control the size of the result they get every time.

What have we done ?

If we have to expose a list API without pagination as a web service, preparing the whole list in one HTTP request on our end is a bad customer experience. Often they will time out or their browser will run out of memory once we dump the response.

By returning a paginated result, we have given the control over to the clients to decide what they want from our API.

~Surya.

Retry and Recover

Coding against eventually consistent systems

Eventual consistency is a key concept in most distributed systems. Lets consider a distributed storage system, we can say it is eventually consistent if the data that we write/update will eventually be available to all the users at some point if not immediately. When we write an application that depends on such a system, our application shouldn't suck at intermediate failures and should be able to quickly recover from transient issues. Amazon S3 is a good candidate for a storage service that is eventually consistent. So lets write a very simple piece of code which,

1. Creates a bucket in Amazon S3
2. Adds an item to the bucket

If we write something like the following, we are in deep trouble.

AmazonS3 s3 = new AmazonS3Client(awsCredentials);
String bucket = "awesome_bucket";
s3.createBucket(bucket);
s3.putObject(bucket, "my_key", file);

The call to createBucket() doesn't guarentee the availability of bucket immediately. As a result,

s3.putObject() may fail with a BucketNotFoundException and that's bad.

A better code would be,

AmazonS3 s3 = new AmazonS3Client(awsCredentials);
String bucket = "awesome_bucket";
boolean bucketExists = false;
int tries = 0;
while (tries < MAX_RETRIES) {
    tries++;
    s3.createBucket(bucket);
    if ((bucketExists = s3.doesBucketExist(bucket))) {
        break;
    }
    Thread.sleep(1000);
}
if (bucketExists) {
    s3.putObject(bucket, "my_key", file);
}

Let's go one step further and come up with a nice abstraction. A functor could've made things much simpler. Anyways,

public void eventually(Runnable retryableTask, 
        Callable<Boolean> validatorTask,
        long bailOutTimeInMs) throws Exception {

    int noOpInMs = 1000;
    Exception retryableTaskException;
    while (bailOutTimeInMs > 0) {
        try {
            retryableTask.run();
        } catch (RuntimeException e) {
            retryableTaskException = e;
        }
        if (taskValidator.call()) {
            return;
        }
        Thread.sleep(noOpInMs);
        bailOutTimeInMs -= noOpInMs;
        noOpInMs *= 1.2;
    }
    throw retryableTaskException;
}

Basically the above wrapper takes in a retryable task and a validator task and sleeps between two tries with a geometric back off timer. If validator task returns success, then we return.

Lets try our example using the wrapper,

eventually(
    new Runnable() {
        @Override
        public void run() {
            s3.createBucket(bucket);
        }
    },
    new Callable() {
        @Override
        public Boolean call() {
            return s3.doesBucketExist(bucket);
        }
    });

This way, we can submit any task that needs to be retried conditionally. If we want to make the validator return an object, we can use generics and pass a Callable <T> validatorTask.

What have we done ?

• We made sure that our service is resilient to intermittent inconsistencies caused by dependent services that are eventually consistent.
• Written a template to which we can submit retry-validate logic so that we don't end up writing while loops and conditional breaks all over our business logic and trash the code.

~Surya