The Big Cap Theory Part 4

Part 3 of this series is available to read here.

After many weeks, we now we finally come to the explanation of the BASE principal. With the growing adoption of Micro Services and Reactive design, the BASE principal resolves many of the problems we discussed earlier. Tt does require a shift in expectation, but let’s start with the definition.

The BASE Principal defines an attitude to data processing: much more relaxed and patient. As opposed to waiting for each transaction to persisted and propagated before completing and leaving, which delays processing overall as each transaction must wait for resources. BASE specifies a different set of expectations and requirements.

* Basic Availability

* Soft State

* Eventual Consistency

Basic Availability expects every component of a distributed architecture to be able to respond, even if the response is simply a failure. This might not sound like a major change, but it drastically alters the options available to a client. In the past, clients would send their request and be blocked while waiting for a response. Obvious questions arise. What if the receiving system fails in some way? We resolved this with timeouts on the client side. This means the sender gives up if the recipient doesn’t respond in time. This was silly. What about long running transactions? This also didn’t address if the delay was in the response and not in execution. The transaction might have completed yet we wouldn’t know one way or the other. Instead, with Basic Availability, the recipient is expected to respond if it won’t be able to process the request. This enables the client to make a choice on how to proceed. This introduces the complication of designing those alternatives. As an example, take Amazon’s recommendation engine. Most of the time it shows you customized recommendations, but when rendering the view for our browsers, if the recommendation engine can’t process out requests, it says so. This way, the client, our browser, for example, can decide if it wants to wait, or pursue an alternative. Have you ever seen Amazon recommend a standard list, like the New York Times Best Sellers? This would be an example of the client deciding not to request a recommendation because it got a negative response and instead would retrieve usable data from a cache. Basic Availability gives the client the information to make more intelligent decisions on how to gracefully degrade if there’s a fault. It’s up to us to design and implement that alternative.

Soft State means that the overall architecture will be nearly continually changing state. This sounds scarier than it actually is. It is easier to implement in a distributed micro service architecture than it might seem. This simply means that a client should expect the architecture to queue anything it can’t process immediately, and continue to process and propagate changes. This is a side effect of Basic Availability, and it typically results in the inclusion of more messaging and caching technologies. Let’s work through an order processing system as an example. Under the classic ACID model that we’re familiar with, we have the typical experience. You create your order, likely by browsing through the site and adding products to your shopping cart. Then, you submit your order to start the transaction, and you wait, and possibly wait longer. Maybe click the “Submit” button a couple more times. If this were the 90’s, we’d end up submitting additional orders. Eventually you’ll get your order confirmation, but why did you need to wait? With BASE, you would still build your order and submit it.  Rather than wait for the entire order to be processed, what if your submission went to a queue for order processing? You would get a response stating that your order has been received, and you will get an order confirmation and tracking number when it’s complete. This way, the architecture can absorb spikes in activity by simply caching transactions for future processing. This does mean we have to change our expectations on how applications behave. Instead, we’d treat all client interactions like a drive-thru window where they submit their transaction and wait around for a result. We move to treating transactions like mailing a letter or asking a coworker for something. You send off your request, and rather than waiting, you move on to other tasks and just expect that a response will come to you eventually.

Eventual Consistency means that the distributed system, provided it’s allowed to process its backlog, will eventually propagate all relevant changes to the rest of the system. Message queues will be processed, caches will be updated, and data persisted. Really, this is the simplest aspect to address. Because, really, what application would be designed to stop processing when there was more work to do? The important technology to make this possible is a queuing or caching solution where you can compile a backlog of work for the system to process, and workflows must change. Typically, the most efficient process is to submit all work to the caching or queueing solution and simply define a workflow that continuously processes that queue.

This is just a simple explanation of the BASE principal. The leading article and accompanying white paper on the subject can be found at Dataversity. I highly recommending reading their work on the topic as they go into much more detail. In future articles, we’ll examine how to design a distributed architecture based on the BASE Principal.

About the Author