NoSQL: The Magic Bullet

Many have this belief that noSQL will be their magic bullet. Their salvation to a failing system. Their redemption to slow response times. Yet, nine times out of ten the issue is not with their current solution. It is due to poor design, or in most cases, inaccurate data models.

With noSQL, the way data is accessed needs addressed upfront. You can’t skip this step. The data access pattern is the driving force in the noSQL world. In the traditional world, you can cheat performance. For example, with a traditional database, a fact table is created with the required dimensions. Then, indexes are added to each dimension. Viola, decent performance with littler effort. Not much upfront design needed. The SLA achieved, and everyone is happy. As the access patterns are discovered, indexes are re-factored for optimal performance. Problem solved, customers delighted. In the noSQL world, it is not that easy.

Let’s take HBase for example. The rowkey design is crucial for performance and distribution of data across the regions.  If this step is rushed, or done in a haphazard way, data will saturate a region (hot spot).  This will cause the cluster to come tumbling down.  Yes, I did this, I speak from experience folks, and it is not pretty.  Then we have to consider the model itself. Do we want one column family or more? If more than two are needed, is this a sign of a bad design unfolding? How will this design impact performance? Are timestamps required for data retrieval? Will other data elements be needed other than the rowkey? And the list goes on.

NoSQL does not come without risk. And the design of how data is stored and accessed is critical. If we are going to move into the noSQL world, then we need to commit one hundred percent.

Remember, most projects do not fail because of the chosen technology. They fail because the solution was rushed into production without proper design and testing.  The use cases and the data access patterns that drove them were not understood. If we rush a noSQL solution into production, we cannot cheat a fix later as in the RDBMS world.

With that said, the point is this. It is critical in the noSQL world to understand the requirements and data access patterns. How the UI will interact with that data.  If this concept is not understood, then we will run the risk of complete and total failure.  Dev-ops will break down our doors in the middle of the night because senior leadership is screaming at the top of their lungs due to lost revenue.   Customers will no longer use the product because they cannot access the data that drives their revenue. And the million dollar visualization layers the UI team painfully built over the last twenty-four hours will be rendered useless. All because the solution was rushed into production due to unrealistic demands, just like the RDBMS system that is currently failing. But this time, failure comes at a much greater cost and impact to the business.

Just another random rant on a rainy Seattle day.