Over the past few short years, there have been dramatic developments in how we design and rollout services and systems to users. Apart from simple ‘brochure websites’ and sites based on content publishing platforms, the default target of choice seems to have become the ubiquitous ‘cloud’. When I went to my first introduction open day to Azure a number of years ago, I came away thinking ‘interesting concepts, but no-where near ready prime-time’. That was in the early days of Azure of course, when everyone was still learning. In those days (only a scant 5 or so years ago), building at extreme scale, for peak efficiency, and then sharing that scale out to others was a big big challenge. At the time, there were many new paths being explored, and a whole host of new design patterns known as ‘cloud patterns’ were tested and developed. There are options open to us now that simply were not there a few short years ago, and we need to bear these in mind when designing solutions for the new cloud based web.
While there are many worthwhile frontend changes that have come about over the past few years which many of us are using, typically one of the most expensive parts of the production chain is how and where we store the data related to our application. This is what we are going to address here. If you have always traditionally used the same old stack, you need to think about checking out what’s new in cloud database offerings and adjust your thinking for design to be in line with this. If you dont, you can be sure your competitor will, so it’s worthwhile.
(Oh, and by the way – when I say competitor, I don’t simply mean a company competing for the business your company gets, I also refer to other software developers and engineers in your company/organisation or area, who are competing for the next promotion, career progression or job that you may go for!).
Our most traditional method of handling data has been to analyse, de-normalise, and stick into a relational database like SQL Server. Apart from some small scale applications, it seemed that an RDBMS ended up as the default staging ground for all of our data. From an in-house (or ‘on premises’) point of view, this was fine, but costly, requiring not only operating system licenses but also database access and management licenses. When we move to the web, our default thinking is to de-normalise and fit everything into the SQL server. This may or may not be the optimal solution, and even when it is, we need to ask ourselves what kind of SQL service is best suited to our needs. (note I said service, not server!) .
When we look at putting a system into production on Azure, we dont just have SQL data storage on offer, we also have document databases, blob storage, table storage, and file storage, to say nothing of the standard RDBMS offerings that are available to us that we can spin up in Virtual Machines (VMs). While VMs are really great for allowing us maximum flexibility and doing exactly what we want, they also come at the price of having to manage them. I was recently designing an architecture for a new website – whereas a short while ago I would definitely have decided to use SQL, I really questioned the use case of it this time, and ended up deciding on a filtered blob storage solution using Azure DocumentDB instead. For a large volume of data, it works out at about 0.10 cent per month, versus a minimum of $50 if I were to run an SQL instance.
So, your homework for this week is not to learn something new, but to make yourself aware of some of the new things that are out there as alternatives to simply falling back on SQL as the default storage.
Here are some links to get you going,