Gone Full Unikernel
Thu, Jun 16, 2016
It’s been almost a year since the last blogpost – what has the DeferPanic team been up to?
Well, if you ever had an account with us, chances are you received a notice about us sunsetting the Go APM service. Spoiler Alert: (We aren’t removing it entirely.
The last year has been like a whirlwhind – I honestly don’t know where the time went especially the past few months.
WTH is a Unikernel??
For the purposes of this article we are defining a unikernel as a small, secure, bootable virtual machine image that includes only the necessary bits for an application to work – and in our case that mostly means web services. Unlike the JeOS (Just Enough Operating System) movement from a few years ago by stripping out un-needed parts unikernels piece in only what they need instead. That – and there’s a “few” architectural changes.
How small are these? Try 5, 10, 20 megabyte small. Depending on your needs you can go down into the kilobyte range – and that’s not just the app – that’s everything.
How secure? Go ahead and try to fork a shell – we’ll wait. What’s more, is the next time the app is deployed you’ll be looking at different memory. The fourteen year old hacker experience will never be the same again.
Oh the teardrops in my eyes, Nostalgia.
Why are you all running unikernels?
The simplest answer here is that we wanted too and we got a bit carried away.
The longer answer is that I was reading every whitepaper on the subject last year and spent some time looking around to see if someone had already done the work to make Go run as a unikernel – turns out no one had. So we bribed some people with money and blew some of our own time and https://github.com/deferpanic/gorump was birthed.
I’ve always had a keen interest in operating systems and have long been a microkernel proponent so this was a very easy thing for me to say yes too.
As soon as we started productionizing the service though, we ran into problem after problem and started having to write a bunch of random tooling. We aren’t 100% dogfooding yet but it’s our goal and baby steps .
Who would’ve thunk going from a multiple process, multiple address space, multiple user system to a single process, single address space, no user system would involve so much work in changing your workflow, testing, tooling, infrastructure, etc.?
What was interesting was right around the same time – a funny thing started happening – projects started springing up everywhere in the unikernel ecosystem.
There was even a mini unikernel conference earlier this year and some last year as well ! Thought that just geeks wielding hypervisors were unikernel afficionados? No. Companies like NEC and Ericsson had already dived into the deep end. Later companies like IBM and EMC got into the action too. I wouldn’t classify any of these companies as ‘small’ or a ‘startup’.
Can I run a unikernel?
That might be why you are reading this article huh?
What’s the status?
So with any new ecosystem there are going to be many challenges. (Understatement of the year.)
Unikernels just happen to present a full rewrite of everything you thought you knew about a software stack – in other words plenty of challenges. However, I’m assuming if you are reading this article there’s a good chance your title has the word ‘engineer’ in it and engineers are good at solving problems.
Testing, dependencies, production support, CI/CD, etc. In fact if you look at any of those containerization ecosystem charts such as the one from CBInsights the unikernel ecosystem will utterly dwarf it because there is just so much more to be built.
Lots of exciting interesting things coming about. We are in the process of productionizing https://deferpanic.net . It’s more of a playground right now.
Our intent is to give the ecosystem as much of a nurturing environment as we can to help foster the growth.
Can I use Google Cloud or AWS?
You could – although you won’t write much more than a toy app – not until things are changed. Today’s ‘cloud services’ (other people’s computers) are inherently unfit for unikernel production (to borrow a phrase from one of our admirers @ Joyent).
DeferPanic.com is/was mostly on Google Cloud (hell, this blog is on Google Cloud) and I honestly can’t say enough good about them, but neither they nor AWS could provide what we needed – that is hypervisor level orchestration.
That was a pretty hard requirement right off the bat for us.
You can talk all day about how neither the big public cloud providers are a good fit for this ecosystem. Things like instance size or hourly billing on AWS are the easy things to spot.
The harder questions that you won’t even know are problems until you walk down the road are things like – “How do I run a database migration w/out popping a shell?” or “How do I even connect to the instance to do configuration management?”
These were questions that we couldn’t simply point at existing tooling to do for us – we had to come with new solutions.
You see a lot of people who talk about unikernels only see it through the lens of IoT – and that’s a great worldview to have.
I on the other hand am one of the heretics advocating it’s use for everyday webservices. When you say ‘microservices’ or ‘immutable infrastructure’ – that’s unikernels my friend.
So we started off with a few dedicated servers rented out from various providers. Then we proceeded to burn through quite a few vendors in the span of a couple of months.
I think one of the most underrated things that “public cloud” has done to the developer conciousness at large is the expectation that you might have downtime once or twice a year.
We had power and/or network issues with so many providers we threw up our hands in disgust and started racking our own servers. One only needs to examine the status page history to see the difference that made.
Does this decision make things much harder? Of course it does – it removes another layer of abstraction that we have to now deal with it but it is at least fully our responsibility now with no one to blame but ourselves and has the added benefit of autonomy to allow us to grow the way we want to.
Of course, as you can tell – we’re already managing the underlying infrastructure platform as well and that’s it’s own little box of sharp knives.
We are at a very special time in the history of computing. Over 25 years ago both the first web browsers and Linux arrived on the scene. What’s interesting is that this approach is arguably the way things were going to be in the future and we missed the chance.
We now as a community, have a chance again.
Two big arguments for unikernels that are commonly cited are dramatically improved security and dramatically improved consolidation. While this is technically true if the question is ‘why now’ I think the answer is more culturally based.
Let’s look at the past 15 years of computing. VMWare productionized and brought massive virtualization to market ~15 years ago. Today if you talk to a CIO there’s a damn good chance they are somewhat virtualized.
Five years after that a small company from Seattle unleashed something called ‘infrastructure-as-a-service’ to the world and it forever changed how developers use/interact with servers. Nowadays very few developers interact with actual real hardware – it’s been completely abstracted away.
A few years after that we had Heroku come on the scene with ‘platform as a service’. I honestly did not understand when it came out why it was so popular. Turns out that there are more developers out there that don’t want to touch servers at all versus those who do. This notion was intensified in the past few years with the rebirth/popularization of containers. After all, Bill Joy (who made vi) also made chroot back in the 70s and of course PHK introduced jails in the late 90s but it just goes to show you what was going on.
This progression is what I’ve been calling the “Developer Divorce From the Server” and it’s a cultural trend that’s been seriously affecting the entire development lifecycle for the past 15 years at least.
We’re super excited about the direction we are headed. Though we have a ton of work ahead our hope is to enable the ecosystem and build what cloud infrastructure really should be.