Unikernels2021-11-19T12:14:41+00:00Original AuthorsThe search for the killer app of unikernels2017-08-11T01:00:00+00:00/blog/2017/the-search-for-the-killer-app-of-unikernels
<p>When a radically different technology comes along it usually takes time before we figure out how to apply it. When we had steam engines running factories there was one engine in each factory with a giant driveshaft running through the whole factory. When the electric engine came along people started replacing the giant steam engine with a giant electric motor. It took time before people understood that they could deploy several small motors in different parts of the factory and connect electric cables rather than having a common driveshaft. It takes time to understand the technology and its applicability.</p>
<p><img src="http://unikernel.org/images/steam_engine.jpg" alt="Steam engine" /></p>
<p>The situation with unikernels is similar. We have this new thing and to some extent we’re using it to replace some general purpose operating system workloads. But we’re still very much limited by how we think about operating systems and computers.</p>
<p>Unikernels are radically different. Naturally the question of the killer app has come up on a number of occasions. As unikernels are quite different from the dominant operating systems of today it isn’t as easy to spot what it will be. Here I’ll try to answer why it’s hard to spot the killer app.</p>
<h3 id="defining-characteristics-of-unikernels">Defining characteristics of unikernels</h3>
<p>Let’s start with what our characteristics are. Unikernels are fast, secure and small. This makes them a good fit for a number of applications. One approach was to consider serverless architectures. Booting <a href="https://mirage.io/">MirageOS</a> in a few milliseconds was the driving characteristic behind such approaches. Galois, through their <a href="https://github.com/GaloisInc/HaLVM">HaLVM</a> unikernel, are mainly relying on Unikernels being small and frugal to pack a lot of virtual machines into a small little box in their <a href="https://formal.tech/cyberchaff/">CyberChaff</a> product. <a href="http://www.includeos.org/">IncludeOS</a> is focused on network appliances like firewalls, load balancers and web based APIs, relying on security and performance to get adoption.</p>
<p>But there is an overlooked characteristic of unikernels. They are flexible. Unlike Linux and Windows you can easily boot a unikernel that relies on completely different infrastructure and protocols. Porting something as obscure as <a href="https://en.wikipedia.org/wiki/IBM_Systems_Network_Architecture">SNA</a> to a unikernel is a lot simpler than porting it to Linux or Windows. Likewise, we could well envision having HTTP support in a unikernel without having an IP stack. This sounds crazy and you’re likely asking yourself why you would ever do such a thing. I’ll get back to why and how, but bear with me for now.</p>
<p>Let us have a look at something the software industry has been struggling with for a while: running native code in browsers. We’ve always known it is a horrible idea but we’ve still spent hundred of thousands of hours making it a reality. If only we were able to build a properly secure sandbox it’d be great. Turns out making such a sandbox is really hard and this is why Java Applets, ActiveX controllers and NaCL have been abandoned (to be truthful, I’m not certain about why Google decided to discontinue NaCL; they’re a bit opaque on why).</p>
<h3 id="how-can-unikernels-allow-one-to-run-native-code-in-the-browser">How can unikernels allow one to run native code in the browser?</h3>
<p>It’s reasonably simple. Modern CPUs provide virtualization capabilities with good security built in and we can leverage this hardware to provide isolation between server-supplied code and the client it’s running on.</p>
<p>Imagine the following. You have a webpage that needs to run native code. It could be some brand new form of DRM or some real intensive cryptography that needs the full power your CPU can provide. The webpage downloads a unikernel image and prepares the image. The browser sets up the virtual machine, creates page tables and initializes the unikernel. The CPU doesn’t need to boot this virtual machine in Ring 0; it can run in Ring 3 of the CPU as it will never need to talk to hardware, neither virtualized or physical.</p>
<p>This VM would not have an IP stack but a small, secure, and well defined interface. It can basically do two or three things to communicate with the host:</p>
<ul>
<li>Talk HTTP to its origin server. Browsers have HTTP clients built in.</li>
<li>Manipulate the DOM the same way Javascript can do.</li>
<li>(Optional - only if we would like this to have persistent state) Manipulate some local storage - contained in a file.</li>
</ul>
<p>All of these mechanisms can be transported out of the VM through hypercall (like <a href="https://www.tptp.cc/mirrors/siyobik.info/instruction/VMCALL.html">VMCALL</a>). The VM will run at native speed and the CPU will trap whenever it needs to push data out of the VM.</p>
<p>The browser needs to provide these three interfaces and the unikernel needs to implement them as well. This is where HTTP without IP comes in. What would paravirtualized HTTP without IP look like? The application builds a HTTP request, puts it in memory and signals the hypervisor that it should deliver a message to the virtual machine monitor (VMM - in this case it will be the browser). Basically a classical struct should do it.</p>
<p>The browser takes a careful look at the pointer and the data it is pointing at. This code will need to be properly vetted as a buffer overflow here could potentially allow the unikernel to execute code outside its hardware prison.</p>
<p>The response to the request is fetched by the browser and put into shared memory as well. The browser signals the unikernel through an interrupt. This is more or less how paravirtualized drivers in the current virtual machines work. Since they aren’t really talking to hardware the CPU can run unprivileged (Ring 3).</p>
<p>Manipulating the DOM and getting DOM events into the unikernel can be made in a similar fashion. The unikernel builds a notification and calls the hypervisor which in turns delivers the message to the VMM. I’m not really a front-end guy so I’m a bit uncertain on what the exact API would look like.
Storage, if one should decide to implement it, can be either virtio or something even simpler. Presenting a block device to a virtual machine in a secure manner is a solved problem.</p>
<h3 id="pushing-it-further---securing-the-vm-from-the-prying-eyes-of-the-browser">Pushing it further - securing the VM from the prying eyes of the browser</h3>
<p>One could take this a step further and leverage the support for encrypted virtual machines. Intel has been supporting this for a number of years through their <a href="https://en.wikipedia.org/wiki/Software_Guard_Extensions">Software Guard Extensions</a> and AMD just came out with something similar in their EPYC CPUs. This would allow one to ship signed and encrypted code that could potentially be completely opaque to the user. This would naturally be a bit controversial if used for something like DRM, but there could be security applications that could benefit from running securely on a potentially compromised platform.</p>
<p>We have no plans of implementing this. While it certainly is pretty cool, there is little opportunity for basing any commercial success on such an implementation. The point of this exercise is to show you what the possibilities are. Replacing Linux with a unikernel is one thing, but the really exciting stuff will happen when we realize that the way we view operating systems is holding us back. The whole idea of having to emulate a whole computer with BIOS, PCI bus, and a floppy drive in order to run a virtual machine is pretty silly. Once we can free ourselves from that limitation we can start finding real applications for unikernels. In the meantime they can displace a few Linux VMs.</p>
<p><em>Discuss this post on <a href="https://devel.unikernel.org/t/the-search-for-the-killer-app-of-unikernels/270">devel.unikernel.org</a></em></p>
Unikernels are secure. Here is why.2017-07-10T01:00:00+00:00/blog/2017/unikernels-are-secure
<p><em><a href="https://twitter.com/perbu">Per Buer</a> is the CEO of <a href="http://www.includeos.org/">IncludeOS</a>. IncludeOS is a clean-slate unikernel written in C++ with performance and security in mind. Per Buer is the founder and previous CEO/CTO of Varnish Software.</em></p>
<p>We’ve created a video that explains this in 7 minutes, so you’ll have the option of watching it instead of reading it.</p>
<div class="flex-video">
<iframe width="560" height="315" src="https://www.youtube.com/embed/aoomQn7gLm4?ecver=1" frameborder="0" allowfullscreen=""></iframe>
</div>
<p>There have been put forth various arguments for why unikernels are the better choice security wise and also some contradictory opinions on why they are a disaster. I believe that from a security perspective unikernels can offer a level of security that is unprecedented in mainstream computing.</p>
<h3 id="a-smaller-codebase">A smaller codebase</h3>
<p>Classic operating systems are nothing if not generic. They support everything and the kitchen sink. Since they ship in their compiled form and since users cannot be expected to compile functionality as it is needed, everything needs to come prebuilt and activated. Case in point; your Windows laptop might come with various services activated (bluetooth, file sharing, name resolution, and similar services). You might not use them but they are there. Go to some random security conference and these services will likely be the attack vector that is used to break into your laptop — even though you’ve never used them.</p>
<p>Unikernels use sophisticated build systems that analyze the code you’re using and only link in the code that is actually used. The unused code doesn’t make it into the image created and doesn’t pose a security risk. Typically, unikernel images are in the 500KB-32MB range. Our own load balancer appliances weigh in at around 2MB.</p>
<h3 id="no-shell">No shell</h3>
<p>Unikernels have no shells. Most attacks I’ve seen invoke <code class="language-plaintext highlighter-rouge">/bin/sh</code> to modify the system they are attacking. Without a shell the attacker doesn’t have this opportunity. This forces the attacker to use machine code to subvert the system, decreasing the likelihood of succeeding with the attack.</p>
<h3 id="were-immutable---no-support-for-reconfiguring-the-vm">We’re immutable - no support for reconfiguring the VM</h3>
<p>Whenever we need to change a service that is running on a unikernel the service gets rebuilt and redeployed. So most likely the system doesn’t have the ability to reconfigure itself. So the attacker will likely have to inject this code as well, something that would be next to impossible. Even if the attack is successful, the VM will get back to a known state the next time there is a configuration change.</p>
<h3 id="no-system-calls">No System calls</h3>
<p>A binary application on Linux uses <a href="https://en.wikipedia.org/wiki/System_call">system calls</a> to talk to the operating system. The mechanism for doing system calls is well defined. System calls are numbered and on x86-64 you just put the number indicating the syscall in the <code class="language-plaintext highlighter-rouge">%rax</code> register and call the instruction syscall. So once you know what operating system the computer is running it’s very easy to invoke the operating system.</p>
<p>Have a look at this minimal hello world example written in C. This code, which is architecture specific (macOS), just writes “Hello World” on file descriptor zero using system call <code class="language-plaintext highlighter-rouge">#4</code>. In order to compile this on Linux replace the <code class="language-plaintext highlighter-rouge">4</code> with a <code class="language-plaintext highlighter-rouge">1</code> and it should work on Linux.</p>
<pre>
#include <sys/syscall.h>
int main() {
// Syscall #4 is write on OS X
syscall(4,0, "Hello world\n", 12);
}
</pre>
<p>System calls are, as you can tell, quite portable. Every installation of Linux supports the same system calls. This is practical when you’re using them to attack a computer system. If the target system is running Linux you know that <code class="language-plaintext highlighter-rouge">write()</code> is invoked with system call <code class="language-plaintext highlighter-rouge">#1</code>. So just put <code class="language-plaintext highlighter-rouge">1</code> into <code class="language-plaintext highlighter-rouge">%rax</code>, supply a pointer, length and file descriptor and invoke the syscall instruction. Adding a row to <code class="language-plaintext highlighter-rouge">/etc/passwd</code> is likely just around 10 instructions. <code class="language-plaintext highlighter-rouge">open()</code> the file, <code class="language-plaintext highlighter-rouge">lseek()</code> to the end of the file and <code class="language-plaintext highlighter-rouge">write()</code> a new row.</p>
<p>Unikernels on the other hand don’t have system calls. They only have function calls. For an attacker this means they’ll have to know the exact memory layout of your application in order to invoke the operating system. You’ll need to know the exact 64 bit address of the function you’re invoking. Good luck guessing that address. Our own unikernel, IncludeOS, randomizes addresses at each build, so even with access to source code you still don’t know the memory layout.</p>
<p>If the application doesn’t have any supplied code to do <code class="language-plaintext highlighter-rouge">write()</code>, the task goes from being hard to impossible.</p>
<h3 id="removing-hardware-emulation">Removing hardware emulation</h3>
<p>There are two aspects of unikernel security. One is the vector into the VM and the other is the vector from the VM into the hypervisor. Today the hypervisor offers a very complex set of interfaces to the VM. PCI bus, GPU, serial, floppy drives.</p>
<p>These interfaces pose a risk to the Virtual Machine Monitor (VMM). As shown by the <a href="http://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2015-3456">2015 VENOM attack</a>, they can be used to break out of a VM and into the VMM. In the VENOM attack the attacker could attack <a href="http://www.qemu.org">QEMU</a> through the floppy interface provided by it.</p>
<p>IBM research has spearheaded a new effort with their <a href="https://github.com/Solo5/solo5/tree/master/ukvm">ukvm</a> project. Ukvm is a replacement for QEMU specifically built for unikernels. <a href="https://github.com/Solo5/solo5">Solo5</a> provides a framework for unikernels to boot a VM that is backed by ukvm instead of QEMU. Both <a href="http://www.includeos.org/">IncludeOS</a> and <a href="https://mirage.io/">MirageOS</a> are already capable of booting using the Solo5 core and efforts are underway to port <a href="https://github.com/GaloisInc/HaLVM">HaLVM</a> to Solo5/ukvm. The <a href="http://muen.sk/">Muen Separation Kernel</a> also has a Solo5 port and it’ll be exciting to see if the more traditional operating systems will follow suit.</p>
<p>Simplifying the emulated virtual machines will significantly increase VMM security.</p>
<h3 id="cutting-off-access-to-ring-0--creating-proper-immutable-vms">Cutting off access to ring 0 — creating proper immutable VMs</h3>
<p>One of the criticisms I’ve heard raised against unikernels is that the whole application now runs in “kernel space” and has root privileges. Today, this is true, but the only reason is because unikernels need to manage virtual hardware and page tables to run. However, if the hypervisor can set up the VM before it is booted and provide paravirtualized interfaces to the hardware, we don’t need access to ring 0.</p>
<p>What we need is this:</p>
<ul>
<li>packet interface for the network</li>
<li>a block interface for some storage</li>
<li>a serial port to output console data</li>
</ul>
<p>These can likely be invoked through the <a href="https://www.tptp.cc/mirrors/siyobik.info/instruction/VMCALL.html">VMCALL</a> instruction. This will trap and invoke the Virtual Machine Monitor. Since these interfaces are rather simplistic we can spend a moderate amount of effort securing these and making sure that they are reasonably bug free.</p>
<p>If the VM runs in ring 3 it is incapable of modifying its own page tables. If the hypervisor loads the unikernel and sets the executable pages immutable before booting it, the VM cannot alter itself. This dramatically reduces the aperture of the attack. As long as there are no writeable and executable pages in the unikernel I’m strugling to see a way it can be subverted. If the application has a bug you might still be able to crash it and make it restart, but you have no way of subjugating the VM.</p>
<p>The hardware is still potentially vulnerable and as such the VM might be subject to attacks like the bitbanging attack we’ve seen on various ARM platforms. However, the track record of x86-64 is very good and I wouldn’t have any problem relying on the platform for workloads with high security requirements.</p>
<h2 id="conclusions">Conclusions</h2>
<p>The perimeter security defence of unikernels are today far superior to traditional operating systems. The absence of system calls and shells blinds the attacker and even if the application is buggy, subverting a unikernel VM is really hard. As our virtual machine monitors mature we can expect unikernel security to advance a lot further. The future for unikernels is secure.</p>
<p><em>Edit: discuss this post on <a href="https://devel.unikernel.org/t/unikernels-are-secure-here-is-why/258">devel.unikernel.org</a></em></p>
<p><sub><a href="https://commons.wikimedia.org/wiki/File:Broad_chain_closeup.jpg">Image</a> is (c) 2006 Toni Lozano and used under a Creative Commons 2.0 Generic License</sub></p>
A Unikernel eBook from O'Reilly2016-10-06T00:01:00+00:00/blog/2016/unikernel-ebook
<p><a href="http://www.oreilly.com/webops-perf/free/unikernels.csp"><img src="http://unikernel.org/images/oreilly-ebook.gif" alt="eBook Cover" /></a></p>
<p>I am pleased to announce that my FREE unikernel eBook is now available
from O’Reilly.</p>
<p>I have been giving talks about unikernels for the past 2 years at
conferences throughout North America. This eBook is my attempt to
present most of the information from these talks in a written form.
It is not a technical HowTo book, but rather an introduction to the
basic concept of unikernels and an explanation of their value.</p>
<p>I hope this eBook will be a useful tool for introducing people to the
whys and wherefores of unikernels.</p>
<p>You can download your copy here:
<a href="http://www.oreilly.com/webops-perf/free/unikernels.csp">http://www.oreilly.com/webops-perf/free/unikernels.csp</a></p>
NFV Platforms with MirageOS Unikernels2016-07-18T05:00:00+00:00/blog/2016/unikernel-nfv-platform
<p><em>Wassim Haddad is at Ericsson Silicon Valley where he currently works on
distributed cloud infrastructure. Heikki Mahkonen and Ravi Manghirmalani work
at Ericsson Research at Silicon Valley in the advanced Networking and
Transport labs. The Ericsson team has a diverse background in different NFV,
SDN and Cloud related R&D projects.</em></p>
<h3 id="the-push-towards-nfv">The push towards NFV</h3>
<p>The Network Function Virtualization (NFV) paradigm breaks away from
traditional “monolithic” approaches, which normally build network functions by
tightly coupling application code to the underlying hardware. Decoupling these
components offers a new approach to designing and deploying network services.
One that brings a high degree of flexibility in terms of separating their
lifecycle management and enabling much more efficient scaling.
Moreover, the move away from specialized hardware coupled with a “virtualize
everything” trend is fuelling operators and service providers’ expectations of
significant cost reductions. This is undoubtedly a strong motivation behind
NFV adoption.</p>
<p>Current NFV market trends point towards two key technologies: Cloud
Orchestration (e.g., OpenStack) to provision and manage workflows, and Software
Defined Networking (SDN) to enable dynamic connectivity between different
workflows as well as network slicing. In parallel, there is also a strong
desire to migrate from virtual machines towards microservice enablers,
particularly Docker containers, to boost performance and hardware utilization.
On the other side, it is evident that without intelligent and dynamic traffic
steering between different VMs and/or containers, large-scale NFV deployments
(e.g., for 4G/5G end-to-end slicing, virtual CPE, etc.) cannot take place. For
this purpose, cloud orchestration and SDN have to coordinate their actions,
which further complicates the overall architecture.</p>
<p>There are multiple advantages behind adopting containers as they offer higher
density, a single operating system, and faster startup/shutdown. However, the
ever-growing kernel complexities and vulnerabilities, together with the
requirement to ensure strong isolation between different applications, have
been frequently cited as barriers to widespread adoption.
Unikernels, which sit on a <a href="http://unikernel.org/blog/2015/unikernels-meet-docker">continuum</a> with containers,
offer a different approach.</p>
<p>We built a new standalone software platform to unify automation, orchestration
and the ‘stitching together’ of a designated set of NFVs. Our
platform does not rely on current cloud orchestration or SDN technologies.
Instead, the platform architecture goes beyond existing virtual machines and
containers, by introducing the concept of ‘nanoservices’ using MirageOS
unikernels.</p>
<p><a href="https://mirage.io">MirageOS</a> is a library operating system written entirely in a type-safe high
level programming language, <a href="http://ocaml.org">OCaml</a>.
MirageOS restructures all system components as modules which are implemented
as a set of independent libraries. Such decomposition enables programmers to
select and link together only the set of libraries that the application
requires, resulting in at least an order of magnitude reduction in code size
and a correspondingly much smaller attack surface. The compilation of the
modular stack of application code, system libraries, and configuration
produces the so called ‘unikernel’.</p>
<p>By applying unikernel technology in the NFV space, we constructed a set of
specialized, highly secure and scalable ‘nano-NFVs’ that can be streamed into
our next-gen cloud. Our nano-NFVs show higher performance compared to
existing NFVs, and have a much smaller memory footprint (i.e., between hundred
kilobytes to few megabytes). As one example, excluding configuration and
memory channel setup, our nano-NFV boots up within 10-20ms and is
automatically removed when the request is fulfilled thus, enabling
‘zero-footprint’ or ‘serverless’ clouds — whereby the service is available
only when needed. Furthermore, our platform embeds intelligent ‘traffic
steering’ capabilities achieved via shared memory circuits, which enable
operators to slice their infrastructure as they wish (e.g., per user and/or
per device and/or per flow).</p>
<h3 id="implementing-a-proof-of-concept-nfv-platform">Implementing a proof of concept NFV platform</h3>
<p><img src="http://unikernel.org/images/ericsson-nfv/fig1.png" alt="(a) Typical NFV setup and (b) NFV setup using Jitsu and vchan packet I/O" /></p>
<p>Currently, our NFV platform runs a DHCP server, Network Address
Translation (NAT), and firewall services. Our platform is implemented in OCaml
using the MirageOS development environment and uses a number of existing
MirageOS libraries. The left side of the figure above shows how our platform
can be implemented from a set of unikernel-based NFVs, with each having its own
network stack. In this case, each NFV needs to be orchestrated beforehand and
the virtual switch needs to have packet-forwarding rules, in order to support
service function chaining.</p>
<p>Our ‘Just-in-time summoning (<a href="http://unikernel.org/files/2015-nsdi-jitsu.pdf">jitsu</a>)’ of NFVs implementation is shown on
the right side of the figure. In this case, the only unikernel running a
network stack is the <em>Packet I/O</em> unikernel, which needs to be orchestrated
before incoming IP packets can be consumed. In fact, when the <em>Packet I/O</em>
unikernel receives an IP packet, it checks first if a matching shared memory
circuit exists for it. If a shared memory circuit has already been created,
the packet is bridged to it for handling. Otherwise, a memory circuit can be
created by utilizing the orchestrator unikernel. In such scenarios, IP packets
are forwarded to the orchestrator unikernel which in turn, looks up the
subscriber policy in the Irmin datastore then creates a dedicated NFV
unikernel for the specific IP flow defined by the 4-tuple: (IP source address,
source port number, IP destination address, and destination port number).
For example, if the received packet is a DHCP request, then a DHCP server
is created and connected to the <em>Packet I/O</em> unikernel via a shared memory
circuit.</p>
<p>If the packet belongs to a different IP flow, a service function chain is
created by stitching together NAT and firewall unikernels with shared memory
circuits. Once the set of dedicated NFVs is setup, packets belonging to the
corresponding IP flow are pushed back to the <em>Packet I/O</em> unikernel, which
forwards them to the just-in-time created memory circuits. In the reverse
direction, packets received from the shared memory are bridged to the
networking stack then sent out to the network interface.</p>
<p>Our NFV platform implements the shared memory by using the <a href="https://github.com/mirage/ocaml-vchan">vchan</a> library,
which enables connecting two Xen domains to each other with a bidirectional
memory channel over which bytes can be sent. We also implemented framing to
enforce IP packet boundaries through the shared memory. In order to connect
domains with vchan memory circuits, <a href="https://github.com/mirage/ocaml-xenstore">Xenstore</a> is used to configure endpoint
domain IDs. Each unikernel reads the domain ID on boot up time, which ensures
that the platform is immutable. If circuits need to be changed then associated
NFVs need to be restarted and configured.</p>
<p>The immutability aspect underlying our platform architecture allows assigning
dedicated pre-configured unikernels to serve a particular user. This means
that once instantiated, these unikernels will process packets according to
their original configurations, i.e., the one available at compile time. Any
attempt to re-configure a unikernel at runtime will result in re-compiling and
re-launching the unikernel.</p>
<p>Our NFV orchestrator is running on Xen Dom0 and uses the <a href="http://irmin.io">Irmin</a> and
<a href="https://github.com/xapi-project/ocaml-xen-lowlevel-libs">Xenctrl</a> libraries to implement just-in-time summoning of NFVs. Irmin is
used to store and retrieve the subscriber policies when new NFVs are required
to process incoming packets. As the orchestrator is running on Dom0, it can
use the Xenctrl library to create and destroy new unikernel domains on demand.</p>
<p>The orchestrator unikernel consists of OCaml code that handles packet I/O and
framing from the shared memory, binding to the Irmin datastore, creation of
new unikernels, and periodic garbage collection for idle unikernels. To
determine idle unikernels, the <em>Packet I/O</em> unikernel stores a timestamp for
packets forwarded to memory circuits. The orchestrator compares this timestamp
to a threshold and if needed, unikernels can be deleted. We implemented a CRUD
API for the Irmin datastore to retrieve and update subscriber policy records.
This API provides simple utility functions, e.g., <code class="language-plaintext highlighter-rouge">find_subscriber</code>, <code class="language-plaintext highlighter-rouge">get_mac</code>,
<code class="language-plaintext highlighter-rouge">get_service</code>. Inside the API, these functions use the Irmin API to connect to
the actual datastore.</p>
<p>The DHCP server uses the <a href="https://mirage.io/blog/introducing-charrua-dhcp">charrua-core</a> DHCP server library.
DHCP configuration is passed to the server through Xenstore. Packet I/O is
done through a single vchan memory circuit that is connected to the
<em>Packet I/O</em> unikernel. DHCP Requests received are passed to the charrua-core
library by using the <code class="language-plaintext highlighter-rouge">Dhcp_server.Input.input_pkt</code> function. This function
returns DHCP Reply packets, which are forwarded back over the vchan memory
circuit to <em>Packet I/O</em> which pushes them down the wire.</p>
<p>For NAT, we are using the <a href="https://github.com/yomimono/simple-nat">simple-nat</a> library that handles TCP and UDP
flows (for ICMP testing we implemented simple IP translation mechanism).
The NAT unikernel has two memory circuits one for uplink and one for downlink
direction. Packets are passed between these circuits and in between passed
through the simple-nat library by invoking the <code class="language-plaintext highlighter-rouge">Nat.translate</code> function.
For ICMP flows, we do the IP address rewriting in place in the vchan receive
function. Currently, in our proof of concept, the firewall is a simple
pass-through NFV but our plan is to use the
<a href="https://github.com/talex5/qubes-mirage-firewall">QubesOS unikernel firewall</a> as a library.</p>
<p>The unveiling of our lightweight NFV proof of concept was in January, 2016, at
the 14th annual <a href="https://www.socallinuxexpo.org/scale/14x/unikernels-and-more-cloud-innovators-forum">Southern California Linux Expo (SCALE 14x)</a> in
Pasadena, California (<a href="http://www.slideshare.net/xen_com_mgr/unikernels-meet-nfvs-architecture-performance-and-challenges-wassim-haddad-heikki-mahkonen-ravi-manghirmalani-ericsson">slides</a>, <a href="https://www.youtube.com/watch?v=it3g4XWskkc">video</a>).
A post with other details is available on the
<a href="https://www.ericsson.com/research-blog/sdn/unikernels-meet-nfv/">Ericsson Research blog</a>. Our results in the figure below show
creation, boot up, configuration and memory channel setup times that are
around tens of milliseconds for typical NFVs such as a DHCP server, NAT, and
firewall. In addition, our approach enables service chaining between NFVs
without having to inject complex rule sets in the virtual switches. Also, the
resource requirement for NFVs could be kept to a minimum since only a single
vCPU and a few tens of MB of memory were required for each service. Note that
a <a href="https://hannes.nqsb.io/Posts/BottomUp">recent study</a> showed that it is possible to further shrink the
NFV image size to less than 10MB. Upcoming work on
<a href="https://github.com/ocaml/ocaml/pull/608">dead code elimination in OCaml</a> will also provide immediate impact with
minimal effort, aside from upgrading the compiler — a key benefit of clean
slate unikernels.</p>
<p><img src="http://unikernel.org/images/ericsson-nfv/fig2.png" alt="Unikernel NFV PoC platform" /></p>
<h3 id="next-steps">Next Steps</h3>
<p>We believe unikernels are well suited for adoption by the new emerging class
of IoT and 5G applications, primarily because of their smaller, more secure
memory footprint. These features mean that network providers will be more open
to hosting third party applications at various points in their network and to
monetize low-latency guarantees, geo-local information and personalized
customizations. New distributed computing paradigms, software abstractions, and
methodology for service and network function compositions will emerge because
of the fact unikernels can be activated in a relatively short duration
resulting in greener datacenters.</p>
<p>We are investigating areas which can profit from the capabilities offered by
unikernels. We believe this could bring in a new era of telecom innovation and
accelerate the adoption of third party value added services.</p>
<p><em>Edit: discuss this post on <a href="https://devel.unikernel.org/t/nfv-platforms-with-mirageos-unikernels/175">devel.unikernel.org</a></em></p>
<p><em>Thanks to <a href="https://twitter.com/amirmc">Amir</a>, <a href="http://mort.io">Mort</a>, and <a href="http://anil.recoil.org">Anil</a>
for their comments on earlier drafts.</em></p>
CyberChaff: HaLVM unikernels protecting corporate networks2016-05-18T00:01:00+00:00/blog/2016/halvm-cyberchaff
<p>Unikernel technologies, specifically the libraries, are applicable in many
ways (e.g. the recent <a href="https://blog.docker.com/2016/03/docker-for-mac-windows-beta/">Docker for Mac and Windows products</a>). However,
unikernels themselves can enable new categories of products. One of the most
prominent products is a network security tool called CyberChaff, based on open
source HaLVM unikernels. Today Formaltech, a Galois subsidiary, <a href="https://galois.com/news/cyberchaff-reed-college/">revealed</a>
that Reed College is one of their happy CyberChaff users!</p>
<h3 id="defending-a-network-with-cyberchaff">Defending a Network With CyberChaff</h3>
<p>CyberChaff is designed to detect one of the early and critical steps in a
security breach: the point when an attacker pivots from their initial entry
point to the more juicy parts of the network. This step, the pivot, typically
involves scanning the network for hosts that may be better positioned, appear
to have more privileges, or are running critical services.</p>
<p>To impair this step of the attack, CyberChaff introduces hundreds (or
thousands) of false, lightweight nodes on the network. These hosts are
indistinguishable from real hosts when scanned by the attacker, and are each
implemented as their own HaLVM unikernel. See the diagram below where green
nodes are the real hosts and the orange nodes are HaLVM CyberChaff nodes. This
means that an attacker is faced with a huge surface area of potential decoys,
which mask the real hosts. If they scan one of the decoys, an alarm goes off.
If they try to interact with one of the decoys, via HTTP or some other
protocol, an alarm goes off. IT staff are notified of the intrusion, and can
react as they wish. Further, since each CyberChaff node is a
unikernel, and their only purpose is to look like a real system, there is
nothing to actually gain access to! Even if the attacker manages to find a
flaw in the CyberChaff software, all they will have gained access to is a
virtual machine with very, very limited functionality.</p>
<p><img src="http://unikernel.org/images/cyberchaff-reed/before-after-cyberchaff.jpg" alt="Pic of a network after CyberChaff" /></p>
<!-- ![Pic of a network before CyberChaff](/images/cyberchaff-reed/before-cyberchaff.png)
![Pic of a network after CyberChaff](/images/cyberchaff-reed/after-cyberchaff.png) -->
<h3 id="under-the-hood">Under the Hood</h3>
<p>So how are CyberChaff nodes actually implemented?</p>
<p>As mentioned, each CyberChaff node consists of a HaLVM unikernel implementing
our core CyberChaff engine. Based on its configuration, it can choose whatever
it wants to implement: servers, workstations, laptops, switches, mobile
phones, and even the navigation computer in a sailboat. It does so by including
configuration information at all layers of the network stack: custom
functionality that allows it to appear as different operating systems to low
level scans, custom protocol handlers to mimic the services available on the
network, and even higher-level protocol implementations to capture information
about the attacker. For example, CyberChaff can be configured to expose
an Apple-manufactured network card, running a Microsoft Operating System, with
a particular version of Nginx running on a given port and SSH capturing the
attacker’s login credentials. Of course, such a Frankenstein system might be
obvious to an attacker, so usually administrators try to be more subtle!</p>
<p>Once they’re up, CyberChaff nodes sit on a network, consuming few resources
and generating only a little traffic. Why generate traffic? Well, one of the
quieter ways for an attacker to scan a network is to silently listen for
certain forms of management traffic. For example, they might list to ARP
requests and responses to figure out who is on the network. Thus, CyberChaff
generates a small, managed amount of traffic to make sure it looks legitimate.
Future extensions will target the generation of more interesting traffic,
such as encrypted SSH or web sessions, or even unencrypted web crawling.</p>
<p>Overall, creating and deploying a product of this kind is possible due to the
unikernel approach. Since each node is small and single-purpose, it becomes
much easier to deploy large numbers of them across a network. This is in
contrast to creating ‘honey pots’ with traditional systems, which can be
expensive, are cumbersome to maintain, and unfeasible to deploy at scale.</p>
<p>It’s exciting we can now talk about unikernel products in the real world and
this is only the beginning of what they can offer in terms of efficiency,
capability, and portability.</p>
<h3 id="get-involved">Get involved!</h3>
<p>You can learn more about <a href="https://galois.com/news/cyberchaff-reed-college/">Reed’s experience with CyberChaff</a>, find out
more at <a href="https://formal.tech/products/cyberchaff/">Fomaltech</a>, checkout the <a href="https://github.com/GaloisInc/HaLVM">HaLVM codebase</a>, and
join the conversation at <a href="https://devel.unikernel.org/c/halvm">devel.unikernel.org</a>!</p>
Unikernel Systems joins Docker2016-01-21T14:00:00+00:00/blog/2016/uniks-joins-docker
<!-- ![Unikernel Systems webpage](http://unikernel.org/images/logo-uniks.png) -->
<p>Today, <a href="http://unikernel.com">Unikernel Systems</a> announced that it is joining Docker. Please
read the main notice posted at <a href="http://unikernel.com">unikernel.com</a> and the Docker
blog post at <a href="http://blog.docker.com/2016/01/unikernel/">http://blog.docker.com/2016/01/unikernel/</a>.</p>
<p>An important part of the work of Unikernel Systems is to support the growing
unikernel movement, which includes contributing to a number of open source
projects as well as supporting this community website, <a href="http://unikernel.org">unikernel.org</a>. The
Unikernel Systems team will continue this work.</p>
<p>Through further interaction with the Docker project and community, the open
source unikernel community will receive increased visibility and contributions
which will accelerate the adoption of unikernels.</p>
<p>Links:</p>
<ul>
<li><a href="http://unikernel.com#notice">Unikernel Systems announcement</a></li>
<li><a href="http://blog.docker.com/2016/01/unikernel/">Docker blog post</a></li>
<li><a href="https://www.docker.com/press-release-01212016docker-acquires-unikernel-systems-extend-breadth-docker-platfrom">Docker Press Release</a></li>
</ul>
<p><em>Discuss this on <a href="https://devel.unikernel.org/t/unikernel-systems-joins-docker/71">devel.unikernel.org</a></em></p>
IncludeOS is now free and open source!2015-12-15T14:30:00+00:00/blog/2015/includeos-is-now-open-source
<p><img src="http://unikernel.org/images/IncludeOS_logo.jpg" alt="IncludeOS" /></p>
<p><em>Alfred Bratterud is Assistant Professor and PhD scholar at Oslo and Akershus University College of Applied Science where he is currently working full time leading the development of IncludeOS at the NetSys research group.</em></p>
<p>We’ve finally lifted the lid on <a href="http://www.includeos.org">IncludeOS</a>, just in time for the
<a href="http://2015.cloudcom.org/">IEEE CloudCom</a> paper presentation recently. A preprint of the paper is
<a href="https://github.com/hioa-cs/IncludeOS/blob/master/doc/papers/IncludeOS_IEEE_CloudCom2015_PREPRINT.pdf">available from our repo</a>. However, we’ve done quite a lot of work
since the paper was written, so here’s an update on what IncludeOS is now, and
what you can expect in the near future.</p>
<h3 id="its-a-bit-like-a-jvm-but-for-x86-c">It’s a bit like a JVM, but for x86 C++</h3>
<p>A Java Virtual Machine is a portable language runtime environment. Java is
portable across hardware architectures and operating systems because it uses a
common instruction set. Once you’ve started a Java program, you can’t log into
it (unless your program itself provides the facilities), and you can’t boot up
any other programs inside it.</p>
<p>IncludeOS is like a safe language runtime for C++ programs, compiled into the
x86 instruction set. This has the obvious advantage of removing one layer of
abstraction, compared to Java: with hardware virtualization the code will
execute directly on the CPU. Like with a Java program and other unikernels, an
IncludeOS service is a program, with the language runtime attached. It’s
called IncludeOS because you start by saying <code class="language-plaintext highlighter-rouge">#include <os></code> and whatever your
service needs from the operating system will be included into the virtual
machine.</p>
<p>Think about the original reasons for creating Java; one main reason was
definitely portability. With pervasive hardware virtualization you can now get
similar portability for compiled C++ programs. We’re currently only portable
across x86 operating systems, but with VirtualBox, you can run the same binary
on Windows, Linux and MacOS X. What’s more, “Hello World” in IncludeOS on KVM
has about <em>a third of the memory footprint</em> of “Hello world” in Java, running
directly on the server, even when we take the whole Qemu process into account
(8.45 MB vs 28.29 MB)</p>
<p>From this interesting premise, rather than working towards running a JVM or
supporting another high-level language, we want to focus on creating an
intuitive and modern C++ hardware API, with minimal overhead. But of course,
if somebody wants to use IncludeOS as a platform for supporting high level
language runtimes, that would be great too. Before that can happen though,
we’ll have to get a stable API. We’re currently on 0.7.0, meaning that
anything can change at any time.</p>
<h3 id="networking-from-scratch">Networking from scratch</h3>
<p>One thing you’ll notice is that our network stack is currently being written
from scratch. Are we crazy? Possibly. We obviously considered — and wanted
to — port the stack from somewhere else, but not being intimately familiar
with any existing kernel, I found that to be even harder than just doing it
from scratch. I got a lot of help from reading the nicely annotated and
cross-linked <a href="http://www.jbox.dk/sanos/source/">SanOS source code</a>, and once the virtio network driver was
working, the rest wasn’t nearly as hard as one might think - at least up to
now. Clearly it will take time to make it complete, but the efforts it took to
get from a driver, up to rudimentary TCP was really insignificant compared to
the challenges we’ve faced when trying to integrate third-party libraries such
as newlib (Standard C) and libc++ (Standard C++). Getting to read RFC’s and
implementing the TCP handshake from scratch has really been a blast — a highly
recommended exercise.</p>
<h3 id="c-is-a-new-language">C++ is a new language</h3>
<p>If you haven’t looked at C++ after 2011, you should look now — “It feels like
a new language” (Bjarne Stroustrup). While you can always argue that adding
new feature won’t necessarily make the language any better, it most definitely
makes it more powerful. Here are a couple of really nice new language features
we’re using in IncludeOS:</p>
<ol>
<li>Real Lambdas. Not function pointers, not just call operators, but lambdas just like you know and love from functional languages.</li>
<li>C++11 implementation of delegates (i.e. pointers to a member function in a class instance), which are exactly as fast as function pointers. We’re using these everywhere. Reading <a href="https://github.com/hioa-cs/IncludeOS/blob/master/api/utility/delegate.hpp">the implementation</a> is a great way to have your brain explode, but they work like magic.</li>
<li>User defined literals, and std::literals for units. You can set timers by passing in <code class="language-plaintext highlighter-rouge">100ms</code>, <code class="language-plaintext highlighter-rouge">10s</code> or <code class="language-plaintext highlighter-rouge">50min</code> as parameters to the same function.</li>
<li>Shared pointers. These behave much like normal C-style pointers, but will keep track of the number of instances of itself. Whenever a shared pointer exits the last scope, the destructor of the object pointed to gets called. We use this to have network packets release their buffers back to a buffer store, once they’re no longer used by anyone.</li>
<li>A lot of wisdom, packed into concise <a href="https://github.com/isocpp/CppCoreGuidelines">guidelines</a> from the Standard C++ foundation (a.k.a. ISO C++ / the Jedi Council).</li>
</ol>
<p>Obviously, with C++ you can mess up the memory of your own program if you want
to, but compared to C you have so many more tools for writing encapsulated and
type-safe code. And of course, if performance or energy efficiency is key to
your app, C++ is an obvious choice. As a test case for the paper, a bootable
disk image consisting of a simple DNS server with OS included was shown to
require only 158 kb of disk space and to require 5-20% less CPU-time,
depending on hardware, compared to the same binary running on Linux. A lot of
this performance comes from greatly reduced complexity, but a lot also comes
for free by just using C++.</p>
<h3 id="we-want-to-make-node">We want to make Node++</h3>
<p>IncludeOS is pretty similar to Node.js in a few respects: It’s single threaded
at the moment, but highly efficient due to non-blocking I/O. This means that
you’ll have to use a callback-based programming style, just like with
Javascript. The events you’ll be subscribing to are all rooted in a physical
interrupt — in fact, you can add callbacks to interrupts directly, if you want.
An interesting thing to keep in mind is that I/O in modern hardware is
non-blocking by default, with Direct Memory Access (DMA). A network packet
arriving into IncludeOS will result in an interrupt, but only after the whole
packet is written directly into memory by the device. At that point, IncludeOS
will defer the interrupt, and as soon as the CPU is done with whatever it was
doing, it will call the callback delegate (possibly a lambda) of whoever’s
subscribing to that interrupt. That delegate can again use the same technique,
to fire events higher up the stack. In our view, this kind of event-based
programming fits hardware better than sequential, blocking programming. Also,
GUI-programming has always been like this, so the techniques are well known.</p>
<p>In the <a href="https://github.com/hioa-cs/IncludeOS/wiki/Roadmap">2016 Roadmap</a> our explicit goal is to become “Node++”: we want to
enable people to develop tiny, self-contained, Node.js-style web services,
with RESTful API’s, in highly efficient C++.</p>
<h3 id="thanks-for-the-warm-welcome">Thanks for the warm welcome!</h3>
<p>Opening up a repository after over a year, to much expectation from people you
look up to, is pretty scary. There are so many things I’d like to improve, fix
and add, but at some point you just have to let go and let it rip. The hope
was that opening up would help the process — and indeed I got a pull request
fixing all my typos the very next day. The inclusive response from
<a href="https://isocpp.org/">ISO C++</a>, and the immediate welcome on twitter from the <a href="https://mirage.io/">MirageOS</a> team
have made us feel like family, more than competition, which is what we’d like
to be. We hope to be able to contribute something back to the Unikernel effort
as a whole.</p>
<blockquote class="twitter-tweet" lang="en"><p lang="en" dir="ltr">The IncludeOS <a href="https://twitter.com/hashtag/unikernel?src=hash">#unikernel</a> is now free and <a href="https://twitter.com/hashtag/opensource?src=hash">#opensource</a>. Find our research prototype at <a href="https://t.co/1gnag1W2hA">https://t.co/1gnag1W2hA</a> <a href="https://twitter.com/hashtag/cloudcom?src=hash">#cloudcom</a> <a href="https://twitter.com/hashtag/includeos?src=hash">#includeos</a></p>— Alfred Bratterud (@AlfredBratterud) <a href="https://twitter.com/AlfredBratterud/status/671809686820143104">December 1, 2015</a></blockquote>
<script async="" src="//platform.twitter.com/widgets.js" charset="utf-8"></script>
<h3 id="try-it-out">Try it out!</h3>
<p>If you’re interested in playing around with IncludeOS, the best place to start
is at the <a href="https://github.com/hioa-cs/IncludeOS/blob/master/README.md">README</a>, right on the front of our repository. You’ll surely get
it to work if you have a clean Ubuntu 14.04 VM in VirtualBox, or an Ubuntu
14.04 server, but we have also been able to build on MacOS X directly. Once
you’ve built an image, it should run anywhere with VirtualBox or KVM!</p>
<p><em>Edit: discuss this post on <a href="http://devel.unikernel.org/t/includeos-is-now-free-and-open-source/23/1">devel.unikernel.org</a></em></p>
<p><em>Thanks to <a href="https://twitter.com/amirmc">Amir</a>, <a href="http://anil.recoil.org">Anil</a>, <a href="https://github.com/yallop">Jeremy</a> and <a href="http://mort.io">Mort</a> for their comments on earlier drafts.</em></p>
Contain Your Unikernels!2015-11-26T18:00:00+00:00/blog/2015/contain-your-unikernels
<p>
<img alt="screenshot" src="http://unikernel.org/images/contain-your-unikernels-crop.png" style="box-shadow: 5px 5px 10px;" />
</p>
<p>After DockerCon EU in Barcelona several people asked me: “Is this for real?”.
Yes it is, and today we are releasing the code for the entire “Unikernels, meet
Docker!” <a href="http://unikernel.org/blog/2015/unikernels-meet-docker">demo</a> on GitHub.</p>
<p>To get started, clone the <a href="https://github.com/Unikernel-Systems/DockerConEU2015-demo">DockerConEU2015-demo</a> repository and follow
the instructions in README.md. You will need a Linux host with Docker and KVM
installed.</p>
<p>Apart from the MySQL, Nginx and PHP with Nibbleblog unikernels shown in the
demo, the repository also contains some simpler examples to get you started
that we did not have time to show live in the short time-slot. There’s also an
in-progress <a href="https://github.com/mato/opam-rumprun">MirageOS/KVM port</a>, so stay tuned for a future post on
that.</p>
<p>Presented as a ‘cool hack’ in the closing session of the conference, this demo
is just a taste of what is possible. Next, I’m going to work with the wider
unikernel and Docker developer community on a production quality version of
this demo. The goal is to make unikernel technology easily accessible to as
many developers as possible!</p>
<p>Personally, I would like to thank <a href="http://amirchaudhry.com/">Amir Chaudhry</a>, <a href="https://github.com/justincormack">Justin
Cormack</a>, <a href="http://anil.recoil.org">Anil Madhavapeddy</a>, <a href="http://mort.io/">Richard Mortier</a>, <a href="http://somerandomidiot.com">Mindy
Preston</a> and <a href="https://github.com/yallop">Jeremy Yallop</a> for helping me put the demo
together, <a href="http://docker.com/">Docker</a> for giving us the opportunity to demo at DockerCon,
everyone working on the <a href="http://wiki.rumpkernel.org/Repo:-rumprun">Rumprun unikernel</a> and all the other Open
Source projects that made this demo possible.</p>
<p>Now, go <a href="https://github.com/Unikernel-Systems/DockerConEU2015-demo">try it out</a> and contain your unikernels!</p>
<p><em>Edit: discuss this post on <a href="http://devel.unikernel.org/t/contain-your-unikernels/21/1">devel.unikernel.org</a></em></p>
Unikernels, meet Docker!2015-11-17T17:30:00+00:00/blog/2015/unikernels-meet-docker
<p>Today, unikernels took to the stage at DockerCon EU in Barcelona!</p>
<p>As part of the Cool Hacks session in the closing keynote, Anil Madhavapeddy
(<a href="https://mirage.io/">MirageOS</a> project lead), showed how unikernels can be treated as any
other container. He first used Docker to build a unikernel microservice and
then followed up by deploying a real web application with database, webserver
and PHP code all running as distinct unikernel microservices built using
<a href="http://rumpkernel.org/">Rump Kernels</a>. Docker managed the unikernels just like Linux
containers but <em>without</em> needing to deploy a traditional operating system!</p>
<p>This kind of integration helps put unikernels into the hands of developers
everywhere and combines the familiar tooling and real-world workflows of the
container ecosystem with the improved security, efficiency and specialisation
of unikernels. We’ll finish off this post with details of how <em>you</em> can get
involved — but first, before we go into Anil’s demonstration in more detail,
some background about why unikernels matter, and why it makes sense to use
Docker this way.</p>
<h3 id="why-unikernels">Why Unikernels?</h3>
<p>As companies have moved to using the cloud, there’s been a growing trend
towards single-purpose machine images, but it’s clear that there is
significant room for improvement. At present, every VM has to host a copy of
the OS on which the application runs, especially where strong isolation is
required.</p>
<p>These VM-hosted applications are also affected by OS vulnerabilities,
including exploits that have no relation to applications’ day-to-day functions.
For example, USB drivers — present in an OS but irrelevant to cloud
deployments — have had <a href="http://www.openwall.com/lists/oss-security/2014/09/11/21">multiple vulnerabilities</a> which allow
arbitrary code execution. Patching such code creates a needless maintenance
burden.</p>
<p>Unikernels take a different approach: application code is linked against only
the OS components it specifically requires to produce a specialised, single
address space machine image — thus eliminating unnecessary code. Built using
‘library operating system’ technology, unikernels provide many benefits
compared to a traditional OS, including:</p>
<ul>
<li><em>Improved security properties</em> — as unikernels contain no unnecessary code
deployed, the application’s attack surface is dramatically reduced.</li>
<li><em>Smaller footprints</em> — unikernel code bases are typically several orders of
magnitude smaller than their traditional equivalents and they can be managed
much more easily.</li>
<li><em>Fine-grained optimisation</em> — as unikernels are constructed through a
coherent compiler tool-chain, whole-system optimisation can be carried out
across device drivers and application logic, potentially improving
specialisation further.</li>
<li><em>Fast boot times</em> — as unikernels can boot in less than a second,
provisioning can become highly dynamic.</li>
</ul>
<!-- In addition, creating these library operating systems with modern, high-level languages also means that language features such as type-safety and compile-time checks can hugely improve the quality of deployed software. Combining these benefits with legacy applications will allow us to create next-generation secure services. -->
<p>These benefits are particularly relevant to <a href="http://martinfowler.com/articles/microservices.html">microservices</a> and the
developing concept of ‘immutable infrastructure’ — where VMs are treated as
disposable artefacts and can be regularly re-provisioned solely from
version-controlled code. Modifying such VMs directly isn’t permitted: all
changes must be made to the source code itself.</p>
<p>Unikernels naturally lend themselves to both the microservice architecture
<em>and</em> the concept of immutable infrastructure: both source code and generated
binary are compact enough to be easily version-controlled. If the traditional
stack has allowed us to move towards microservices, then unikernels will move
us towards the world of immutable nanoservices.</p>
<p>Although unikernels provide a path, there are significant challenges to their
adoption in production settings. The unikernel ecosystem is only just taking
off, and this new technology needs to fit in with existing workflows and
tooling.</p>
<p>Enter Docker!</p>
<h3 id="why-docker">Why Docker?</h3>
<p>Linux containers have allowed developers to move much more quickly towards
microservices by allowing a traditional OS to provide functionality to
multiple ‘containerised’ applications sitting above it. Those containers
remain distinct and thus can be independently replaced or modified, a core
piece of the microservices architectural pattern.</p>
<p>Although containerisation technology has been available for some time, there’s
been a recent and rapid increase in the pace of adoption. The last few years
have seen a proliferation of tools that make it easier to use containers at
scale, including registries of ready-made images, tools for orchestration, and
much more. Led and fostered by Docker, this has produced a vibrant, open and
growing ecosystem, which is helping improve everyone’s development workflows.</p>
<p>With an increasing number of supported tools and infrastructure, it’s become
clear that the ecosystem is about much more than just Linux containers
themselves. Can unikernels fit in this ecosystem? If so, where would
unikernels sit in relation to containers?</p>
<p>Containers and unikernels actually sit on a continuum. On the one hand, we
have the traditional method of placing a full OS stack in a VM with a single
application on top. A natural next step is to use containers which run on top
of a single OS, giving better resource usage and allowing each application to
be more self-contained. When viewed this way, unikernels are just another step
on this path and can be thought of as extreme, self-contained applications.
The challenge is to make unikernels as easy to use as containers have become today.</p>
<p><img src="http://unikernel.org/images/intro-post/specialisation.png" alt="Containers and unikernels on a continuum" /></p>
<p>The obvious first step in addressing that challenge is to integrate unikernels
with the existing container infrastructure, specifically the Docker tools and
ecosystem. This helps us to get unikernels into the hands of developers
everywhere, with a widely used and understood packaging model and runtime
framework, effectively by making unikernels just another type of container.</p>
<p>It also unlocks the entire container ecosystem of tools for use with
unikernels, including orchestration and whatever else may be around the corner.
Adoption of existing toolchains will accelerate the progress of unikernels and
also demonstrates the flexibility and breadth of the Docker ecosystem. By
using Docker to abstract away the complexity of the underlying OS, a developer
can chose how they ‘containerise’ their application, whether they target a
traditional Linux container, or a new unikernel ‘container’.</p>
<h3 id="dockercon-demo">DockerCon Demo</h3>
<p>It is exactly these first steps enabling developers to build and run
real-world unikernel microservices, using existing Docker tools, which Anil
demonstrated at DockerCon EU today!</p>
<p>First he used Docker to build a unikernel microservice, and then he ran a
cluster of unikernels to deploy a real web application with database,
webserver and PHP code. The whole build system is wrapped in an easy-to-use
Dockerfile and each microservice is turned into a specialised unikernel. Those
unikernels run in their own KVM virtual machine with hardware protection.
Docker manages the unikernel containers just like Linux containers, including
networking!</p>
<p>This early work makes unikernels a usable target for a Docker deployment!
Since unikernels can now be managed by Docker, it can bring all the benefits
of the existing ecosystem to the orchestration and management of unikernels.</p>
<p>The demo consisted of using the typical components found in a LAMP stack,
specifically Nginx, MySQL and PHP. These unmodified, off-the-shelf components
were built using the Docker toolchain but instead of the typical OS they were
built as unikernels! A cluster of these unikernels was created where each of
them was specialised for the particular app they were running. Watch the
screencast below for more details!</p>
<div class="flex-video">
<iframe width="560" height="315" src="https://www.youtube.com/embed/CkfXHBb-M4A" frameborder="0" allowfullscreen=""></iframe>
</div>
<h3 id="get-involved">Get involved!</h3>
<!-- The code for the demo example is available at: TODO:XXXX. Try this out for yourself to get an idea of the benefits of such integration.
-->
<p>The demo described here is just the beginning. There are many implementations
of unikernels and there’s plenty of work ahead to ensure they can all reap the
benefits of integration, as well as improving Docker itself to make the most
of these new technologies. Look over the collection of unikernel projects and
contribute your experiences to this blog!</p>
<p><em><strong>Update (26 Nov)</strong> — We’ve released the code so you can follow what we did.
See the blog post at “<a href="http://unikernel.org/blog/2015/contain-your-unikernels/">Contain Your Unikernels!</a>” then check out the
<a href="https://github.com/Unikernel-Systems/DockerConEU2015-demo/">code repository</a>!</em></p>
<p><em>Edit: discuss this post on <a href="http://devel.unikernel.org/t/unikernels-meet-docker/20/1">devel.unikernel.org</a></em></p>
<p><em>Thanks to <a href="http://anil.recoil.org">Anil</a>, <a href="https://github.com/balrajsingh">Balraj</a>, <a href="https://github.com/dsheets">David</a>, <a href="https://github.com/yallop">Jeremy</a>, <a href="https://github.com/justincormack">Justin</a>,
<a href="https://lucina.net">Martin</a>, <a href="http://somerandomidiot.com">Mindy</a> and <a href="http://gazagnaire.org">Thomas</a> for their comments on earlier drafts.</em></p>
A community site for Unikernels2015-11-17T16:30:00+00:00/blog/2015/unikernelorg-site
<p><img src="http://unikernel.org/images/noun_192875_cc.png" alt="community" /></p>
<p>Word about unikernels is spreading and more people are trying to learn about this new approach to programming the cloud. This community site aims to collate information about the various projects and provide a focal point for early adopters to understand more about the technology and become involved in the projects themselves.</p>
<p><small>Image Credit: Blake Thomson from Noun Project</small></p>