Cheap and Easy Virtual Machine Containers with LXC

The thing that most virtual-machine solutions have in common is the overhead required to run them. There’s a substantial resource cost for the management of these containers, aside from that for the machine itself.

There’s an alternative to all of that, called LXC. In fact, though it used to require an impossible number of steps to configure, it’s now sufficient to just pass in a couple of command-line parameters and leave everything else to defaults.

LXC is somewhere between chroot and QEMU. It imposes resource control using the cgroups functionality that comes packaged in the kernel, which is, essentially, the next evolution of ulimit. Although the resource-control is somewhat disabled by default, you can set limits even so far as disk I/O rates.

It’s important to know that, though LXC works terrifically, it should only be used in either personal systems or any other system that’s sufficiently fenced-off from outside threats. This is because it doesn’t benefit from 100% isolation like most VM’s do (a tradeoff for its lightweightedness). An example of this is that the container shares the same sysfs as the host, due to limitations in sysfs. Therefore, changing sysfs from the container will affect the host.

Though there are security concerns, I have been told authoritatively that there is a less likely chance of a rogue application causing issues for the larger host than any other critical problem that systems usually encounter in production. So, a couple of built-in security concessions are the only plausible risks.

System Containers

There’s an easy way and a hard way to create system containers. The hard way is to create and populate it with all of the configuration that is required of any new system (see here). The easy way is to simply use the “lxc-create” tool and tell it to follow a template.

These are the templates available in my installation:

$ ls -1 /usr/share/lxc/templates

You can only use a template that’s compatible with the system on which you are working. Otherwise, you’ll find that “yum”, for instance, is missing if you try to build a Fedora instance on Ubuntu, as well as categorically-similar issues with the other templates. On my Ubuntu, I can create containers with the “busybox” (which creates instantaneously), “debian” and “ubuntu” (7 minutes), “ubuntu-cloud” (6 minutes), and “sshd” (see below) templates. Note that any required, downloaded images are cached, and subsequent builds only take a minute or two.

The current steps to build an Ubuntu container (from my Ubuntu box, after installing the lxc package):

$ sudo lxc-create -t ubuntu -n <container name>

Depending on the template, you might see something like this upon completion:

# The default user is 'ubuntu' with password 'ubuntu'!
# Use the 'sudo' command to run tasks as root in the container.

I named my container “ubuntu-2”. The container directories get created in /var/lib/lxc, and have a reasonable size:

$ ls -l /var/lib/lxc
total 4
drwxr-xr-x 3 root root 4096 Nov  6 02:32 ubuntu-2

$ sudo du -sh /var/lib/lxc/ubuntu-2/
263M    /var/lib/lxc/ubuntu-2/

To start the container as a daemon:

$ sudo lxc-start -n ubuntu-2 -d

Or, to start the container as a foreground machine, complete with console (using another, BusyBox-based, container, which shows this better):

$ sudo lxc-start -n busybox-1
udhcpc (v1.20.2) started
Sending discover...
Sending select for
Lease of obtained, lease time 3600

Please press Enter to activate this console. 

To list the currently-running containers:

$ sudo lxc-ls --fancy
NAME       STATE    IPV4        IPV6  AUTOSTART  
busybox-1  STOPPED  -           -     NO         
debian-1   RUNNING  -     NO         
ubuntu-1   RUNNING  -     NO         
ubuntu-2   RUNNING  -     NO         

Very cool. To connect via SSH:

$ ssh ubuntu@
The authenticity of host ' (' can't be established.
ECDSA key fingerprint is 73:8c:31:53:76:36:93:6e:59:ee:3f:d3:6f:27:13:c7.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added '' (ECDSA) to the list of known hosts.
ubuntu@'s password: 

The programs included with the Ubuntu system are free software;
the exact distribution terms for each program are described in the
individual files in /usr/share/doc/*/copyright.

Ubuntu comes with ABSOLUTELY NO WARRANTY, to the extent permitted by
applicable law.

Welcome to Ubuntu 13.04 (GNU/Linux 3.8.0-30-generic i686)

 * Documentation:

To stop the container:

$ sudo lxc-stop -n ubuntu-2

If you don’t want, or need, to build a new system, you can also configure an “ssh container”, where a read-only mount of the current filesystem is combined with an SSH server to create the facade of a separate machine instance. It’s unclear whether there’s a provision to allow changes (such as implementing a ramdisk to produce the illusion of a read-write experience similar to a disc-based “live” Linux distribution).

Application Containers

In addition to hosting “system” containers, LXC can also host “application” containers. Quite obviously, the latter simply host applications with all of the benefits of the resource-control that we’ve already mentioned, as well as, most likely, its security limitations.

$ sudo lxc-execute -n <container name> <command>

You might see an error like the following:

$ sudo lxc-execute -n app_container_1 touch /tmp/aa
lxc-execute: Permission denied - failed to change apparmor profile to lxc-container-default
lxc-execute: invalid sequence number 1. expected 4
lxc-execute: failed to spawn 'app_container_1'

The workaround is:

$ cat > test.conf <<EOF
lxc.aa_profile = unconfined
lxc.rootfs = /

$ sudo lxc-execute -f test.conf -n app-container-1 touch /tmp/aa

When the application container launches, you’ll be able to see it in the lxc-ls list (above). You’ll also be able to find it in the ps list. Obviously the command-above just touches a file before returning, so it won’t be alive long-enough for you to be able to see it running.

Development Support

Naturally, everything we’ve mentioned can be done from code (Python, Lua, and Go, currently). This is a Python example mentioned on the LXC homepage (whose link was at the beginning of the article):

import lxc
container = lxc.Container("p1")

As mentioned, LXC isn’t the right-kind of container for serving from the DMZ in a corporate environment, but it is awesome as a fast, easily-constructed, hold-no-prisoners system container, where you want to run a dozen on a commodity box with minimal resource consumption.

Create a Development OpenStack Instance in Two Steps

OpenStack is the result of collaboration between Rackspace and NASA. As of this year, it’s the hottest cloud platform available. The whole thing is API driven (including a subset of APIs that are Amazon compatible). It’s also built on Python, so extensibility comes packaged.

Though building a cloud is a timely task, developers have DevStack: a rapid utility with which to build a full, development (read: non-secure) OpenStack instance on a single box. As of right now, it finishes downloading and configuring in about seven-minutes on commodity hardware. Since they build on the principle of Git-clones rather than packaging, there are no/minimal dependency problems, nor are there any compilations. It’s a joy.

Installation steps:

git clone
cd devstack && ./

The only input required during the standard set-up is a few passwords near the beginning. Officially, Ubuntu 12.04 (Precise), Fedora 18, and CentOS/RHEL 6.4 are supported, at this time, but I was effortlessly able to build on Ubuntu 13.04, as well.

Once a few hundred megabytes have been downloaded and configured, the process will output something similar to the following and quit. The platform will be started and good to go to. You might try to deploy a machine instance using one of the packaged images. You shouldn’t have any problem, and, if you’re familiar with AWS, you will already know the steps.

Horizon is now available at
Keystone is serving at
Examples on using novaclient command line is in
The default users are: admin and demo
The password: test
This is your host ip: completed in 422 seconds.

Horizon is the dashboard. The web-server is going to be running on :80, so if you already have a web-server running there, then stop it first.

OpenStack - 1 - Login

The dashboard will have two default users: “admin” and “demo”. Their password will be the password you gave during initial startup.

To stop the cloud, run “./”.

For a walkthrough of what the setup (“”) script does, go here.

Selected Screenshots

OpenStack - 2 - Hypervisors

OpenStack - 3 - Instances

OpenStack - 3 - Instance Details

OpenStack - 4 - Flavors

OpenStack - 5 - Images

OpenStack - 6 - System Info 3

OpenStack - 6 - System Info 2

OpenStack - 6 - System Info 1