Better Resource Throttling for Processes with cgroups

Traditionally, the only solution for resource control for processes has been ulimit. It allows you a short list of constraints that apply to every process in the system:

$ ulimit -a
core file size          (blocks, -c) 0
data seg size           (kbytes, -d) unlimited
scheduling priority             (-e) 0
file size               (blocks, -f) unlimited
pending signals                 (-i) 30890
max locked memory       (kbytes, -l) 64
max memory size         (kbytes, -m) unlimited
open files                      (-n) 1024
pipe size            (512 bytes, -p) 8
POSIX message queues     (bytes, -q) 819200
real-time priority              (-r) 0
stack size              (kbytes, -s) 8192
cpu time               (seconds, -t) unlimited
max user processes              (-u) 30890
virtual memory          (kbytes, -v) unlimited
file locks                      (-x) unlimited

However, this does nothing for per-process settings or anything related to throttling velocities. This is where the cgroups kernel extension comes in. In addition to the above, it can even allow hierarchies of settings to be defined (among many other features).

Though cgroups can be used in everyday system administration, it is also used in projects such as LXC to produce light-weight “system” and “application” containers, where the former simulates a virtual-machine with less overhead (and slightly less isolation), and the latter can run individual processes.

As a comparison between the granularity of the ulimit settings (above) and the cgroups settings, here is a list of the latter:

blkio.io_merged
blkio.io_queued
blkio.io_service_bytes
blkio.io_serviced
blkio.io_service_time
blkio.io_wait_time
blkio.reset_stats
blkio.sectors
blkio.throttle.io_service_bytes
blkio.throttle.io_serviced
blkio.throttle.read_bps_device
blkio.throttle.read_iops_device
blkio.throttle.write_bps_device
blkio.throttle.write_iops_device
blkio.time
blkio.weight
blkio.weight_device
cgroup.clone_children
cgroup.event_control
cgroup.procs
cpuacct.stat
cpuacct.usage
cpuacct.usage_percpu
cpu.cfs_period_us
cpu.cfs_quota_us
cpu.rt_period_us
cpu.rt_runtime_us
cpuset.cpu_exclusive
cpuset.cpus
cpuset.mem_exclusive
cpuset.mem_hardwall
cpuset.memory_migrate
cpuset.memory_pressure
cpuset.memory_pressure_enabled
cpuset.memory_spread_page
cpuset.memory_spread_slab
cpuset.mems
cpuset.sched_load_balance
cpuset.sched_relax_domain_level
cpu.shares
cpu.stat
devices.allow
devices.deny
devices.list
hugetlb.2MB.failcnt
hugetlb.2MB.limit_in_bytes
hugetlb.2MB.max_usage_in_bytes
hugetlb.2MB.usage_in_bytes
memory.failcnt
memory.force_empty
memory.limit_in_bytes
memory.max_usage_in_bytes
memory.memsw.failcnt
memory.memsw.limit_in_bytes
memory.memsw.max_usage_in_bytes
memory.memsw.usage_in_bytes
memory.move_charge_at_immigrate
memory.oom_control
memory.soft_limit_in_bytes
memory.stat
memory.swappiness
memory.usage_in_bytes
memory.use_hierarchy
notify_on_release
release_agent
tasks

Though cgroups just sits behind a sysfs interface (/sys/fs/cgroup), and settings are configured and processes are enrolled by simply echoing them into the right interface files, a detailed description of how to configure cgroups lays outside the scope of this article. You might go here for more info.

Advertisements