cgroups is a Linux kernel feature to limit, account, and isolate resource usage of process
groups.
This work was started by engineers at Google in 2006 under the name "process containers".
In late 2007, it was renamed to "Control Groups" due to the confusion caused by multiple meanings
of the term "container" in the Linux kernel, and merged into kernel version 2.6.24.
Since then, many new features and controllers have been added, such as support for kernfs,
firewalling, and the introduction of a unified hierarchy.
Features One of the design goals of cgroups is to provide
a unified interface to many different use cases, from controlling single processes to
whole operating system-level virtualization.
Cgroups provides: Resource limitation: groups can be set to
not exceed a set memory limit — this also includes file system cache.
The original paper was presented at Linux Symposium and can be found at Containers:
Challenges with the memory resource controller and its performance.
Prioritization: some groups may get a larger share of CPU or disk I/O throughput.
Accounting: to measure how much resources certain systems use for e.g. billing purposes.
Control: freezing groups or checkpointing and restarting.
Usage A control group is a collection of processes
that are bound by the same criteria.
These groups can be hierarchical, where each group inherits limits from its parent group.
The kernel provides access to multiple controllers through the cgroup interface.
For instance, the "memory" controller limits memory use, "cpuacct" accounts CPU usage,
etc.
Control groups can be used in multiple ways: By accessing the cgroup virtual file system
manually.
By creating and managing groups on the fly using tools like cgcreate, cgexec, and cgclassify.
Through the "rules engine daemon" that can automatically move processes of certain users,
groups, or commands to cgroups as specified in its configuration.
Indirectly through other software that uses cgroups, such as Linux Containers virtualization,
libvirt, systemd, Open Grid Scheduler/Grid Engine, and Google's lmctfy.
The Linux kernel documentation contains full technical details of the setup and use of
control groups.
Redesign Redesign of cgroups was started in 2013, with
additional changes brought by versions 3.15 and 3.16 of the Linux kernel.
Namespace isolation While not technically part of the cgroups
work, a related feature of the Linux kernel is namespace isolation, where groups of processes
are separated such that they cannot "see" resources in other groups.
For example, a PID namespace provides a separate enumeration of process identifiers within
each namespace.
Also available are mount, UTS, network and SysV IPC namespaces.
The PID namespace provides isolation for the allocation of process identifiers, lists of
processes and their details.
While the new namespace is isolated from other siblings, processes in its "parent" namespace
still see all processes in child namespaces—albeit with different PID numbers.
Network namespace isolates the network interface controllers, iptables firewall rules, routing
tables etc.
Network namespaces can be connected with each other using the "veth" virtual Ethernet device.
"UTS" namespace allows changing the hostname.
Mount namespace allows creating a different file system layout, or making certain mount
points read-only.
IPC namespace isolates the System V inter-process communication between namespaces.
User namespace isolates the user IDs between namespaces.
Namespaces are created with the "unshare" command or syscall, or as new flags in a "clone"
syscall.
The "ns" subsystem was added early in cgroups development to integrate namespaces and control
groups.
If the "ns" cgroup was mounted, each namespace would also create a new group in the cgroup
hierarchy.
This was an experiment that was later judged to be a poor fit for the cgroups API, and
removed from the kernel.
Linux namespaces were inspired by the more general namespace functionality used heavily
throughout Plan 9 from Bell Labs.
Unified hierarchy Whenever designing a software, the software
engineer picks solutions which overall best blend the exigencies regarding stability,
security, performance, as well as maintainability, programmability and usability.
By their nature, they balance each other, e.g., a mighty API to user space, that doesn't
offer too much functionality, but carelessly exposes some key inner working, might seriously
compromise stability and security.
That is especially true if that software is part of the Linux kernel.
Tejun Heo, decided to alter cgroups to prevent these scenarios.
He designed and implemented a unified hierarchy with only one user space entity that has exclusive
access to the facilities offered by cgroups.
Kernfs was introduced into the Linux kernel with version 3.14, the main author being Tejun
Heo.
One of the main motivators for a separate kernfs is the cgroups file-system.
Kernfs is basically the splitting off of some of the sysfs logic into an independent entity
so that other kernel subsystems can more easily implement their own virtual file-system with
handling for device connect and disconnect, dynamic creation and removal as needed or
unneeded, and other attributes.
Redesign continued into version 3.15 of the Linux kernel.
It is anticipated that, in the future, debugfs will also become based on kernfs.
Kernel memory control groups Kernel memory control groups were merged into
version 3.8 of the Linux kernel mainline.
The kmemcg controller can limit the amount of memory that the kernel can utilize to manage
its own internal processes.
Adoption Various projects are using cgroups as their
basis, including the following: CoreOS
Docker Kubernetes
lmctfy LXC
Mesos and Mesosphere See also
Operating system–level virtualization implementations References
External links Linux kernel documentation on cgroups
Linux kernel Namespaces and cgroups by Rami Rosen