23. Dec 2005
Recent state of udevHotplugging
Since version 2.6 the kernel is capable to add or remove almost any device on a running system. It is necessary to propagate these changes to userspace, to configure the devices at discovery, or to notify users of the device about the state change. In addition to that, almost all interesting kernel state is exported by the sysfs filesystem. For every device the kernel has detected and initialized, a directory with the device name is created, containing attribute files with device specific properties. Everytime a device is added or removed, the kernel sends an event to make userspace aware of that.
The hotplug package
With every kernel event a /sbin/hotplug process was started. That process invoked a script in the /etc/hotplug/*.agent directory, selected by the subsystem the kernel has passed with the event. These agents scripts possibly loaded modules or prepared the device for userspace to use.
Replacement of hotplug by udev
The hotplug infrastructure made it possible to run udev to present only the current available devices in the /dev directory. Udev started as just a /etc/hotplug.d/ user. It plugged into the hotplug event chain like any other user of the hotplug infrastructure. With the growing adoption of udev, it was faced to work around various problems from sysfs timing issues, out-of-order events of the simple forked helper processes and out-of-memory situations, if too many event processes had been forked in a very short time. With every workaround, udev took over parts of the former hotplug infrastructure, until udev just replaced the entire hotplug package.
Kernel device event management
On a udev system, besides the maintenance of the dynamic /dev directory, the kernel driver core event handling is handed-over exclusively to udev. One of the first actions during bootup, the /dev directory gets a tmpfs filesystem mounted. All initial and static device nodes from the /lib/udev/devices directory are copied to the empty /dev directory. After that, the udevd daemon is started. The daemon listens to uevents from the driver core. Therefore the execution of /sbin/hotplug is not necessary and completely disabled by setting /sys/kernel/uevent_helper (or /proc/sys/kernel/hotplug on older kernels) to an empty string. All added or removed devices will cause an uevent to be sent to the udev daemon, which runs an event process to match against udev rules and possibly create a device node and symlinks and run specified programs to set up the device.
Event processing
The udev daemon reads and parses all provided rules from the /etc/udev/rules.d/*.rules files once at startup and keeps them in memory. If rules files are changed, added or removed, the daemon receives an inotify event and updates the in-memory representation of the rules. Every event matches against the set of provides rules. The rules can add or change event environment keys, request a specific name for the device node to be created, add symlinks pointing to the node or add programs to be run after the device node is created. The driver core uevents are received from a kernel netlink socket, which look like this on the wire:
recv(4, "add@/class/input/input9/mouse2\0ACTION=add\0
DEVPATH=/class/input/input9/mouse2\0SUBSYSTEM=input\0
SEQNUM=1064\0PHYSDEVPATH=/devices/pci0000:00/0000:00:1d.1/usb2/2-2/2-2:1.0\0
PHYSDEVBUS=usb\0PHYSDEVDRIVER=usbhid\0MAJOR=13\0MINOR=34\0", 2048, 0) = 221
An udev rule can match on any property the kernel adds to the event itself, exports in sysfs or the rule can request additional information from external programs. The rules syntax and the provided keys to match or import data is described in the udev man pages. An advanced example is the persistent device naming, implemented to provide stable names for all disks devices - regardless of their order of recognition or the connection used to plug the device:
/dev/disk
|-- by-id
| |-- scsi-SATA_HTS726060M9AT00_MRH453M4HWHG7B -> ../../sda
| |-- scsi-SATA_HTS726060M9AT00_MRH453M4HWHG7B-part1 -> ../../sda1
| |-- scsi-SATA_HTS726060M9AT00_MRH453M4HWHG7B-part6 -> ../../sda6
| |-- scsi-SATA_HTS726060M9AT00_MRH453M4HWHG7B-part7 -> ../../sda7
| |-- usb-Generic_STORAGE_DEVICE_02773 -> ../../sdd
| `-- usb-Generic_STORAGE_DEVICE_02773-part1 -> ../../sdd1
|-- by-label
| |-- Photos -> ../../sdd1
| |-- SUSE10 -> ../../sda7
| `-- devel -> ../../sda6
|-- by-path
| |-- pci-0000:00:1f.2-scsi-0:0:0:0 -> ../../sda
| |-- pci-0000:00:1f.2-scsi-0:0:0:0-part1 -> ../../sda1
| |-- pci-0000:00:1f.2-scsi-0:0:0:0-part6 -> ../../sda6
| |-- pci-0000:00:1f.2-scsi-0:0:0:0-part7 -> ../../sda7
| |-- pci-0000:00:1f.2-scsi-1:0:0:0 -> ../../sr0
| |-- usb-02773:0:0:2 -> ../../sdd
| |-- usb-02773:0:0:2-part1 -> ../../sdd1
`-- by-uuid
|-- 159a47a4-e6e6-40be-a757-a629991479ae -> ../../sda7
|-- 3e999973-00c9-4917-9442-b7633bd95b9e -> ../../sda6
`-- 4210-8F8C -> ../../sdd1
Event queue management
The udev daemon takes care of the right order of event execution and serializes events for devices which depend on other events. Events for child devices will be delayed until the event for the parent device has returned. That way, partition events will wait for the main block device event to finish, so the partition event can import the parents information, already stored in the udev database.
Udevd throttles the execution of events and limits the running processes, if there are already too many in running state. If the event carries a TIMEOUT key passed by the kernel, the event will run immedediately, regardless of the throttling state.
The current state of the event queue is visible in /etc/.udev/queue. If that directory exists, then events are currently queued or already running. Every event in this directory is represented as a symlink to the corresponding sysfs device. With the removal of the last event, the whole directory goes away. All event processes which have failed, cause of an error, or executed programs returning a failure, will be represented in /etc/.udev/failed. A later successful event for the same device, will remove the symlink from that directory.
Booting and Coldplugging
All device events happening during the boot process until the udev daemon is running are obviously lost, because the infrastructure to handle these events lives on the root filesystem and is not available at that time. To cover that hole, the kernel now provides an uevent file for every device in the sysfs filesystem. By writing add to that file, the kernel will just emit exactly the same event again, as the one lost during bootup. A simple loop over all uevent files in /sys triggers all events again, to create the device nodes and do the device setup.
From userspace, there is no difference between a device coldplug sequence and a device discovery during runtime. In both cases, the same rules are used to match and the same configured programs are run. This replaces the former *.agent and *.rc scripts in the /etc/hotplug directory.
Certain steps during bootup will need to synchronize with the kernel event handling. This can be accomplished by watching the event queue directory at /etc/.udev/queue. If events fail cause of missing dependencies at this stage, they can be retried at a later stage, by picking up the stored symlinks from the /etc/.udev/failed directory.
Kernel module loading
The kernel bus drivers probe for devices. For every detected device, the kernel creates an internal device structure and the driver core sends an event to udev. Devices identify themselves by an id, which tells what kind of device it is. Usually these id's consist of vendor and product id and other susbsytem specific values. Every bus has its own scheme for these id's. The kernel takes this information, composes a MODALIAS string from it and and sends that string along with the event. For an USB mouse it looks like this: MODALIAS=usb:v046DpC03Ed2000dc00dsc00dp00ic03isc01ip02.
Every device driver carries a list of known id's for devices it can handle. The list is contained in the kernel module file itself. The program depmod reads the id lists and creates the file modules.alias in the kernel's /lib/modules directory for all currently available modules. With this infrastructure, module loading is as easy as calling modprobe for every event that carries a MODALIAS key. If modprobe $MODALIAS is called, it matches the device alias composed for the device with the aliases provided by the module. If a matching entry is found, that module will be loaded. This replaces the entire hotplug shell script logic to search matching modules in the *.map files. The former /etc/hotplug/blacklist file is replaced by the module-init-tools native blacklist command.
Debugging events
udevmonitor visualizes the driver core events and the udev event processes. This is a sequence of events while connecting an USB mouse:
UEVENT[1132632714.285362] add@/devices/pci0000:00/0000:00:1d.1/usb2/2-2 UEVENT[1132632714.288166] add@/devices/pci0000:00/0000:00:1d.1/usb2/2-2/2-2:1.0 UEVENT[1132632714.309485] add@/class/input/input6 UEVENT[1132632714.309511] add@/class/input/input6/mouse2 UEVENT[1132632714.309524] add@/class/usb_device/usbdev2.12 UDEV [1132632714.348966] add@/devices/pci0000:00/0000:00:1d.1/usb2/2-2 UDEV [1132632714.420947] add@/devices/pci0000:00/0000:00:1d.1/usb2/2-2/2-2:1.0 UDEV [1132632714.427298] add@/class/input/input6 UDEV [1132632714.434223] add@/class/usb_device/usbdev2.12 UDEV [1132632714.439934] add@/class/input/input6/mouse2
The UEVENT lines show the events the kernel sends over netlink, the UDEV lines show the finished udev event handlers. The timing is printed in microseconds. The time between UEVENT and UDEV is the time udev took to process this event or was queued to synchronize it with other events.
udevmonitor --env shows the complete event environment:
UDEV [1132633002.937243] add@/class/input/input7 UDEV_LOG=3 ACTION=add DEVPATH=/class/input/input7 SUBSYSTEM=input SEQNUM=1043 PHYSDEVPATH=/devices/pci0000:00/0000:00:1d.1/usb2/2-2/2-2:1.0 PHYSDEVBUS=usb PHYSDEVDRIVER=usbhid PRODUCT=3/46d/c03e/2000 NAME="Logitech USB-PS/2 Optical Mouse" PHYS="usb-0000:00:1d.1-2/input0" UNIQ="" EV=7 KEY=70000 0 0 0 0 0 0 0 0 REL=103
Udev also sends messages to syslog. The default syslog priority, which controls which messages are sent to syslog, is specified in the udev configuration file /etc/udev/udev.conf. The log priority of the running daemon can be changed with udevcontrol log_priority=<level/number>.
Files from udev, replacing the hotplug packages content
/etc/hotplug/*.agent -> no longer needed or moved to /lib/udev
/etc/hotplug/*.rc -> replaced by /sys/*/uevent trigger
/etc/hotplug/blacklist -> replaced by "blacklist" option in modprobe.conf
/etc/dev.d/* -> replaced by udev rule RUN key
/etc/hotplug.d/* -> replaced by udev rule RUN key
/sbin/hotplug -> replaced by udevd listening to netlink, only used in
initramfs, until the rootfs can be mounted, than it is disabled
/etc/udev/udev.conf -> main udev config file
/etc/udev/rules.d/* -> udev event matching rules
/lib/udev/devices/* -> static /dev content
/lib/udev/* -> helper programs called from udev rules
/dev/* -> replaced by dynamic udev and static content
in /lib/udev/devices/*
