Linux and hardware haven't always been the best of friends. Just 20 years ago, you couldn't just buy a mainstream computer and expect it to work when you installed Linux. If wanted to be sure that your new acquisition would play nicely with Linux, you often had weeks of very detailed research ahead of you. Which chipsets were supported? Which modems, printers, sound cards, and network adapters had the necessary drivers?

If you didn't check carefully, the worst case is that you would be left with hardware that was practically impossible to get to work with Linux. Just remember the notorious winmodems (Figure 1) or printers that did nothing without binary drivers that the vendor didn't bother to provide for Linux systems. If you are looking for proof that not everything was better in the past, the Linux hardware saga is a rich source.

Figure 1: Softmodems or winmodems were a nuisance in the early years of Linux development because they didn't work without proprietary drivers. Over 20 years later, Intel has done the same stunt with webcams. Wikipedia, Douglas Whitaker, CC BY-SA 2.5

Fortunately, the times are changing. Individual computer components are increasingly merging into a single logical unit. Sound cards became standard components at some point and supported use on Linux with appropriate drivers. Graphics chipsets, also an eternal nuisance, now come with support from NVIDIA or AMD. If you don't play games, you can make do with the chipsets integrated in AMD and Intel CPUs, which also perform reliably on Linux. Hardly anyone needs modems anymore, but LTE or 5G modems can usually be fed at commands via an emulated serial interface. Printers usually have their own network connection and offer support for KPL/PD or PostScript. Even for devices connected locally via USB, wide ranging driver support exists today. All is well, you might think, in the world of Linux and modern hardware.

In 2023, it seems strange that Linux users are again scrambling to compile kernel modules, battling with broken drivers on their desktops, and using elaborate work-arounds. This epidemic has been triggered by hardware that hardly anyone suspected would have come up lacking in support: webcams.

The Unknown Giant

The recent turn of events in webcams has its origins in something that took place more than 20 years ago: the founding of the Mobile Industry Processor Interface Alliance, or MIPI for short. MIPI was established in 2003 with the aim of creating protocol and interface standards for the interaction of processors and peripheral devices.

Most admins assume the term peripherals refers to mice, keyboards, printers, scanners, and similar devices, but the common definition also includes monitors and attachments such as external sound cards or webcams. For quite a while, MIPI didn't get much publicity or media coverage. But today, the consortium includes more than 300 manufacturers, so MIPI has definitely become a force in the industry.

One of MIPI's main fields of activity is defining protocols for the exchange of images and sounds between a host and the peripheral device. The CSI protocols are some of the most widely used protocols in the MIPI Alliance. CSI stands for Camera Serial Interface, and three versions of it are currently available. The most popular is undoubtedly 2019, the latest revision of MIPI CSI-2. CSI-2 is still used in many MIPI-compatible cameras today.

But how does a "normal" webcam differ from a MIPI device? In simple terms, CSI simply defines the interface that the camera and host use to exchange data. Like all interface definitions, CSI has had to adapt to the fact that data volumes have grown over the years. Today's webcams often need to handle 4K streams. MIPI CSI-2 from 2019 already supports significantly higher bandwidths than legacy USB 3.0 with its realistic 3.6 GB/s throughput. But the usable bandwidth is not the only difference between typical old-style webcams and the MIPI system.

The MIPI standard explicitly stipulates that the camera sensors and the components responsible for image evaluation must not be combined in the same assembly. This means that, in MIPI setups, the camera uses its sensors to produce raw audio and image data, which it then passes through to the host. The host handles the brunt of the work, computing an image with sound, which can then be used for video conferencing or other activities.

Several factors play an important role. On the one hand, decoupling the sensor from the chip enables a variety of combinations that can be adapted to the application. Inexpensive notebooks contain equally inexpensive video processing chips, mid-range devices use inexpensive chips with expensive sensors, and high-end machines use the best they can get – or so the theory goes (Figure 2).

Figure 2: MIPI CSI-2 describes an interface for communication between peripheral devices and host chips so that they can be logically separated from each other and combined as required. © NXP Semiconductors

On the other hand, this setup means you can integrate special chips that not only assemble an image for the webcam but also process the data, for example, with AI support. The famous AI problem that automatically calculates the number of bees in a hive using a webcam benefits massively from the MIPI interface: The data arrives in precisely the raw format that the computer needs to process directly, instead of fiddling around with a mess of pixels from a mid-range webcam.

The Crux of the Matter

Thus far, the gradual migration to MIPI technology seems very promising, but appearances are deceiving. The reason for the problems Linux users are having with their webcams is related to the image processing chips that comply with the MIPI specification. The CSI-2 protocol itself causes virtually no problems. The Linux kernel has supported it for quite some time and has no trouble supporting a data exchange between peripherals and the MIPI chip (Figure 3).

Figure 3: The Linux kernel is quite capable of handling the CSI-2 specification and also implements a transport path for it. However, it has trouble with the Intel drivers. Users are therefore forced to start tinkering with their systems.

The situation is not quite so harmonious with the image processing chips, which have been making inroads into the current Intel and AMD CPU specifications for some time now. The IPU6 image processing unit included with Intel's "Tiger Lake" and "Alder Lake" CPU series specifications is explicitly intended as a potential CPU component for a MIPI-compatible camera based on the CSI-2 standard. The crux of the matter is that, to use the IPU6 chip, you need proprietary firmware and a special driver.

Firmware is unlikely to faze most Linux users; after all, there are quite a few drivers that you can only use with proprietary firmware. The kernel developers and the major distributors have long since found ways to cushion this problem from the user's point of view in the form of sensible workarounds. But things are different with IPU6.

Although Intel provides a driver for the cameras, this driver does not work well. For the current Linux kernel, there is no trouble-free way to build it. To make things worse, the way it is written means that it cannot be integrated into the official Linux kernel in the foreseeable future and must therefore be considered an out-of-tree development. This prompted kernel veteran Greg Kroah-Hartman to explicitly warn users against buying new devices with Intel's "Alder Lake" CPUs and MIPI webcams [1]. Usable support under Linux is unlikely in the foreseeable future, he emphasized.

If you take a closer look at the situation on Lenovo's current 10th-generation flagship X1 Carbon, the extent of the disaster quickly becomes clear. After booting a recent desktop Linux, such as openSUSE or Fedora, the device's built-in webcam is simply not available. The webcam appears in the list of integrated hardware, but no driver is loaded. A usable webcam is also conspicuously absent in tools such as Google Meet. If you have a lemon like this on your desk, you can only take part in video conferences without sending video images – which is farcical given that we are living in the third decade of the 21st century.

To make things even worse, without resorting to a compiler, this debacle is practically impossible to clean up. One of the Linux community's greatest achievements over the past two decades has been to make messing around with drivers on consumer hardware unnecessary. When I first started exploring Linux in 1998, it was nothing out of the ordinary to build a custom kernel because of a special driver or even a single important patch. However, distribution kernels have steadily become more reliable and complete in recent years. Kernel building skills are likely to be rusty even among old hands.

Ways Out

All told, the sparsely documented approaches to getting an IPU6 chip to work on Linux are not very encouraging. Users who are brave enough to tackle this problem have a real marathon ahead of them if they ever want to use the webcam.

Martin Sofaru, who owns a seventh-generation Lenovo X1 Yoga, took on the task of solving the IPU6 problem out of personal interest. He describes the process that ultimately led to success in several bug reports on GitHub.

Everything starts with the Linux kernel. The kernel is not always able to read the correct CSI pin assignment from the system's ACPI information and requires a patch [2] by Red Hat employee Hans de Goede (Figure 4). You then need to add another code patch [3] for current kernels to ensure that Linux uses the IPU6 chip's IOMMU interface instead of unsuccessfully placing it under the aegis of the CPU's IOMMU management. Finally, a third patch [4] ensures that the Linux kernel detects ACPI device LATT2021, which is installed in some laptops with an IPU6 chip and without which the camera cannot be controlled.

Figure 4: If your own distributor has not already integrated the patches needed for IPU6, the first step on the way to using your webcam is to patch the kernel.

First, pick up the source code for your current kernel and check whether it already contains the three code additions, or if you need to apply them. Canonical has now included at least the first two patches in its own kernel for Ubuntu 22.10. This was not the case for openSUSE and Fedora, at least at the time this article went to press. If in doubt, your only option is to pick up the source code for the kernel you are currently running and compare the code patches item by item.

Unfortunately, after customizing the kernel, you have only completed about 10 percent of the work. Once the kernel is up and running, the next step is to compile the required drivers and install the proprietary firmware. Intel delivers the firmware in its own Git directory [5], and an installation guide tells you what to do: Copy the files from the include/ and /lib folders to /usr/include/ and /usr/lib/.

This illustrates another dimension of the mess. The proprietary components for IPU6 include not only the firmware itself, but also a bunch of userland libraries for accessing IPU6. Ubuntu and Arch Linux offer corresponding packages, but Ubuntu's packages are provided externally by the OEM Solutions Group [6]. openSUSE and Fedora, on the other hand, require far more manual work. You need to retrieve the files directly from GitHub and unpack them on your system.

The story is similar for the driver packages. Ubuntu and Arch Linux have packages available for building the modules for your own kernel with just a few commands. Ubuntu unsurprisingly uses DKMS (Dynamic Kernel Management Support), which offers multiple benefits. Complementing the module sources themselves, the OEM Solutions Group has also included some patches from the Intel module code bug reports, without which the kernel module simply won't work.

Intel's official GitHub repository is obviously not a hive of activity and the company doesn't seem to want to make it easy to use its hardware on Linux. Once again, Fedora and openSUSE users will have more work to do than users of some other distributions.