Kernel News

Kernel News

Article from Issue 191/2016

Chronicler Zack Brown reports on the latest news, views, dilemmas, and developments within the Linux kernel community.

Zack's Kernel News

ISH Support

Srinivas Pandruvada from Intel announced a set of patches to better support integrated sensor hub (ISH) chips under Linux. ISH chips support motion detection and other sensors, more tightly integrated than earlier external sensor hardware. Srinivas's code didn't change the application binary interface (ABI) at all, so existing software should run fine and properly detect and use ISH chips without needing to be recompiled. But this initial release was mostly intended as a request for comments, rather than an actual submission for inclusion in the kernel tree.

Atri Bhattacharya, Daniel Turton, and Grant Likely all thanked Srinivas for sending these patches along. Atri reported that the patches correctly detected the ISH chip on his Lenovo Yoga 260, though it had trouble using the ISH sensors. In particular, suspend and wakeup became very slow, and he noticed a lot of time outs in the logs.

Daniel, also using a Yoga 260, reported that the kernel compiled OK with Srinivas's patches but wouldn't detect the ISH chip. He posted some debugging information for Srinivas.

Grant, testing on an HP Spectre x360 G2, successfully built the kernel, which successfully detected the ISH chip, and successfully created control files under /sys/bus/iio/devices/, but he saw no obvious way to test the code.

To Grant, Srinivas offered a few suggestions: run cat on the files; modify them to control screen rotation and brightness. Grant tried some of these things and reported back that the system did detect the ISH chip accelerometers, but the sensor events didn't seem to be reaching the kernel.

Srinivas offered suggestions to fix that issue, as well as Atri's issue with suspend and wakeup. There was a bunch more debugging, also joined by Bastien Nocera. No solutions were found during the course of discussion.

I really enjoy seeing companies like Intel submitting code early and concerning themselves more with collaborating with the kernel developers than presenting a perfectly working set of patches. It took a long time for some of those big industry players to get used to a public development process, in an era where marketing and legal departments are terrified of anything going even slightly wrong.

Per-Task Livepatching

Josh Poimboeuf posted some patches against livepatch – the kernel feature that lets users update a running kernel without having to reboot. Specifically, Josh had implemented per-task updates, where the kernel would migrate running processes over to using a given patch one by one, as it deemed each task ready to make the switch. This is good for patches that alter the data that's available to a process, or that in some way change the internal kernel structures that need to be referenced.

Josh's code used several techniques to determine when a process was ready to receive a patch. Presumably these techniques will be the things that future generations of kernel hackers will debate and improve. But Josh also included a feature to apply patches immediately, if they were known to change no semantics or data.

Petr Mladek had some technique comments but overall averred, "It is really great work! I am checking this patch from left, right, top, and even bottom, and all seems to work well together."

Miroslav Benes and Jessica Yu also offered some technical criticisms. Jessica asked, "What about tasks sleeping on affected functions in uninterruptible sleep (possibly indefinitely)? Since all signals are ignored, we wouldn't be able to patch those tasks in this way, right? Would that be an unsupported case?"

Jiri Kosina agreed that this could be an issue but didn't see any solution beyond documenting it as a possibility, or perhaps trying to detect the situation and alert the user. And, Josh said that Jessica's case would probably be a very rare occurrence and "not something we need to worry about for now."

Related to Jiri's idea of a user alert, David Laight suggested, "please can we have a flag for the sleep and/or process so that an uninterruptible sleep doesn't trigger the 'hung task' detector and also stops the process counting towards the 'load average'. In particular some kernel threads are not signalable, and do not want to be woken by signals (they exit on a specific request)."

They continued to discuss the technical details briefly. Livepatching is one of those weird, incredible things that would have been deemed impossible 20 years ago, like Linus's famous pronouncement that Linux would probably only ever support the 386 processor. Somehow, these big, unlikely features are allowed to come gradually into the kernel, as long as they don't spread their insanity through the rest of the code.

TSN Support in the Works

Henrik Austad had been working on a Time Sensitive Network (TSN) driver for a while and finally decided to send it in for a first review. TSN is mostly used for streaming data across a network, either media or communications.

Henrik's work was met with a serious rebuke from Richard Cochran, who said:

"Your series does not even begin to address the real issues. I did not review the patches too carefully (because the important stuff is missing), but surely configfs is the wrong interface for this. In the end, we will be able to support TSN using the existing networking and audio interfaces, adding appropriate extensions.

Your patch features a buffer shared by networking and audio. This isn't strictly necessary for TSN, and it may be harmful. The Listeners are supposed to calculate the delay from frame reception to the DA conversion. They can easily include the time needed for a userspace program to parse the frames, copy (and combine/convert) the data, and restart the audio transfer. A flexible TSN implementation will leave all of the format and encoding task to the userland. After all, TSN will some include more that just AV data, as you know.

Let's take a look at the big picture. One aspect of TSN is already fully supported, namely the gPTP. Using the linuxptp user stack and a modern kernel, you have a complete 802.1AS-2011 solution.

Here is what is missing to support audio TSN:

User Space

1. A proper userland stack for AVDECC, MAAP, FQTSS, and so on. The OpenAVB project does not offer much beyond simple examples.

2. A userspace audio application that puts it all together, making use of the services in #1, the linuxptp gPTP service, the ALSA services, and the network connections. This program will have all the knowledge about packet formats, AV encodings, and the local HW capabilities. This program cannot yet be written, as we still need some kernel work in the audio and networking subsystems.

Kernel Space

1. Providing frames with a future transmit time. For normal sockets, this can be in the CMESG data. For mmap'ed buffers, we will need a new format. (I think Arnd is working on a new layout.)

2. Time-based qdisc for transmitted frames. For MACs that support this (like the i210), we only have to place the frame into the correct queue. For normal HW, we want to be able to reserve a time window in which non-TSN frames are blocked. This is some work, but in the end it should be a generic solution that not only works "perfectly" with TSN HW but also provides best effort service using any NIC.

3. ALSA support for tunable AD/DA clocks. The rate of the Listener's DA clock must match that of the Talker and the other Listeners. Either you adjust it in HW using a VCO or similar, or you do adaptive sample rate conversion in the application. (And, that is another reason for not having a shared kernel buffer.) For the Talker, either you adjust the AD clock to match the PTP time, or you measure the frequency offset.

4. ALSA support for time triggered playback. The patch series completely ignore the critical issue of media clock recovery. The Listener must buffer the stream in order to play it exactly at a specified time. It cannot simply send the stream ASAP to the audio HW, because some other Listener might need longer. AFAICT, there is nothing in ALSA that allows you to say, sample X should be played at time Y.

These are some ideas about implementing TSN. Maybe some of it is wrong (especially about ALSA), but we definitely need a proper design to get the kernel parts right. There is plenty of work to do, but we really don't need some hacky, in-kernel buffer with hard coded audio formats."

Arnd Bergmann replied to Richard's kernel item #1, regarding transit timings. Arnd said, "the timestamps in the current v3 format are sufficient until 2106 as long as we treat them as 'unsigned', so we don't need the new format for y2038, but if we get a new format, that should definitely use 64-bit timestamps."

Henrik also replied to Richard, saying that his patches were intended to get the discussion going, not to be a real solution to the various problems. So, he tried to address each of Richard's objections in turn. Takashi Sakamoto also joined in with his own technical critique of Henrik's patches. The discussion delved into specific standards documents and kernel behaviors.

Overall, it seems as though Henrik's patches got some of the right people talking; though it's clear that in terms of actual code and feature support, TSN remains in a very early stage of development – essentially nowhere. That is exactly what I found so cool about this discussion. It helped clarify everyone's understanding of what a proper set of features would look like, and who are likely to be the key reviewers of future patches.

Peter Zijlstra wanted to address an issue with spinlocks that had bitten a bunch of users. Specifically, he'd noticed cases where code waiting for a locked resource might check the lock earlier than expected, and miss out on crucial state changes. It was a highly internal-to-the-code issue, and Linus Torvalds pointed out the problems with conceiving of it that way:


We don't start adding more of this 'after_ctrl_dep' crap.

It's completely impossible to understand, and even people who have been locking experts have gotten it wrong.

So it is *completely* unacceptable to have it in drivers.

This needs to be either hidden inside the basic spinlock functions, _or_ it needs to be a clear and unambiguous interface. Anything that starts talking about control dependencies is not it.

Note that this really is about naming and use, not about implementation. So something like 'spin_sync_after_unlock_wait()' is acceptable, even if the actual _implementation_ were to be exactly the same as the 'after_ctrl_dep()' crap.

The difference is that one talks about incomprehensible implementation details that nobody outside of the person who *implemented* the spinlock code is supposed to understand (and, seriously, I have my doubts even the spinlock implementer understands it, judging by the last time this happened), and the other is a much simpler semantic guarantee.

So, don't talk about 'acquire'. And most certainly don't talk about 'control dependencies'. Not if we end up having things like *drivers* using this like in this example libata."

At this point, Tejun Heo came in, saying that Peter's issue could possibly be resolved by simply removing the offending code from the kernel, as it wasn't needed anymore. Peter replied, "that would be great; I was sorta lost in there, but it looked like if you need the spin_unlock_wait() you also need the extra barrier thing. If you can remove it, better still."

Meanwhile, Peter said he'd also be fine shoving his fix into the main spinlock implementation as per Linus's recommendation.

The discussion petered out around there. But, it's interesting because the kernel's locking features have gone through such hellish transformations over time, in particular with the transition from the Big Kernel Lock (BKL) to the various smaller locks now sprinkled throughout the kernel. Typically, kernel locks make the code more and more complicated even as they allow support for smoother and smoother multitasking. There's a strong pressure to reduce locking complexity and to make locks as intuitive as possible. Hence, Linus's uncompromising use of the hammer.

Buy this article as PDF

Express-Checkout as PDF
Price $2.95
(incl. VAT)

Buy Linux Magazine

Get it on Google Play

US / Canada

Get it on Google Play

UK / Australia

Related content

  • Reiserfs Experiencing Turbulent Updates

    When Jeff Mahoney sent in a bunch of patches for reiserfs, he assumed that the filesystem would be frozen in maintenance mode from that point on. Things turned out differently.

  • Kernel News

    Chronicler Zack Brown reports on the latest news, views, dilemmas, and developments within the Linux kernel community.

  • Kernel News

    Chronicler Zack Brown reports on the latest news, views, dilemmas, and developments within the Linux kernel community.

  • Kernel News

    Chronicler Zack Brown reports on the latest news, views, dilemmas, and developments within the Linux kernel community.

  • Kernel News

    Chronicler Zack Brown reports on the latest news, views, dilemmas, and developments within the Linux kernel community.

comments powered by Disqus
Subscribe to our Linux Newsletters
Find Linux and Open Source Jobs
Subscribe to our ADMIN Newsletters

Support Our Work

Linux Magazine content is made possible with support from readers like you. Please consider contributing when you’ve found an article to be beneficial.

Learn More