One week before Halloween we took a short break from everyday programming routine and retreated into a cottage away from our Dresden headquarters. The idea was to do some hiking, socialize over a glass of beer, cook and dine together and explore L4Re in rather unusual ways. Since L4Re is written in modern C++ we couldn’t help but name our event L4Re::Treat<2018>. Our retreat took place in the Czech Republic, near the city of Liberec, just under the Ještěd mountain.
We asked our engineers to provide a short write-up of their impressions of the retreat and here is how they saw it.
The past Retreat was an animating experience for me. There was a relaxed working atmosphere and there were many interesting discussions. I have the impression that nobody was bothered by anyone despite the fact that we were working in the same room most of the time. We did a couple of walks as welcome changes and I will never forget the nice restaurant where we had lunch after climbing the top of Ještěd. I also enjoyed the restaurant with nice Czech beer at the last evening. Also I want to mention the nice meals we enjoyed together at the accommodation.
During the Retreat I was mainly hacking the built-in Fiasco kernel debugger (JDB). I fixed a couple of cosmetic issues which often bother me during my daily work, for example fixing the behavior of the “stack cursor” in the TCB view after following an address + coming back to the TCB view and a cosmetic problem when showing page table entries and a few more.
A bigger project was to re-merge the disassembler code from binutils to the kernel debugger and enabling the disassembler for ARM 32/64. Later (after the Retreat) I spent some more time in doing it right and using capstone instead of binutils with the benefit of the better license (MIT-like instead of GPL) and slightly less effort in adapting the code to JDB.
Btw, the binutils change was merged a few weeks ago but the (better) capstone replacement is still waiting for reviewer approval.
I used the retreat as an opportunity to learn the basics of using JDB to inspect the memory of a process, and also had discussions about virtual memory with Toni, Marius and Frank. To help building my understanding, I asked Marius if it would be possible for us to use JDB to change the message printed by the hello example while it was running. This example comes with L4Re and its code is shown below:
Marius proposed us to use the command readelf to inspect the binary of the hello program. By doing that, we managed to extract information about the ‘rodata’ (read only data) section of the program. The important thing about this section is that it tell us where the “Hello world” string will be located at runtime when the program is transfered to the virtual memory. Having taken note of this address, we then started an instance of L4Re and used JDB to pause the execution of this instance while it was running the hello program. This is as simple as pressing ‘Esc’ while your instance is running. If you try to reproduce our experiment, then make sure you dodn’t turn off the JDB module when you compile Fiasco. So, on JDB we then did a page walk through the memory of the hello task, and went to the exact location where the hello string was supposed to be located. No surprises there: it was where we expect it to be, and with some knowledge about the way that JDB works, and the architecture that we had compiled this example for, we were able to change the “Hello string” with the assistance of a ASCII table.
I also enjoyed very much the opportunity to get to know better some of my colleagues from Kernkonzept through social events. I joined them for hiking, and I also played snooker and table football with some of them. We went out for lunch and dinner, and it was the first time I tried some Czech dishes! I got to learn that some of my colleagues have a common interest in photography, and some are very skilled at cooking as well! I am very glad I had the opportunity to join this edition of the retreat! Perhaps during the next one we could also try to make our own beer!
We had a nice travel to the town below Ještěd. Everyone worked on their projects and food prep and even cleaning found enough volunteers. I enjoyed the mix of hacking and hiking giving plenty of opportunity to get to know the new colleagues and talk about the different projects.
I was hacking on virtualization of IncludeOS and have not expected to hack ACPI and BIOS code. I tried to implement it the ‘right way’ and created a new x86_64-l4re platform for the port, but this turned out to have issues with the linking stage and I ended up changing the original x86_64 platform to get it at least compiling.
On the uvmm side I looked into multiboot compliance, as this is the expected boot environment of IncludeOS. Both parts are unfinished and I haven’t had IncludeOS running on L4Re/uvmm.
I arrived at our first retreat with a couple of topics in mind. As Christian showed interest in working on the Raspberry Pi as our playground platform we decided to first enable the Pi 0 and Pi 0 W. This looked manageable on the first half a day because the Pi 0 [W] mainly consists of the hardware of the original Raspberry Pi B. The only difference is the UART used on the Pi 0 W which is the same as the UART on the Raspberry Pi 3. So, supporting that should be fairly easy as the code should already be in place.
But boy, were we wrong.
To make a long story short it took us roughly 1 1/2 day to get anything running even on the original Raspberry Pi B. The reason for that was that the gcc cross toolchain we used silently dropped support for ARMv6. Still it happily accepted the ‘-march=armv6’ option but the provided libgcc was ARMv7 only. This implementation contains instructions that may cause an illegal instruction exception on ARMv6. Now the hypothesis was that we were hit by such an exception well before the first character was written to the UART.
Because I tend to keep old toolchains around we finally succeeded with a gcc 4.8 cross toolchain. We now had the ‘hello’ example running on the original Raspberry Pi B. While at this point still an educated guess it almost proved our hypothesis from above. To finally verify it I used QEMU and took advantage of its advanced logging and debugging features namely the ‘in_asm’ and ‘int’ option to log target assembly code and exceptions. And there it was in the logs, an illegal instruction exception.
From there enabling the right UARTs on Pi 0 and Pi 0 W was an easy exercise. The remainder of the time then was just enough to resurrect some older GPIO and SPI examples to play around with some LEDs and LED matrix displays.
I am especially happy with a project Sarah worked on. She decided to pick up the Fiasco-self-virtualizing idea I proposed. The idea was to run Fiasco/L4Re in a virtual machine on Fiasco and uvmm. It turned out to be fairly straight forward work which raised interesting questions along the way. If you are on the JDB prompt how do you figure out which Fiasco instance entered the debugger? This kind of reminded me of the movie Inception from 2010.
Overall I really enjoyed L4Re::Treat<2018>. It provided a really relaxed environment. I would like to see it continued in 2019 because you know: Once it’s by chance, two times it’s a series and after the third time it’s a tradition.