Texts

My brother’s award-winning book.

Making PDFs readable for everyone.

Co-op principles for everyone.

Princípios para qualquer organização.

Principles for any organization.

Beyond the browser.

Chromium and friends in Seoul.

Wildfires and Igalia’s 20th.

Skateboards to VR.

Wedding, baby, and grandma’s birthday.

A beach.

It won’t run Rift apps.

About a year ago Ozone platform abstraction layer started to take its shape in Chromium and we were excited how easily we could support Wayland platform using it. Everything went great and we helped a few Chromium targets like Crosswalk to debut in the Tizen ecosystem for example. More recently, different Ozone platforms have been implemented and only now it’s becoming evident the significance of Ozone for Chromium Linux, generally speaking. In this post I will focus in the Ozone-GBM platform, which is the platform that gives support for hardware-accelerated graphics and input support in Linux, without relying on any traditional windowing system like X11, Wayland etc. I hope that after reading this document besides a better knowledge on Ozone-GBM itself, the reader will also have an overall idea how Linux graphics and Chromium architecture function together. LINUX, OPENGL, EGL, GBM AND MESA # When building modern and beautiful UIs, the GPU is the hardware engine responsible to produce compelling graphics representation of their images and output them to the screen. On the software side, OpenGL is the well-established API for rendering graphics using a GPU and Mesa is the Linux open-source library implementing it. In practice, OpenGL (and its subsets OpenGL ES 1, OpenGL ES 2, etc) usually comes accompanied with a few other APIs to handle platform and system details, one of them being EGL. EGL is the generic interface for dealing with different windowing systems operations like surfaces binding, rendering synchronization, among others. Besides EGL, there’s also the KMS/DRM kernel subsystem together with its userspace library used for performing display mode setting and frame-buffer related operations. But when we are mixing together these three major components – GL, EGL and KMS/DRM – there is a missing component for performing memory allocation. Previously this was solved by hardware specific drivers on X window system (X11), but if we wanted the user application directly allocate memory then it would be a real problem, which would be solved only later by Mesa GBM. Mesa GBM (Generic Buffer Manager) basically provides a EGL native window type (just like Wayland and X11), so one could obtain a real EGL surface and create render target buffers. With that then, GL can be used to render into these buffers, which will be shown to the display by queuing a page flip via KMS/DRM API. It’s important to note that the only public implementation of GBM is the one based on KMS/DRM, therefore although it’s a “generic” library for accelerated buffers so far we have seen only one out there (!). Besides, by now you may have noted that Ozone-GBM is probably not the most suggestive name about what is in fact this platform. CHROMIUM UI FRAMEWORK, AURA AND VIEWS # Jumping now to Chromium, in the graphics side of the things we have Aura/Views UI framework. Aura is the acceleration graphics system in Chromium. Views is the widget toolkit based on Aura, so one could build buttons, dialogs etc easily with Views but not with Aura. Aura has a very simple concept of window which receives messages for input event handling, painting, and other low level tasks. On the other hand, Views widgets (views::NativeWidget) are responsible for translating platform-specific notifications into cross platform events. For example in Chrome Browser, what we see in our desktop screen including the Web contents (“renderer”), main tool bar, window frame, search bar (“omnibox”) and etc are constructed basically from a bunch of Views, each backed up by a Aura window, which in turn are hosted within one single Aura root window (aura::WindowTreeHost, one per display actually). aura::WindowTreeHost bridges between Aura windows and the native root window, i.e. where the real platform-specific code lives. Depending on the platform there will be a root window that wraps one X11 window (aura::WindowTreeHostX11) or one that wraps an entire and robust graphics system (ui::DriWindow), in case of Ozone-GBM. Before we move on, it’s important to explain a few concepts used in Chromium. Compositing is the phase of rendering that combines layer’s textures into a final screen image. Drawing is the compositor combining layers into the final screen image; while painting is the population of layers’ backings (bitmaps with software rasterization; textures in hardware rasterization). That being said, the root window inherits the compositor object (ui::Compositor), so whenever a draw is triggered in the root window aura::WindowTreeHost, Chromium compositor walks through the child windows, painting the invalidated regions of them and making sure the final result will be presented on the screen (ScheduleDraw()). FULLSCREEN VS. DESKTOP TARGETS # There’s a variety of targets that Chromium lets us build such as Chrome Browser, ChromeOS, Chromecast, Crosswalk runtime and many others. We basically can split these targets in two classes, depending how the user-interface behaves on each target: the first class, where a Chromium target controls the entire display, e.g. ChromeOS or Chromecast second class, the ones that are only part of a multi-window desktop UI, e.g. Chrome Browser in a desktop system In the second class of targets, aura::WindowTreeHost (more recently actually split into ui::PlatformWindow) needs to implement all the interactions with the windowing system (resize, maximize, close, etc) and controls the dispatch of the native input events to high-level delegated Chromium interfaces. This is rather complex mechanism and although it’s difficult to get it correctly this type of support was the first that existed in Linux Aura infrastructure (“Aura-X11”). Interestingly, this same support was being used so far for the first class of targets. Note though that the underlying windowing system for these fullscreen targets could be very simple, because the platform window corresponds practically to a physical display (“why maximize or resize a root window? It doesn’t make sense”). In other words, fullscreen Chromium targets require minimal interaction with the windowing system and the few other tasks that were being carried through it, like input devices, display management or media support, one would argue that are not tasks inherently of the window system. OZONE-GBM # Finally, and now speaking with authority, we can define it: Ozone-GBM is the native fullscreen-only platform that delegates the composition tasks of Aura root window to a new Chromium platform window based on KMS/DRM, ui::DriWindow. In ui::DriWindow, EGL is used for drawing and OpenGL ES 2.0 for rendering. Also, display synchronization (“page flip”), display configurations are handled by this platform. A smart logic to use hardware display planes (“overlay”) is also implemented in there. Besides, Ozone-GBM has an internal implementation of the Linux evdev subsystem (ui::EventFactoryEvdev), so events emitted by input devices can be processed and handled accordingly to the needs of Chromium targets – touchscreen devices, touchpads, keyboards and mice are somewhat supported by Ozone’s evdev driver. SUMMARY # Ozone is a platform abstraction layer beneath the Aura window system that is used for low level input and graphics. In Linux graphics, Mesa GBM provides the ability for user application directly allocate accelerated graphics buffers. Fullscreen Chromium targets such as ChromeOS, Chromecast among others require minimal interaction with the native windowing system. Ozone-GBM is the native fullscreen-only platform that delegates the composition tasks of the root window to a new platform window based on KMS/DRM. It uses EGL/GLES2 for accelerated rendering. Besides, Ozone-GBM has an internal implementation of the Linux evdev subsystem. Fullscreen Chromium targets can use Ozone-GBM to remove unnecessary dependency on full-fledged window systems like X11, Wayland, DirectFB, etc. Out of the scope of this article but worth to mention, is Ozone-DRI implementation. It uses KMS/DRM infrastructure as well but scanout content is displayed via software rendering, using double-buffered Skia surfaces. This implementation could be particularly useful for budget devices, for example those that don’t have GPU at all. (Besides, Ozone-DRI also has a quite bad nomenclature because “DRI” refers to X11 direct rendering, even though X11 is not used at all there).

Some state-of-the-art progress of the project we’re proudly working with: https://www.youtube.com/watch?v=EJB2pznc6iY

A few weeks ago we released Ozone-Wayland and now we’d like to detail for you the development process and strategy behind it… ah, and the title is not developing Chromium, the browser; it’s developing Chromium,** the project**! You will understand why next.

The following message was sent out this morning – I’m copying it here and attaching a cute screenshot of my desktop :) Ozone is a set of C++ classes in Chromium for abstracting different window systems on Linux. It provides abstraction for the construction of accelerated surfaces underlying Aura UI framework, input devices assignment and event handling.

Let’s forget for a second about video drivers, whether it has acceleration or not, and all the related issues with hardware support on Wayland. This is all solved. Let’s talk about the user interface (UI) and ways to customize it all over the computing continuum – from phones, tablets and TV box to desktop PCs, Invehicle Infotainment (IVI), aeroplane systems, among others.

Wayland 1.0 release is knocking the door and people keep asking “why Wayland if we got X already”, or things like performance, memory consumption, power savings and other kind of advantages on having Wayland instead X. Those are very important points to consider, of course, but for one individual actually programming the graphics system the answer should be straightforward: Wayland API is damn small.

A rather cool feature on Weston compositor is xwayland, to support X11 native applications on Wayland. It’s a quite important feature because gives the compatibility with the “old” windowing system, so say you have an application written on Motif/Xt or even something more “fancy” like a Web browser all tied with GTK2 and whatever dependency, then you better not bother yourself re-writing it to native Wayland or porting to a modern toolkit – it should just work seamlessly on it. Hence, X on Wayland fits pretty well with our overall transition plan.

Do you wanna contribute to a funky open-source project? Are you tired of your nerdy and boring community of developers? Are you the one that wants to get rid of X because it’s a giant, old and fat dinosaur in your system? :) Cool, I have a project to solve your problems!

To run Qt applications on Wayland is fairly simple nowadays. Thank to Qt developers, they are following up quite well our last changes on Wayland protocol and updating accordingly on Qt5 code base – by the way, the fresh and just released Qt 4.8 does not ship the latest protocol additions, so that’s not the one I’m referring.

Hello Intel Finland :) Here I am now; four months already at OTC big team.

Actually this is not a big news but it’s nice to see on practice X being used on the advanced age :) Here I’ve played with some UI configuration settings, browsed for while, toyed with facebook, opened some photos in the viewer and etc, all for about 5 minutes:

Following is the census of 1.10 window for all X infrastructure - raw numbers here. I did it in a similar way as the previous version. Worth to mention that there’s almost no relation between the cycles of development from each of the components listed below, which can lead to some misunderstanding. Anyway, still a nice indicative to see and evaluate how the free desktop community behaved.

I am a candidate for the elections 2011 of the board of directors :) Following are my answers based on the questionnaire formulated by the members. You can check other candidates answers here.

Moikka. If you follow here [0] you will see a set of Debian packages that Graphics team at Nokia are continuously working to deploy a X11 implementation for MeeGo-Harmattan, where we target embedded systems only. Feel free to use it!

Topic’s name is a funny (and friendly) devotion to GNOME Census. So let’s take a look at some numbers from the time Xorg 1.9 was in development - raw data is here. Would be unfair to measure only the work that happened e.g. in X server or in the drivers being developed and come up with the statistics about “who developed X”. X and X development community are quite extensive and don’t concern only “graphics” related, i.e., pixel information that appears on your display screen. This is a very common mistake.

That’s what I’m using for MeeGo now. Autoconf parameters, theeere we go: –disable-static –disable-aiglx –disable-config-dbus –disable-config-hal –disable-dbe –disable-dga –disable-dpms –disable-dri –disable-glx –disable-glx-tls –disable-int10-module –disable-ipv6 –disable-screensaver –disable-secure-rpc –disable-tcp-transport –disable-vbe –disable-vgahw –disable-xdm-auth-1 –disable-xinerama –disable-xwin –disable-xaa –disable-xace –disable-xdmcp –disable-xf86vidmode –disable-xfree86-utils –disable-xnest –disable-xvmc –disable-libdrm –enable-config-udev –enable-dri2 –enable-null-root-cursor –enable-record –enable-unit-tests –enable-visibility –enable-xorg –with-sha1=libsha1

Last week I’ve been in Brazil at 11th International Free Software Forum (FISL) talking about Linux Graphics for Small Devices*. I tried to cover a bit of everything that I learned in the world I’ve been immersed in some near past - I guess there aren’t many news for freedesktopers though. Anyway, everyone is very welcome to give any kind of feedback and comment on it. Just follow here.

Usually faster the number of commits, faster the development process. But development without strict reviewing doesn’t guarantee quality on the code. Unfortunately, rarely you will see people willing or even just being able to review the changes you are proposing. Attract reviewers is hard!

Previously, the message was for toolkit, now it targets new upcoming developers… okay, if I’d be offensive I could say it targets vendor distributions which care for desktop on Linux :)

We all know since 2006 (?) that the world is turning to use XCB instead Xlib as the interface between X clients and the server. The advantages of using XCB is well know but, even so, it’s being adopted very slowly for some reason - for instance no toolkits yet ported it. If this serves as a motivation, Alan got a really nice measurements porting a simple X app.

So here goes some statistics of the Xorg process running. All the informations were fetch from /proc/pidof Xorg/{smaps, status}. I used also a script found on the Web to parse and organize these informations; Mikhail Gusarov has extended this script to show a very useful output.

This series of documents explore how the memory is used by the Xorg server. They aim to eventually shrinks the memory footprint of the server and its related components, like X clients, modules being loaded and drivers. Embedded devices with constrained resources are the main focus here. All texts are mostly based on x86 and ARM architectures, under Linux 2.6.33 with Xorg from upstream.

For the first time in life, Xorg is being used in a single platform and for a given device only (other devices have used an X11 implementation but using other non-canonical servers, such kdrive’s based - Tiny-X).

I like UI and I’m not gay. Some time ago someone posted a comment, complaining about the white fonts over the dark background in my blog, which makes the read be difficult. The guy teased me, recommending an ironic page where shows how hard is to read texts like this. He was right…

Just a thought is this something that should be deleted? This seems to be using about 7% of my RAM I don’t have words for it.

Portland, OR I was so proud for the forth time be part of the X crew, this time in Portland. Really amazing the city. I had the opportunity to play tourist and visit some nice places.

So last week I posted on lkml an old patch that we were carrying for a long time in the Linux community. It basically brings the multiple (old) video cards functionally again on Linux and X server (and this time doing on the right and beauty way). For the people that was following multiseat implementations, this is a HUGE step: we will finally be able to discard the old and ugly hack (a mix of Xorg, several Xephyr servers + evdev) and and go to a clean way, starting multiple X servers in parallel. Cool! Well, not that much, because it might take some time to be in your beloved distribution :)

… here I’am :)

Being quick about my last months: finished my Master course, moving to Helsinki tomorrow (!), got my first real job. Yessh, three great steps in my crazy life and I’m very happy with it! Hope to stay closer with the desktop communities again ;)

One thing that I learned with open communities is when you send a proposal idea and no one replies. This seems to be odd at first but it isn’t. If you did something strange or wrong then at least someone will reply. OTOH if you did something that can be promiser and no one objects, then great! 99% that you did something interesting :)

Trying to put some order with all the multiseat documentation found on Web, today I updated the following: Multiseat’s article, in X.Org Foundation wiki: http://wiki.x.org/wiki/Development/Documentation/Multiseat Multiseat’s history, in Wikipedia: http://en.wikipedia.org/wiki/Multiseat#History So if you are trying to setup or help in development, probably the foundation’s wiki has the most up to date and centralized references. Also, please, if you have more helpful information, don’t be afraid to update the wikis. Our future grandsons will be thankful :)

This week our laboratory at university released the MDM utility to ease the process of installation and configuration of a multiseat box. The idea is that the end-user should not use some boring and hard howtos anymore to deploy it. Just installing a distro package must be enough now. Try it, use it, report the bugs and send the patches! :)

Unfortunately that model which I described some weeks ago to put the input event delivery of the X server in a separate thread wouldn’t be an advantage. I precipitated myself thinking that it could be feasible. Sorry :(

Input events routed through another thread/process can have bad effects on latency because we can’t guarantee that it will get scheduled at the right moment. Although this is hard to see happening with the current X server threaded implementation, we must design something to avoid it. One way to improve the responsiveness is to give a high priority to the input thread and also adjust the CPU scheduling. (Note that this will not avoid problems related with page faults which usually happen in the X input flow.)

Given that GSoC ‘08 is getting close to the end, strategy number 2 showed more feasible to proceed my work. Strategy #3 would be a lot of fun but would imply a hell massive codification as well (also a little out of our scope). Unfortunately no-no for now.

GSoC summary #1 - July 29 The current implementation of X Window System relies in a signal scheme to manage the input event coming from hardware devices. This scheme frequently get blocked when lot of IO is occurring (for instance, when the process is swapping in/out). Get blocked means for instance a jumping cursor on the screen and in GUI is always desirable to prioritize the system responsiveness for end users. The human/computer interface should be smooth and this is the most user visible aspect of a system.

The current DRM kernel modesetting tree is already taking care to update the cursor registers and paint it to the screen. Very cool [0]. What I’ve done today is a shortcut between the kernel input layer and DRM to update the cursor directly on screen without the X server be notified always. Of course, a lot of issues raised up together. So let’s try to delegates the tasks again.

I really don’t like to read large blog posts. Anyway… What I did so far is a separated thread that takes care only the injection stage on the X server queue. Who is interested with the results, please read some past posts in my blog. It is currently in a very good shape (synced with post-mpx merge, all input devices are inside the thread and etc). The implementation looks like this: thread #1 deals with - injection of input events from devices thread #2 deals with - processing of input events to clients - requests from known clients (rendering things) - new client that tries to connect (pretty easy to do)

For my SoC project I need some mechanism to evaluate the improvement of the input thread inside X. So I wrote a simple kernel driver that emulates the mouse device moving and emitting bits of a simple pattern. I don’t know if something like this already exists or if there are other ways to do it, but the fact is that the solution I thought took me only few hours between the moment that I imagined, collected some ideas on the Web and implemented it.

I’m very happy to say that I was selected again to work on Google Summer of Code with X.Org Foundation. Daniel will be my mentor again. Thanks Google. Thanks X.Org! In the last year we did a nice work separating the input event generation code of the X server into a different thread. We saw some performance improvement there specially because the implementation is not using signals anymore to wake up the server when some device emits an event. The reason why is that when a process is in the uninterruptible sleep (D state) signals are delayed and the mouse cursor lags.

I was thinking how we could make remotely X clients totally connective with the server when both are behind a NAT/firewall. We can imagine one big motivation to do this: a scenario where someone using his thin and poor machine wants to use the resources of some “fat” machines which he simply doesn’t know where they are seated. Those fat machines could be arranged through a P2P network of “X11 pool of resources” and the list of machines displayed to the user select his desired one (e.g. with minor lag/load). Someone more capitalist than me could go further and imagine a provider selling X11 resources to mobile devices. Or just open your home machine’s web browser in any place of the world. Well, the field of applications would be huge.

Today, Paulo Zanoni help me to put in a shape the VgaArbiter wiki page. The primary intention of that page is to bring more developers in that project and some users++ who could also help testing. Feedback is very welcome. Here is the link.

After a long journey I come back in this… So I did a set of benchmarks to evaluate the VGA arbitration versus the RAC usage. My goal is to evaluate the performance difference of a multi-head/multi-card environment, i.e., an Xorg using the RAC to another using the arbitration.

The Resource Access Control inside Xorg is the guy responsible to take care of the various resources of memory and to share them in a wise manner when two or more graphics devices are active (think multi-head). As an alternative from RAC we can rely on VGA arbiter. So me and Paulo Zanoni spent the last days implementing “the glue” of Xorg to use the arbiter code with this purpose.

So we finally have a working code to do the arbitration of the VGA legacy instructions. The code is separated in three pieces: vgaarb module [0], which is the arbiter itself inside Linux; the libvgaaccess [1], a set of user space functions to access the arbiter; and xf86VGAarbiter [2], the implementation of the library inside Xorg.

I used a part of the money that I earned on the SoC 2007 to buy a laptop to me. My university friends found a good price of a MacBook and we went to the shop in a total of 6 people (!) to buy this laptop trying to get some discount. It cost R$ 3400 (about USD 1800), a very cheap value to the Brazil standards.

Not so much, but here are the news: * For the final evaluation period on the Summer of Code, Daniel suggest me to start my [own X server tree](http://gitweb.freedesktop.org/?p=users/vignatti/xserver.git;a=summary). So I'm maintaining this one with the last bits of the X server input thread implementation and always trying to keep all the things up to date with the upstream tree. Everyone is very welcome to test it and report me the few - I expect - bugs. * The last new regarding the thread implementation is that we're unfortunately dealing with **critical sections** inside the X event queue (mi/mieq.c), so mutex is needed there (just to note: currently, mutex is only implemented using pthread (X11/Xthreads.h), so we also need implement something that is pthread-independent). Probably can exist other pieces of code that might be in an unpredictable way resulting some race conditions (didn't note anything strange yet!). * I finally managed how to implement the page fault notifier without any patch inside the kernel. The read_cr2() can be called directly since the page fault notifier runs with interrupts disabled. The implementation can be found [here](http://web.inf.ufpr.br/vignatti/code/page_fault_notifier.c).

It’s being a great adventure to lock the X input thread on the memory. I’m touching a lot of things that I’d never imagined before :) To trace the pages that are faulting when I move the device pointer I’m using my own ultra mega super kernel’s page fault notifier. It’s very simple, but as the things are not always perfect, it needs a little patch in the kernel.

This week I tried to lock in the physical memory the Xorg’s input code using mlock(). To do this I traced the code minutely and locked all the text and data related to input. I didn’t get success. The mouse still lags when the system is paging (you might remember that with mlockall() all worked wonderful except that it eats much memory). So what might be happening is that something is not locked yet. To guarantee it I searched for a user-space tool that traces page fault. I only found the ’truss’ command on Solaris. Linux (my OS) doesn’t provide no one (‘strace’ don’t do this).

In the last week, I did some cool experiments to see the effects of the D state acting on the X server process when I start it with and without the input thread and always mlock’ing it.

This mail that I’ve sent to xorg mailing list tells the current state of my project. cut here - Hi guys. As you might noted here [1], my GSoC’s project is to do a separate mouse thread for the X server. Now, I’m really stucked with it and I need some good ideas from you before go to the next steps.

(In a puny attempt to write my SoC project progress to my mentor, I decided to expand it and share my thoughts with you) Today, we have two methods to register the pointer devices on Xorg server: (1) under SIGIO and (2) put they fd on EnableDevices set. There is also the silken mouse concept, which means updates fired during sigio handler (in the case of hw cursor).

This week I spent a good time trying to understand the new input hotplug of devices on X and liked a lot. With this new subsystem the multiseat’s life will becomes easier. I’ll briefly explain here to you.

Certainly, my great new is that I was accepted as a student of X.Org Foundation sponsored by Google, on Summer of Code 2007. The project’s name is Moving the mouse handling code into a separate thread and in this link you can see what does this mean specifically.

Here is the presentation I’ve given on 8th International Free Software Forum, at Porto Alegre. My first (really) public talk was really cool and I’ll show to you the video/photos when the organizers get it done.

…this is the subject of my talk which will be held at FISL (International Free Software Forum) on this saturday, April 14th. I’ll post the feedbacks here.

Here is mail that I sent to xorg list a week ago. No one replied it :( On IRC I talked with some developers and asked why they didn’t said nothing about my comments and the answer summarized was that no one is really interested on multiseat (well, at least on first world countries). Strangely, it really give me a good perspective and motivation to keep the work :)

MPX brought to us a new idea of multiseat system. Xat is an application that serves as an intermediary between the X clients and a multiheaded X server. Xat deceives its clients which have the impression that the server has only one head. The objective is that each head behaves like several distinct X servers. We can imagine a scenario where we have a xserver on display :1 with four screens (:1.0, :1.1, :1.2 and :1.3). After Xat being connected we have four displays (:2, :3, :4 and :5) and just one screen per display.

For some reason multiseat doesn’t call much attention in countries that are not of the third world. I can be radical, but in my humble opinion there is no reason to not apply this model of computation in certain types of environment (I’m talking about kiosks, Internet cafe, office, schools, etc). The money saved is enormous, and this is only one of the advantages that I am enumerating here.

This is my first post and even my first blog. Well, the blog’s intention is to bring in evidence my interests about linux (and maybe music).