Author Archives: Marco Martin

Behind Plasma Bigscreen

BlaBlaSoftware

Plasma has been designed from the get go (2006 or so.. it seems at least 2 eternities ago πŸ™‚ ) to not make any assumptions on the type of device and to do a clear separation between the core technology/runtime and the various GUI plugins that end up implementing a full desktop experience.

In an architecture decision informed by previous prototypes we did in KDE4 times for mobile devices UIs, in Plasma 5 we split it further and introduced the concept of a “shell package” which lets further customization between devices than what Plasma in KDE4 times allowed.

Because of that we could do the Plasma Mobile shell without changes to the architecture that runs both the Desktop shell and the mobile version, despite being a completely different UI.

While working on different shells such as Plasma mobile and the Mycroft voice assistant (more on that later) we noticed that the best possible help for building a shell for a new type of device from the ground up is a very minimal shell that doesn’t make many assumptions about the final gui, but just provides few building blocks that will be in the end customized by the developer.

Enter Plasma Nano. Plasma Nano is a minimal shell not intended for end users which the developer will extend to match the final desired user experience (think about subclassing in object oriented languages)

The normal television set is becoming more and more from just a “dumb monitor” to a full featured computer system. But for several factors its UI must be completely different from both a desktop system, and from a mobile system.

Those so called 10 foot user interfaces have some particular constraints. The screen will always be seen from pretty far (altough you can’t be sure exactly the distance the users will look at it) so every graphical element must be very clear and big enough to be seen from a distance (indeed ~10 feet / 3 meters or so) so very big text, clear and spaced (too high information density should be avoided)

Every control should be accessible with the least number possible of buttons. It should be easily controllable with just the remote control, and in particular with just arrow keys, and “ok” button and one to go back. Often tv remote controls have a plethora of buttons, tough which makes the user experience only more confusing.

Another important way of interaction recently in smart tvs to go around the lack of complex input methods is voice control. It is ideal there as the commands to the TV would be very basic like “watch this series on Netflix” and things like that. This of course is giving birth to a slew of very legitimate privacy concerns, but this technology is too good and convenient to throw it away because the current implementations are done by less than honest corporate giants.

Mycroft is an open source project which is aiming to build a fully opensource voice-based personal assistant with a growing number of skills.

Understanding voice is in two independent parts: speech to text (going from the sound file of the recorded voice to a normal string of letters) and actual semantic understanding of the sentence to then translate it to a concrete action (fetching weather data, a Youtube video and so on). The latter part is the bulk of the work for a personal assistant and is what Mycroft implements. it can then use an external service for speech to text.

It can use a variety of services, from Google (speech to text, not full assistant) to a different number of free and proprietary services. In the end, they want to use the Mozilla Deepspeech project, which would make the stack 100% free. You can already configure it to use deep speech or other engines, even if not fully ready yet (want to help out? Mozilla is looking for voice samples to make their product better and ready for mass consumption!)

A voice assistant looks better with a GUI part as well, as the echo show and google assistant demonstrate.

Some of us worked together with the Mycroft people to produce an extensive set of QML bindings for the Mycroft system, (that will provide as wellthe user interface for future Mycroft based smart speakers). They’re a third-party QML module that can be freely used by any Qt-based applciation for voice integration.

Plasma Bigscreen can optionally use those QML bindings to provide parts of the QML UI of the user experience (so yes you can ask via voice to your TV if it’s going to rain tomorrow or to watch the music video of so-and-so on YouTube)

Those QML bindings can provide from Mycroft a variety of GUI features, from simple notifications like a clock if you ask what time it is, to a full featured interactive app, like we did for the Youtube skill, which is a Youtube browser app which can be used both via the voice only, the remote control only, or a combination of both and provides a footprint for future rich voice-interactive user interfaces.

Plasma Bigscreen

Software

Today I want to introduce a project I have been working on together (mostly in the background) with some colleagues of mine… Now with beta status reached, it’s time to more publicly talk about it: enter Plasma Bigscreen.

Smart TVs are becoming more and more complete computers, but unfortunately there the experience tends to be a tight walled garden between proprietary platform, services and privacy-infringing features. Features which are very cool, like voice control, but in order to not pose a threat to the user privacy should be on a free software stack and depending less on proprietary cloud platforms where possible.

Plasma BigScreen is a platform intended to use on smart TVs (trough a powerful enough small computing platform, such as the Raspberry Pi4, or any tiny computer if you need more power) with big remote-friendly UI controls, and Voice activation. What technology did we use for it? Plasma (of course!) and Mycroft.

Plasma Bigscreen main user interface

Plasma Bigscreen is an user interface for TVs providing a 10 foot UI for a smart TV experience (on TVs or any screen with an HDMI connection). It is a Plasma Shell, just like our beloved Plasma Desktop and Plasma Mobile.

It uses the familiar look and feel of Plasma but optimizes all the user interaction to be perfectly readable from the distance and controllable with ha simple basic remote control… from your couch.

Besides interaction with a remote control, it also supports a modern way of iteracting that is revolutionizing the UX on all kind of devices: voice interaction. Optionally, it can integrate seamlessy with Mycroft: an opensource project aimed to offer a completely free voice assistant.

Mycroft offers what are the so-called “skills”. Each skill, takes care of a particular voice interaction. there is an huge variety available, from the weather, do date/time and reminders, to online services clients such as soundcloud and Youtube, with comprehensive QML bindings to show a rich User Interface in a Plasma Environment.

We prepared a beta image for the Raspberry Pi4 for anyone to try, in which we pre-configured Plasma Bigscreen with Mycroft integration ready to go.

If you have a Raspberry Pi4, you can try this image immediately: you can find it here: just flash the file on a MicroSD and is ready to go on any Pi4.

Note that on that image, we point the device to the official Mycroft company’s “Mycroft Home” service, which internally uses the Google STT (Speech to text) which does use google but in a more anonymized fashion as this is not tied to your Google account (nor requires having one). Tough not ideal, it’s possible to configure the Mycroft core to use different services, even those you can self-host like Mozilla Deepspeech.

In here to emphasize voice controls, we prepared some skills (some preinstalled, some can come from the kde store) to fully show voice controls. In particular a Youtube client skill which is perfectly usable both from a remote control and voice only.

Youtube skill shipped in the image

As remote control, you can use the remote of your TV directly if it supports the CEC standard, or any of those remotes with an usb dongle which are “seen” as a keyboard with just few standard keys (some have also an integrated microphone, so giving voice commands is very easy).

In the next days, there will be more posts explaining the architecture and the technical choices we made, in the mean time, have a try and if you want to get in contact with us about the project, you can find directions here.

Fosdem and Plasma Mobile Sprint

BlaBlaSoftware

From January 31st to February 8th I went on a little tour, first at the two days of Fosdem in Brussels, then to Berlin for a KDE sprint about Plasma Mobile.

It was the first time i went to Fodem: it’s an awesome experience, even tough big and messy: which is the awesome of it… and the bad of it at the same time πŸ™‚

Even tough there were 800 talks I didn’t attend that many, some about the Elixir language, some about retrocomputing, some about iot stuff. At Fosdem the best thing to do there.. is meeting a lot of interesting people, rather than attending talks, which are very interesting never the less, which you can find videos here.

I stayed most of the time at the KDE booth. in there we had as usual some hardware to show: besides the usual occasional “normal” laptops running Plasma, we had a PineBook pro running Plasma on Manjaro which runs impressively well for such a low resource machine and was in fact a favorite among the visitors.

Yep, there were a lot of people.

Besides that we had some Pinephones running Plasma Mobile, which they were very popular as well. The Plasma Mobile shell was very stable compared to last year fosdem which made me quite happy.

Looking people using it for the first time, was also a very precious feedback on what are the UI problems new users can encounter.

Feedback that turned out to be very useful the next week, when together other people working on Plasma Mobile and two perople from the UBports project few to Berlin for a Plasma Mobile sprint, graciously hosted by KDAB in their spaces.

There will be many points we will be able to collaborate with UBPorts, starting from background technology such as their telephony services, the content sharing infrastructure, and maybe push notifications. Tough in the end, we want their apps running smoothly on Plasma Mobile and our apps running smoothly on UBports as well, to have as many apps as possible.

Personally, the areas that i worked more at the sprint were the Plasma Mobile homescreen and fixes in Kirigami that were needed for some plasma mobile apps to look better.

On the Plasma Mobile homescreen, now dragging plasmoid thumbnails from the widget explorer to the homescreen works properly (fix in KWin, also drag and drop icons in your desktop wayland session should be fine now)

Plasma Mobile, Clean homescreen look

In order to make the homescreen look a bit more “modern” and less busy, I also removed most of the gray background rectangles that were there and in general simplified the homescreen code a bit, here is the result:

In general, it seems to me that Plasma Mobile is actually really start coming together πŸ™‚

KF6 Sprint in Berlin

BlaBlaSoftware

From the 21st to 24st of Novemember, a bunch of KDE people gathered in Berlin graciously hosted by the MBition offices to discuss about the next big iteration of the KDE frameworks.

Work on Qt6 started, and it will be a big refactor that makes the api quite better, solve some architectural problems in some Qt5 areas (one of my personal favorites is the new QGuiAction class coming out of the split of QAction in a QWidget-less implementation). In order to have that, it needs to be binary incompatible with Qt5 tough.

As it will be incompatible, we need to adapt our software and our frameworks.. a lot of work ahead, but big opportunities as well, so… It’s time for KF6, where we can do the same thing: polish our API and solve some problems we couldn’t do in KF5 in a binary compatible way.

We worked in groups, assisted remotely by David Faure and looked at the frameworks we were less happy and need more a refactor.. The first obvious candidate are those tier 3.

Our fancy Kanban board

The Tier system in frameworks means that frameworks of tier 1 don’t depend from other things than Qt modules and base system libraries. Tier 2 frameworks can also depend from tier 1 frameworks, and tier 3 from tier 2.

In reality there are some tier 3 frameworks which “real” tier would be 5 or 6 as may depend from multiple other tier 3 frameworks. Those of course are the first that need to be looked at. Ideally we should manage to lower the tier of frameworks as much as possible and at least having tier 3 ones that are actually tier 3, and not 4 or 5.

Some typical examples are KIO, KDeclarative, KXmlGui, and Plasma-framework.

For the KDeclarative case, the reason is that its genesis was quite peculiar. When in late KDE4 – early KF5 was starting to be apparent that the focus of the future in GUI programming in Qt was probably going to shift towards QML, we chosen at first to put the bindings to QML of the frameworks we needed in a single umbrella repository, ending up with a git repo full of many tiny QML plugins that depended from one framework or another… ending up with a framework that depended from just about all the others. It will be split out and every useful QML binding will go into the proper framework: as QML is now a super central part of Qt, frameworks need to play well with it to be an important citizen of the Qt ecosystem.

Plasma framework

Now talking about a part that is really near to my hearth: Plasma-framework. It’s a pretty high tier because it depends from KXmlGui and KDeclarative. As we seen, both of those dependencies will be not too hard to remove (famous last words :).

My plan is to have the main plasma library splitted in 3 smaller frameworks, each one tier2 maximum:

  • libplasma: the part that manages load and save of your desktop layout, all that is related of loading a plasmoid and the api that palsmoids will use to interact with the Plasma workspace
  • theming: Plasma uses svg-based themes with significant optimizations like disk-caching of the rendered bits, which support stylesheets for dynamic colors based on your system ones: this should be a framework in itself, usable by any app: probably tier 2
  • dataengines: that’s a technology not much used anymore and is kinda being phased out. Should exist standalone as a “porting aid”

With this, hopefully Plasma will be even leaner, further improving startup time and memory usage, while on the same time applications gains a framework for doing light weight and feature rich svg based graphics theming.

Plasma Edit Mode refinements

GraphicsSoftware

Editing, moving and customizing widgets in Plasma Desktop improved a lot in 5.17, and then in 5.18 it will get a brand new edit mode, to be really efficient editing your desktop layout (and have less visual noise by default).

This week another new feature landed in the edit mode for 5.18: it’s possible to set some plasmoids without background and a nice drop shadow, for an extra clean and modern look for your desktop.

In addition, a plasmoid can specify this backgroundless shadowed mode as its new default, like the digital clock now does (when is on the deskop)

Applet developers will have to opt in this feature: doesn’t always make sense everywhere and some may still be buggy. Some default plasmoids, especially in kdeplasma-addons have been set to support it.

If you are writing a plasmoid and want to support this background removal, in the main QML item of your plasmoid you can do:

Item {
    id: root   
    Plasmoid.backgroundHints: PlasmaCore.Types.DefaultBackground
                             | PlasmaCore.Types.ConfigurableBackground
    ...
}

Or, if you want to default toi the new background-less, automatic shadow:

Item {
    id: root   
    Plasmoid.backgroundHints: PlasmaCore.Types.ShadowBackground
                             | PlasmaCore.Types.ConfigurableBackground
    ...
}

Of course the NoBackground option is still there if a shadow isn’t what the graphic style of the plasmoid needs.

A notice that needs to be kept in mind, is that for color inversion to work automatically, one should never do:

    color: theme.textColor

But rather

    color: PlasmaCore.ColorScope.textColor

Which is a good way to get ready for Qt6 anyways, as context properties are going to be removed in the next Qt major release.

A week in Valencia

Software

From 19th to 25th of June, all the Plasma team gathered in Valencia, graciously hosted by the Slimbook people in their office. This was a special sprint, as it was co-located with the Usability sprint together with some VDG members. While some of the time each team was occupied in their own discussions, there were a big margin of overlap, allowing us to have a lot of discussions about the design and usability of our beloved Plasma desktop shell.

We now have plans in the coming months for several improvements across the board, including further improvements on the new shiny notification framework by Kai Uwe.

Also, we talked (and worked on) plans for further improving our Wayland support, including middle mouse button clipboard, and screen rotation for phone, tablets and 2 in 1 laptops).

On my end, a big part of this sprint was dedicated to an encompassing plan to refactor and redesign how desktop plasmoids work and are managed. I had there both UI discussions with the VDG and loong coding sessions on it.

The Desktop/FolderView containments use a big infrastructure written in Javascript which has some problems and isn’t touched much since a lot of time.

Some time ago I set myself the task of making the management of desktop plasmoids more touchscreen-friendly, so i started modifying that code, until.. I started to design a complete reimplementation written in C++ πŸ™‚

This new implementation is much more robust, is faster and a bit leaner on the memory. Most important, is now a separate QML plugin, so is not anymore an implementation internal in the standard desktop, but if somebody wants to write his/her own containment for personalized plasma shells (for instance for a particular embedded device which is not a traditional desktop/laptop: we want plasma more and more usable as a set of construction blocks for the main UI of any kind of device).

In fact, it’s planned for Plasma Mobile to use the same layout manager component, to make the user experience “similar but different” and have less code duplication, while maintaining the UI very distinct between the two very different device types.

UI-wise in the desktop it doesn’t change much for now. the most notable difference is visible resize handles that make managing the layout and moving/resizing the plasmsoids much easier and more intuitive. Especially with touchscreen: now while manipulating plasmoids via touch, the resize handles become way bigger, and is possible to move and resize via a pinch gesture as well.

Lastly (for now!) the behavior during screen resolution switch improved a lot: if you connect a projector with a smaller resolution that relayouts your desktop, or play a fullscreen game at a tiny resolution, when the resolution is restored, everything gets back to normal, no more applets all over the place after changing resolution πŸ˜‰

All of this should make it into Plasma 5.17.

A mobile Plasma Sprint

Software

I was last week in Berlin at the Plasma Mobile sprint, graciously hosted by Endocode, almost exactly 9 years after the first Plasma Mobile sprint in which we first started to explore Plasma and other software by KDE on mobile phones, which at the time were just starting to become powerful enough to run a full Linux stack (Hi N900!)

Now the project got a wide breath of fresh air: the thing that impressed me the most was how many new faces came at the sprint and are now part of the project.

Compared to 9 years ago, we have a way saner and more robust ecosystem to play on. Instead of a single (and quite underpowered) phone, which was the N900, now we can hack on a wide variety of phones, thanks to ARM being slightly less painful to work with compared to back then (even tough still a long way to go to be considered an open hackable system from nay point of view) Some devices are starting to get upstream mainline kernel support, and for those (unfortunately, most) who don’t there is the Halium project to the rescue, which provides an abstraction layer between the Android kernel and the “proper Linux” userland, making possible to use its graphjics drivers to drive a Wayland session, access audio and connectivity and so on.

We have a Neon version which supports a reference device (the old Nexus-5x) which can be easily adapted to other devices, and PostmarketOS, which is a distribution which supports many different phones and several user interfaces, Plasma Mobile being one of the official ones. At the sprint there were also some people from the PostmarketOS project: one of the things i love most of open source is when different projects collaborate so closely.

We had also some new toys to play with: people from Purism were also present, bringing development kits for their upcoming Librem5 phone, which will support mainline kernel and no need for closed Android drivers. Even tough a lot of work is stil lto do, Plasma Mobile already boots on the device.

Plasma Mobile running on a Librem 5 devkit.

As for Plasma Mobile software in itself, we did many bugfixes on the main shell/homescreen to have a better first impact, and a significant improvement came in KWin about high DPI scaling when running on an Halium system.

Also, many improvoements were done in the Kirigami framework, which is the main toolkit recommended to be used to build applications for Plasma Mobile: as developers of several applications that use Kirigami were present there, we could do very fast feedback and debug sessions.

New Kirigami communication channels

BlaBla

Kirigami used to have a Telegram channel as its main communication channel. this is of course not optimal being a closed service and many potential contributors not having an account on Telegram.
Since today, we also have an IRC channel
#kde-kirigami on freenode and
#kirigami:matrix.org on Matrix
The Telegram channel is still there, and all 3 are bridged between each other, so a message sent by any of the 3 platforms will be received also by users on the other two.
See you there πŸ™‚

Akademy 2018

BlaBlaSoftware

The time for Akademy came this year as well, this year it was in the gorgeous Vienna, Austria.
This year marks my 10th Akademy in a row, starting from my first one in Belgium in 2008.
Talks have been awesome as usual, but what’s always awesome for me year by year is all the face to face conversation with so much diverse and smart people in out awesome KDE community.

Kirigami

For me the highlight was the BOF session on Kirigami, in which some nice plans, together the VDG are starting to form.
Kirigami in a QML based UI framework at the core of some KDE applications, which will become more and more central as more and more QML based applications are made.
So far is still a relatively unknown gem in the KE software and frameworks offering, however as technologically is starting to mature, we’ll start to advertise it more and simplify onboarding.
A big part of that will be about web presence and documentation:

  • A nice media-heavy introductory website which will showcase the features it can offer to your app, together expanded sections of the central Kirigami UX patterns in the new Human Interface Guidelines website.
  • Improving the API documentation
  • The Kirigami channel on Telegram will need IRC and Matrix bridges
  • A series of tutorials how to get started developing applications using the Kirigami toolkit
  • Repurpose the example “Kirigami Gallery” application: It will become a showcase of components and UI patterns the developer is recommended to use: each gallery page will also have documentation text together links to the corresponding HIG page and to the gallery page sourcecode itself, to be used as a source of inpiration and best practiches to be used while developing your application

Kirigami Gallery on the Cards pattern, mobile version

Kirigami Gallery on the Cards pattern, desktop version

If you think you can help on this web presence effort, you are welcome to join πŸ™‚

Plasma

On the Plasma side, many plans of improvement have been discussed and are on their ways, such as better support for touch-based convertible laptops, a completely rewritten and overhauled notification system, and improved Virtual Desktops/Activities infrastructure and UI, on Wayland too.
But, more on all of this in the future πŸ™‚

Vienna

Vienna is a really charming and beautiful city, I would totally recommend going there at least once.


It’s home not only to great musician in the past:

Mozart


But also to Important scientists that contributed so much to the knowledge of humanity and.. contributed a littel bit making possible all the technology we know and love πŸ™‚

Meow!

Kirigami and color palettes

GraphicsSoftware

A new release of Kirigami is about to come with the new version of KDE Frameworks about to be released, 5.38.
This is an important release, which bumps the import version to 2.2, because has a very important new feature: A brand new (but retrocompatible) color theming API, which allows different areas of the application to have different color domains, allowing for instance parts of the application to have a light color palette and parts of the application to have a dark one.
This model is modeled after KColorScheme which is very powerful (even tough sometimes underused): as KcolorScheme will be actually used when running on Plasma Desktop (just a runtime plugin, it won’t depend on it, so dependencies on Android or other platforms are not affected) it will allow us to integrate tightly with the other applications made by KDE.

Here you can see the example Kirigami Gallery which has been set to use different color sets in different areas:

Kirigami Gallery with the Breeze widget style: areas with a different ColorSet, widget and icon colors following

Here, this is how choosing different color sets for different areas of the application looks on Android, using the Material style:

Theme.colorSet

The Theme object in Kirigami used to be a singleton (and still is, for full compatibility when the 2.0 or 2.1 version of the import is used), but is now instead an attached property: which is source-compatible (no rewrite needed in your apps, except bumping the import version to 2.2)
You can color differently an area of you application with the following code:

 Item {
    // tells the Theme no not inherit the color set from parent objects
    Kirigami.Theme.inherit: false
    // uses Complementary color set
    Kirigami.Theme.colorSet: Kirigami.Theme.Complementary
    Rectangle {
        //this color will come from the Complementary set
        color: Kirigami.Theme.backgroundColor
    }
 }

Another example of a (very minimal) full application which has the Global Drawer in the Complementary color set:

import QtQuick 2.6
import org.kde.kirigami 2.2 as Kirigami

Kirigami.ApplicationWindow {
    id: root

    globalDrawer: Kirigami.GlobalDrawer {
        Kirigami.Theme.inherit: false
        Kirigami.Theme.colorSet: Kirigami.Theme.Complementary
        title: "Hello App"
        titleIcon: "applications-graphics"

        actions: [
            Kirigami.Action {
                text: "action 1"
            },
            Kirigami.Action {
                text: "action 1"
            }
        ]
    }
    contextDrawer: Kirigami.ContextDrawer {
        id: contextDrawer
    }

    pageStack.initialPage: mainPageComponent

    Component {
        id: mainPageComponent
        Kirigami.ScrollablePage {
            title: "Hello"
            Rectangle {
                anchors.fill: parent
            }
        }
    }

}

Colored monochrome icons

In the above screenshot, you can also see that the icons used there are colored accordingly to the same text color of the ColorSet of the area they are in. When loading in Plasma Desktop, we have the luxury of loading such icons with KIconLoader, which can process SVG-based icon sets with stylesheets, so we can actually have in the icons areas colored with particular named-colors, which correspond with the Theme’s named colors (like textColor, backgroundColor, highlightColor, negativeTextColor and so on) for instance is important that a record icon like the one shown in the screenshot has a red dot as this is the universal accepted iconography. The monochrome icons in the Breeze icon set are “almost” monochrome, with those few colored accents used very sparsely, just when really needed and give definitely a nice touch of polish to the visual identity.
On other platforms they are just treated as monochorome and the whole icon is colored.

Better integration with Plasma Desktop and QWidget style

A QtQuickControls2 style has been written which uses QStyle to paint controls and will be used by default in next Plasma releases (from 5.11 onwards), this is released as a framework as well in 5.39, called qqc2-desktop-style
Here a screenshot of the Gallery using Oxygen widget style and icons:

Kirigami Gallery on desktop, shown with the Oxygen widget style

Different platforms plugins

The framework qqc2-desktop-style also contains a plugin that is dynamically loaded by Kirigami, which bridget the Kirigami Theme.* concept to KColorScheme, making Kirigami apps follow the same palette, as well as using KIconloader to load icons, being capable to apply color stylesheets to Breeze icons.
On other platforms, simpler plugins will be used to not drag dependencies and integrate better on whatever platform they are (like Material on Android)

What about fancy icons on my QWidget app?

Since several KF5 releases, Breeze icons in QWidget-based apps can follow the system palette, so when using a dark color scheme, icons will become white and so on. However: what about using a light color scheme in some areas and a dark one in some other areas?
Since 5.39 KIconLoader has gained a new method, setCustomPalette(), in which you can override the colors for a particular KIconLoader (the app will then need to use different KIconloader instances intended for different areas of the app). Pay attention that in that case you will have to monitor the QApplication palette change and eventually update that kiconloader palette by hand.