Category Archives: Software

Towards a transition from KWallet to Secret Service

Software
Leave a reply

Historically passwords and credentials in all of our apps and services (such as kio and our Network Manager plasmoid), are stored and managed by our KWallet subsystem and API. In a similar way, GNOME had gnome-keyring, and other similar systems were available. A major problem was the mutually incompatible interface between them, making impossible to have a single place where all the passwords go.

For this reason, a standard DBus interface called Secret Service has been developed, and systems like KWallet, gnome-keyring and KeepassXC have adopted the interface as well.

In the future, we want to eventually port our applications to use the Secret Service API directly, preferably via the QtKeychain library (some applications already do), which will use Secret Service natively on Linux. And as a bonus the Windows/Android native systems on those platforms will work too.

This will make our applications work much better, be more integrated in other desktops or platforms, and be less dependent on the KWallet framework which has big legacy code parts at this point.

Right now, KWallet has the option to bring up a Secret Service compatible interface, and this is a valuable first step, but there are still some potential problems in the migration path:

  • Applications using Secret Service will store their secret data in different places when running in different environment, including our own that have been migrated. If you login into a different session ,every app will have forgotten its credentials.
  • It’s possible to use a different Secret Service provider also in a Plasma session, for instance enabling it in KeepassXC. At that point some apps will save into it, while other ones will keep using KWallet with a different storage backend.
  • Applications ported from KWallet to SecretService will lose their credentials, unless some clunky porting code is written for each application.

A KWallet compatibility layer

Enter the recent refactor that happened in KWallet: it has now been split into 2 different system services now: one that exposes only the Secret Service API, and one that exposes only the KWallet API.

The service exposing the KWallet API has been written from scratch and is now just a thin wrapper around the Secret Service API, translating the KWallet DBus calls to Secret Service ones. The old KWallet service is now a pure Secret Service provider that just happens to use the old KWallet backend, exposing to Secret Service all the entries that had already been stored in the past.

This decouples our KWallet compatibility layer for existing or old applications (not only KDE applications, also third party ones like Chromium use it) with our actual secret storage and SecretService implementation, allowing a separate development roadmap, and even potential future transition to a different backend in a completely transparent way to the user.

You can see this decoupling in the same spirit as the recent KWin Wayland/X11 split: being able to develop the new technology without risking breaking compatibility of the legacy system.

What it means for users and developers

For the immediate future, not much really changes: users will still have all their secrets accessible, and every app they use that was using KWallet will keep working without forgetting anything.

In the same way, for developers, the whole KWallet C++ API keeps working exactly as it was (even though we’re discouraging its use in new projects). Also, the KWallet-to-Secret-Service API proxy layer will save data inside Secret Service with the same schema used by QtKeychain, keeping the data accessible if the application gets ported from using the KWallet C++ API to QtKeychain.

An experimental feature

Disclaimer: The following description is of an experimental feature behind a hidden configuration key: it will take a bit before it’s deemed ready for prime time.

One advantage of having 2 independent services to handle the KWallet API and the Secret Service one is that now it’s possible to chose different backends as well, such as KeepassXC, gnome-keyring or oo7.

Setting the following in kwalletrc:

[Migration]
MigrateTo3rdParty=true
[KSecretD]
Enabled=false

With this, the old KWallet backend (now a service called ksecretd) won’t be started anymore, and instead any Secret Service provider that is running or has been configured to be DBus-activatable will be used. The first time this happens, a data migration procedure will be executed, writing the data in KWallet into the new service, keeping all the user-saved secrets accessible.

New Plasma Edit Mode in 6.1

GraphicsSoftware

In Plasma, when the user wants to do some significant layout change in the desktop, such as adding, moving or removing panels or add and manage desktop widgets, will go in the so called “edit mode”, by a context menu entry when driven by mouse, or by long-press in the desktop when driven by touchscreen.

Let me introduce you the improved edit mode workflow Plasma 6.1 will have:

An “edit mode” is necessary because since is an operation potentially destructive, we really want to avoid the user to for instance remove their own taskbar or some similar operation by mistake, during day to day use.

“Modes” in UI are a delicate thing: they have advantage as well many possible problems, which come in when it’s not immediately clear in which mode the user is in, and can try to use the UI as it was “in the other mode” (see https://www.nngroup.com/articles/modes/ as a quick introduction on advantages and disadvantages of modes).

This was a bit of a problem of the edit mode in plasma, as the edit mode was not clearly visually separated with the normal UI workflow of the day to day plasma usage.

Another problem present was that the action buttons of the edit mode toolbar were often hidden away by either the panel configuration window or the add widgets sidebar.

This new mode zooms out the desktop with the same visual effect of the window overview effect (yay for consistency), to well differentiate between normal and edit mode. Using the same zoom effect it puts the whole desktop “out of the way” when either the add widget sidebar of the panel configuration window appear, making sure the whole desktop area is always reachable, applets can be dropped anywhere in the desktop and the edit mode toolbar is always completely accessible.

Akademy 2023, Plasma 6 and Plasmoids

Software

During this week Akademy 2023 is going on in Thessaloniki, Greece. It’s always awesome, to see many old friends and getting together with that amazing hacker community which is KDE.

There, me and Niccolò gave a talk about what;s happening in Plasma 6 and what will change, Noccolò on more visual things, about some changes we are cooking on the UI and on the visual themes. Here you can find a recording of the talk (alongside all the others of the day)

I talked more about the work I’ve bein doing in the Plasma shell during the last couple of months: code rafactors and how the API for writing plasmoids will change.

There were many things we were not quite happy about and now with the major release is the occasion for streamlining many things.

Now, It’s very important those changes are are well communicated, and easy to do for developes, because there are *a lot* of 3rd party plasmoids on the KDE store, which people are using and enjoying.

Let’s go trough the most important changes:

Dataengines

Dataengines were an API designed in early KDE 4 times, especially one of for our first offereings of Plasmoid API which was the pure JavaScript API, which existed long before the QML existed.

But now in a QML world, their API doesn’t really fit, instead is a much better fit having a QML extension which offers classes with all the needed properties, data models and signals that provide access to the needed data, such as tasks, notifications etc.

Dataengines are now deprecated and moved into a separed library, called “plasma5support” which will be still available for the time being, but consider porting away from it as we plan to eventually drop it.

Base Plasmoid API

The way plasmoids are declared in QML changed a bit: we used to have a magical “plasmoid” context property available from anywhere. This was an instance of a QQuickItem which was both the item where all the plasmoid contents were *and* a wrapper for some of the api for the central plasmoid object: the C++ class Plasma::Applet.

Now the access to plasmoid is an attahced property, the (uppercase) “Plasmoid”, which is directly the access to the instance of the central Plasma::Applet, without an in-between wrapper anymore.

The central QQuickItem is now called “PlasmoidItem”, and must be the root item of the plasmoid, just alike ApplicationWindow is for applications.

PlasmoidItem will have the purely graphical properties, such as the “compactRepresentation” or “fullRepresentation”

Here is a very minimal example of a plasmoid main file under plasma6:

import org.kde.plasma.plasmoid 2.0
PlasmoidItem {
    Plasmoid.title: i18n("hello")
    fullRepresentation: Item {....}
}

Actions

Plasmoids can export actions to their right mouse button menu, such as “mute” for the mixer plasmoid and so on.

In Plasma 5 we had an imperative API to add those actions, which was again coming from that old pure JS API, which really looked a bit out of tune in QML. In Plasma 6 the API has been replaced with a completely declarative API, in this form:

PlasmoidItem {
    Plasmoid.contextualActions: [
        PlasmaCore.Action {
            text: i18n("Foo")
            icon.name: "back"
            onTriggered: {...}
        },
        PlasmaCore.Action {
             ...
        }
    ]
}

PlasmaCore.Action is actually a binding to QAction (not the internal QML action type), so that it can be shared between C++ and QML easily

SVG theming

Plasma Themes don’t really have changed for now (and you can expect any old theme from the store to keep working), but the C++ and QML API for them has been moved to a standalone framework called KSvg. Plasma Svgs have quite some interesting features over the pure QtSvg API, such as disk caching of the rendered images, stylesheet recoloring to system colors and the 9 patch rectangular stretched images of FrameSvg.

Some applications were interested in using that, but couldn’t due to the long dependency chain of plasma-framework, so now they can be used as a much more manageable compact framework, offering both the usual C++, QPainter based api and QML bindings.

import org.kde.ksvg 1.0 as KSvg
FrameSvg {
    imagePath: "widgets/background"
}

Kirigami all the way down

Designing Kirigami in the beginning we lifted two concept from the Plasma API (which again we couldn’t use directly due to the dependency chain) Theme and Units

Theme gives access to the named system colors, and Units to standard spacing and animation durations.

Over the years the Kirigami version got way more advanced then the Plasma version, and having this code duplication didn’t make much more sense, to in Plasma6 whenever referring to a named color or an unit, the Kirigami version should be used, as the Plasma version is going away.

import org.kde.kirigami 2.20 as Kirigami
RowLayout {
    spacing: Kirigami.Units.smallSpacing
    Rectangle {
        color: Kirigami.Theme.backgroundColor
        bordere.color: Kirigami.Theme.textColor
    }
}

Plasma Sprint 2023

BlaBlaSoftware

IT feels like a billion years before the last plasma sprint, which was in 2019 in Valencia, before the pandemic, but finally this year we are back on track, and was great to see again many old friends as well as seeing many new faces for which it was the first sprint.

We were gracefully hosted by Tuxedo Computers in Augsburgh, makers of very nice laptops that come with Linux and KDE Plasma, as well as being KDE patrons.

First of all, everybody got up to speed with a full git build of a Plasma 6 session, so that everybody could participate in development and discussions from the same level.

There were many discussions about Plasma 6, about what we want to do in Plasma and in Kirigami, how we want to change the look and defaults for the new major release. Most of the user-facing changes have been wonderfully described by Nate.

On my part, I worked on mainly 2 things, that were fairly not “glamorous” but quite important never the less (and mildly painful to do) : a refactor of the plasmoid loading code and splitting all the Svg themes code to a new framework with far less dependencies, ideally usable by any application.

Plasma API

I spent most of my hacking time at the sprint on a refactor of the plasmoid loading code, which won’t be really “seen” by the user, but will make the infrastructure much more robust and the API cleaner.

The person which must pay attention to it is the plasmoid author, which will need to adapt the plasmoid code in a few places.

Most notable is that, just like when you are writing a QML application you have to use the ApplicationWindow root QML Item, for a plasmoid you now have to use a PlasmoidItem root object, so something like

Item {
    Plasmoid.compactRepresentation: Label {text: Plasmoid.title}
    Plasmoid.fullRepresentation: Item {...}
}

becomes:

PlasmoidItem {
    compactRepresentation: Label {text: Plasmoid.title}
    fullRepresentation: Item {...}
}

A full porting guide is in progress.

KSvg

Using the Plasma Svg code to support stylesheet recoloring, on disk image cache to speed up loading, and the 9-patches FrameSvg is something the several applications would be interested to, and some actually are already doing, but since plasma-framework has a lot of dependencies, for some applications that is a blocker. All the svg code has now been broken out into a new framework called KSvg, which is still work in progress, but in the end will support all existing plasma themes with no change, and if an application wishes to use it, the svg sets will be loaded from the app own data folder (or anywhere else the application configures it to) instead of the share/plasma/desktoptheme folder, where plasmashell looks for them (so they can also use a complete different theme structure and don’t have to provide the same elements)

How to report Multiscreen bugs

BlaBlaSoftware

As announced previously, Plasma 5.27 will have a significantly reworked multiscreen management, and we want to make sure this will be the best LTS Plasma release we had so far.

Of course, this doesn’t mean it will be perfect from day one, and your feedback is really important, as we want to fix any potential issue as fast as they get noticed.

As you know, for our issue tracking we use Bugzilla at this address. We have different products and components that are involved in the multiscreen management.

First, under New bug, chose the “plasma” category. Then there are 4 possible combinations of products and components, depending on the symptoms:

Possible problemProductComponent
  • The output of the command kscreen-doctor -o looks wrong, such as:
  • The listed “priority” is not the one you set in systemsettings
  • Geometries look wrong
kscreencommon
  • Desktops or panels are on the wrong screen
  • There are black screens but is possible to move the cursor inside them
plasmashellMulti Screen Support
  • Ordinary application windows appear on the wrong screen or get moved in unexpected screens when screens are connected/disconnected
  • Some screens are black and is not possible to move the mouse inside those, but they look enabled in the systemsettings displays module or in the output of the command kscreen-doctor -o
kwinmulti-screen
  • The systemsettings displays module shows settings that don’t match reality
  • The systemsettings displays module shows settings that don’t match the output of the command kscreen-doctor -o
systemsettingskcm_kscreen

In order to have a good complete information on the affected system, its configuration, and the configuration of our multiscreen management, if you can, the following information would be needed:

  • Whether the problem happens in a Wayland or X11 session (or both)
  • A good description of the scenario: how many screens, whether is a laptop or desktop, when the problem happens (startup, connecting/disconnectiong, going out of sleep and things like that)
  • The output the terminal command: kscreen-doctor -o
  • The output of the terminal command: kscreen-console
  • The main plasma configuration file: ~/.config/plasma-org.kde.plasma.desktop-appletsrc

Those items of information already help a lot figuring out what problem is and where it resides.

Afterwards we still may ask for more informations, like an archive of the main screen config files that are the directory content of ~/.local/share/kscreen/ but normally, we wouldn’t need that.

One more word on kscreen-doctor and kscreen-console

Those 2 commands are very useful to understand what Plasma and the rest of the system thinks about every screen that’s connected and how they intend to treat them.

kscreen-doctor

Here is a typical output of the command kscreen-doctor - o:

Output: 1 eDP-1 enabled connected priority 2 Panel Modes: 0:1200x1920@60! 1:1024x768@60 Geometry: 1920,0 960x600 Scale: 2 Rotation: 8 Overscan: 0 Vrr: incapable RgbRange: Automatic
Output: 2 DP-3 enabled connected priority 3 DisplayPort Modes: 0:1024x768@60! 1:800x600@60 2:800x600@56 3:848x480@60 4:640x480@60 5:1024x768@60 Geometry: 1920,600 1024x768 Scale: 1 Rotation: 1 Overscan: 0 Vrr: incapable RgbRange: Automatic
Output: 3 DP-4 enabled connected priority 1 DisplayPort Modes: 0:1920x1080@60*! 1:1920x1080@60 2:1920x1080@60 3:1680x1050@60 4:1600x900@60 5:1280x1024@75 6:1280x1024@60 7:1440x900@60 8:1280x800@60 9:1152x864@75 10:1280x720@60 11:1280x720@60 12:1280x720@60 13:1024x768@75 14:1024x768@70 15:1024x768@60 16:832x624@75 17:800x600@75 18:800x600@72 19:800x600@60 20:800x600@56 21:720x480@60 22:720x480@60 23:720x480@60 24:720x480@60 25:640x480@75 26:640x480@73 27:640x480@67 28:640x480@60 29:640x480@60 30:720x400@70 31:1280x1024@60 32:1024x768@60 33:1280x800@60 34:1920x1080@60 35:1600x900@60 36:1368x768@60 37:1280x720@60 Geometry: 0,0 1920x1080 Scale: 1 Rotation: 1 Overscan: 0 Vrr: incapable RgbRange: Automatic

Here we can see we have 3 outputs, one internal and two via DisplayPort, DP-4 is the primary (priority 1) followed by eDP-1 (internal) and DP-3 (those correcpond to the new reordering UI in the systemsettings screen module).

Important data points, also the screen geometries (in italic in the snippet) which tell their relative positions.

kscreen-console

This gives a bit more verbose information, here is a sample (copied here the data of a single screen, as the output is very long):

Id: 3
Name: "DP-4"
Type: "DisplayPort"
Connected: true
Enabled: true
Priority: 1
Rotation: KScreen::Output::None
Pos: QPoint(0,0)
MMSize: QSize(520, 290)
FollowPreferredMode: false
Size: QSize(1920, 1080)
Scale: 1
Clones: None
Mode: "0"
Preferred Mode: "0"
Preferred modes: ("0")
Modes:
"0" "1920x1080@60" QSize(1920, 1080) 60
"1" "1920x1080@60" QSize(1920, 1080) 60
"10" "1280x720@60" QSize(1280, 720) 60
"11" "1280x720@60" QSize(1280, 720) 60
"12" "1280x720@60" QSize(1280, 720) 59.94
"13" "1024x768@75" QSize(1024, 768) 75.029
"14" "1024x768@70" QSize(1024, 768) 70.069
"15" "1024x768@60" QSize(1024, 768) 60.004
"16" "832x624@75" QSize(832, 624) 74.551
"17" "800x600@75" QSize(800, 600) 75
"18" "800x600@72" QSize(800, 600) 72.188
"19" "800x600@60" QSize(800, 600) 60.317
"2" "1920x1080@60" QSize(1920, 1080) 59.94
"20" "800x600@56" QSize(800, 600) 56.25
"21" "720x480@60" QSize(720, 480) 60
"22" "720x480@60" QSize(720, 480) 60
"23" "720x480@60" QSize(720, 480) 59.94
"24" "720x480@60" QSize(720, 480) 59.94
"25" "640x480@75" QSize(640, 480) 75
"26" "640x480@73" QSize(640, 480) 72.809
"27" "640x480@67" QSize(640, 480) 66.667
"28" "640x480@60" QSize(640, 480) 60
"29" "640x480@60" QSize(640, 480) 59.94
"3" "1680x1050@60" QSize(1680, 1050) 59.883
"30" "720x400@70" QSize(720, 400) 70.082
"31" "1280x1024@60" QSize(1280, 1024) 59.895
"32" "1024x768@60" QSize(1024, 768) 59.92
"33" "1280x800@60" QSize(1280, 800) 59.81
"34" "1920x1080@60" QSize(1920, 1080) 59.963
"35" "1600x900@60" QSize(1600, 900) 59.946
"36" "1368x768@60" QSize(1368, 768) 59.882
"37" "1280x720@60" QSize(1280, 720) 59.855
"4" "1600x900@60" QSize(1600, 900) 60
"5" "1280x1024@75" QSize(1280, 1024) 75.025
"6" "1280x1024@60" QSize(1280, 1024) 60.02
"7" "1440x900@60" QSize(1440, 900) 59.901
"8" "1280x800@60" QSize(1280, 800) 59.91
"9" "1152x864@75" QSize(1152, 864) 75
EDID Info:
Device ID: "xrandr-Samsung Electric Company-S24B300-H4MD302024"
Name: "S24B300"
Vendor: "Samsung Electric Company"
Serial: "H4MD302024"
EISA ID: ""
Hash: "eca6ca3c32c11a47a837d696a970b9d5"
Width: 52
Height: 29
Gamma: 2.2
Red: QQuaternion(scalar:1, vector:(0.640625, 0.335938, 0))
Green: QQuaternion(scalar:1, vector:(0.31543, 0.628906, 0))
Blue: QQuaternion(scalar:1, vector:(0.15918, 0.0585938, 0))
White: QQuaternion(scalar:1, vector:(0.3125, 0.329102, 0))

Important also the section EDID Info, to see if the screen has a good and unique EDID, as invalid Edids, especially in combination with DisplayPort is a known source or problems.

Multi Screen

BlaBlaSoftware

Ah, working with more than one screen, nice and convenient, but always has been the scourge of usability, due to broken hardware and software, on every platform you can imagine. And yes, on Plasma we are notorious to have several problems with regard to multi screens, which spurred many bug reports.

TL;DR

For Plasma 5.27 we did a big refactor on how the screens are managed (and how they are mapped to desktops and panels) which hopefully should solve many of those issues, creating a much more predictable experience, both for “fixed” multi screen setups (like a setup at your desk) and “on the go” (like attaching a projector at a conference).

  • No more default empty desktops after connecting a screen that you already connected in the past
  • No more desktops, wallpapers, widgets and panels lost after restart
  • No more different sets of desktops between X11 and Wayland
  • Better experience when using USB-C based docks

UI wise on the Systemsettings module nothing will change in the most common case, when you have one or two screens, where you will be presented with the usual “primary” checkbox.

When you have 3 or more screens, instead of s simple “Primary” checkbox, you will be presented with a dialog where you can rearrange the screens in a logical order, so that you can say: “This is my screen number one, this is my screen number two etc.”

Reordering screen priorities

This will map exactly to what “screen number one, two etc” are for the Plasma shell, so that whatever desktops, wallpapers, widgets and panels are on the “screen number one” will always be on the “screen number one”, never disappearing again, same for each screen, giving a completely predictable, and stable multiscreen setup.

For 5.27 I worked mostly on the Plasmashell part, but many thanks and shouts to Ivan for his work on KScreen and to Xaver for his work on the KWin and Wayland protocol parts.

Long story: the moving parts

I would like to do a semi-technical but still quite high level description on how our multiscreen infrastructure works, and what did we change.

When you have multiple monitors connected, assuming the kernel and drivers will do the right thing (which… doesn’t always happen), either X11 or Wayland (so in our case the KWin Wayland compositor) will see the screens and have info about them we can enumerate and manage.

KScreen

In Plasma we have a daemon/library to manage the screens, called KScreen. It will save a configuration for each combination of screens it encountered. each screen is identified uniquely by its EDID, and when this fails for the above reasons, the connector name the screen is connected to (under X or libdrm they will have names like HDMI-A-1, DP-2, eDp1 and so forth). connector names are also not optimal, but is the best that can be done in case of a bad monitor (For 5.27 Xaver fixed a quite big issue on this fallback path which likely caused several of the reported bugs).

The user then can chose with the Systemsetting module how the screens are arranged, their resolutions, which is the primary and what not. When a screen is connected or disconnected, KScreen will search for an existing configuration for this set of screens and restore that one.

Plasma

The Plasma shell also has to be aware of multiple screens arrangement, because you want the desktops with their wallpapers, widgets and icons to be on the screen you expect it to, as well all the panels to be always on the “proper” screen you expect it to. And that’s where we had a lot of problems in the past, and various approaches have been attempted.

Current

Currently, up to 5.26 Plasma relied on two concepts for working out which screen a desktop and panel should be. Combining a single primary monitor and using screen connectors names for disambiguate second and third screens. In theory …

There are a couple of things that can go wrong with this: between X11 and Wayland, connector names tend to be different (for instance HDMI-1 vs HDMI-A-1 on my system).

Another problem is related to the the USB-C based docks that have video outputs (such as the Steamdeck one, but is common to any other model and brand). They dynamically create and delete the outputs and connectors when they are plugged and unplugged, but the connector name is not stable between different plugs/unplugs: sometimes after re-plugging the old names are not recycled, but increased (so for instance if those were DP-2 and DP-3, they may become all the sudden DP-4 and DP-5). Since we used to store those names, we have a problem: each time a new connector name is encountered is treated as a completely new thing, giving its new empty default desktop and no panels, causing many wtf moments.

Plasma Containments (technical name of the desktops and panels) always had a screen property associated to them which is just an integer: 0 for the primary screen and then 1,2,etc for the others.

5.27

The new approach builds on a single design, extending the primary monitor concept to multiple monitors, which cleans up the codepaths even for the two monitor case.

So we now dropped this association map between screen number integers and connector names, and we go with this ordered list all the way down, so now Plasma, KWin and KScreen agree exactly who “Screen 1” is.

We have a new Wayland protocol for screen ordering (as well an X11 counterpart) for KScreen and KWin to agree to which one is the real logical ordering of the screens, so Plasmashell will now follow this without attempting to identify monitors by itself, removing a very big failure point.

As said, When your system has only 2 screens, in the Systemsettings module you’ll see exactly the UI it had: just a checkbox to select which one is the “Primary”, so no surprises here. When you have 3 screens or more then you are presented with a dialog where you can visually reorder the screens, to decide which is the first, which is the second etc.

In this way, you’ll always be sure that whichever screen was set as “first” will have your “main” set of panels and desktop widgets, that you will never lose, also whichever you set as “second” will always have the desktop and panels you set it and never change and so on.

KWin and tiling

Software

Personally I haven’t ever been a big user of tiling windowmanagers such as i3, awesome and what not, is not much the workflow style I want 24/7 out of my desktop, but there is definitely something something to say about that kind of multitasking when it makes sense, when is important to see the status of multiple windows at once for some particular task.

Plasma’s KWin has since a long time a basic support for tiles via the quick tiling support, by either dragging a window at edges or corners, or via keyboard shortcuts. This feature is very good, but very basic, and while there are 3rd party tiling extensions such as Bismuth which is a very nice thing, but window geometry managing outside the core always can bring you only so far.

Over the last month I have been working to expand a bit the basic tiling capabilities, both the quick tiling with the current behavior and a more advanced UI and mechanism which lets the user to have a custom tiling layout. Here it is a very short screencast about it.

tiling manager effect

Tiling is done by a full screen editor done as a fullscreen effect where is possible to add /remove/resize tiles. When moving a window if the Shift modifier is pressed, the window will be dropped in the nearest tile. Resizing the tile in the editor will resize all the windows connected, and on the converse, resizing tiled windows will resize the tiles as well. (also the traditional split screen quick tile will gain this feature, so if you have 2 windows tiled to get half of the screen each, resizing one will resize both)

Another use case we thought about for this tiling feature is to address people which have those fancy new ultra wide monitors.

Random stock photo from the interwebs of an ultra wide screen

There are very few applications which have an UI that actually make sense maximized on such proportions (i could imagine perhaps KDEnlive, Krita and very few other productivity apps). Most applications just look wrong, but with custom tiling areas, those can become sub “maximization” zones.

It is not intended to be a full fledged replacement for I3 or Bismuth, but rather an hopefully robust mechanism in the core with a pretty minimal and unobtrusive default ui, then the mechanism will have scripting bindings which should allow for 3rd party extensions to use it and implement interesting variations of the tiling WM out of it.

For now the feature is quite basic, a notable thing missing which i still hope to get in time for 5.27 is having separate tiling layouts per virtual desktop and activity (for now they’re per-phisical screen), but hopefully the full thing should land on a Plasma 5.27 release near you.

Behind Plasma Bigscreen

BlaBlaSoftware

Plasma has been designed from the get go (2006 or so.. it seems at least 2 eternities ago 🙂 ) to not make any assumptions on the type of device and to do a clear separation between the core technology/runtime and the various GUI plugins that end up implementing a full desktop experience.

In an architecture decision informed by previous prototypes we did in KDE4 times for mobile devices UIs, in Plasma 5 we split it further and introduced the concept of a “shell package” which lets further customization between devices than what Plasma in KDE4 times allowed.

Because of that we could do the Plasma Mobile shell without changes to the architecture that runs both the Desktop shell and the mobile version, despite being a completely different UI.

While working on different shells such as Plasma mobile and the Mycroft voice assistant (more on that later) we noticed that the best possible help for building a shell for a new type of device from the ground up is a very minimal shell that doesn’t make many assumptions about the final gui, but just provides few building blocks that will be in the end customized by the developer.

Enter Plasma Nano. Plasma Nano is a minimal shell not intended for end users which the developer will extend to match the final desired user experience (think about subclassing in object oriented languages)

The normal television set is becoming more and more from just a “dumb monitor” to a full featured computer system. But for several factors its UI must be completely different from both a desktop system, and from a mobile system.

Those so called 10 foot user interfaces have some particular constraints. The screen will always be seen from pretty far (altough you can’t be sure exactly the distance the users will look at it) so every graphical element must be very clear and big enough to be seen from a distance (indeed ~10 feet / 3 meters or so) so very big text, clear and spaced (too high information density should be avoided)

Every control should be accessible with the least number possible of buttons. It should be easily controllable with just the remote control, and in particular with just arrow keys, and “ok” button and one to go back. Often tv remote controls have a plethora of buttons, tough which makes the user experience only more confusing.

Another important way of interaction recently in smart tvs to go around the lack of complex input methods is voice control. It is ideal there as the commands to the TV would be very basic like “watch this series on Netflix” and things like that. This of course is giving birth to a slew of very legitimate privacy concerns, but this technology is too good and convenient to throw it away because the current implementations are done by less than honest corporate giants.

Mycroft is an open source project which is aiming to build a fully opensource voice-based personal assistant with a growing number of skills.

Understanding voice is in two independent parts: speech to text (going from the sound file of the recorded voice to a normal string of letters) and actual semantic understanding of the sentence to then translate it to a concrete action (fetching weather data, a Youtube video and so on). The latter part is the bulk of the work for a personal assistant and is what Mycroft implements. it can then use an external service for speech to text.

It can use a variety of services, from Google (speech to text, not full assistant) to a different number of free and proprietary services. In the end, they want to use the Mozilla Deepspeech project, which would make the stack 100% free. You can already configure it to use deep speech or other engines, even if not fully ready yet (want to help out? Mozilla is looking for voice samples to make their product better and ready for mass consumption!)

A voice assistant looks better with a GUI part as well, as the echo show and google assistant demonstrate.

Some of us worked together with the Mycroft people to produce an extensive set of QML bindings for the Mycroft system, (that will provide as wellthe user interface for future Mycroft based smart speakers). They’re a third-party QML module that can be freely used by any Qt-based applciation for voice integration.

Plasma Bigscreen can optionally use those QML bindings to provide parts of the QML UI of the user experience (so yes you can ask via voice to your TV if it’s going to rain tomorrow or to watch the music video of so-and-so on YouTube)

Those QML bindings can provide from Mycroft a variety of GUI features, from simple notifications like a clock if you ask what time it is, to a full featured interactive app, like we did for the Youtube skill, which is a Youtube browser app which can be used both via the voice only, the remote control only, or a combination of both and provides a footprint for future rich voice-interactive user interfaces.

Plasma Bigscreen

Software

Today I want to introduce a project I have been working on together (mostly in the background) with some colleagues of mine… Now with beta status reached, it’s time to more publicly talk about it: enter Plasma Bigscreen.

Smart TVs are becoming more and more complete computers, but unfortunately there the experience tends to be a tight walled garden between proprietary platform, services and privacy-infringing features. Features which are very cool, like voice control, but in order to not pose a threat to the user privacy should be on a free software stack and depending less on proprietary cloud platforms where possible.

Plasma BigScreen is a platform intended to use on smart TVs (trough a powerful enough small computing platform, such as the Raspberry Pi4, or any tiny computer if you need more power) with big remote-friendly UI controls, and Voice activation. What technology did we use for it? Plasma (of course!) and Mycroft.

Plasma Bigscreen main user interface

Plasma Bigscreen is an user interface for TVs providing a 10 foot UI for a smart TV experience (on TVs or any screen with an HDMI connection). It is a Plasma Shell, just like our beloved Plasma Desktop and Plasma Mobile.

It uses the familiar look and feel of Plasma but optimizes all the user interaction to be perfectly readable from the distance and controllable with ha simple basic remote control… from your couch.

Besides interaction with a remote control, it also supports a modern way of iteracting that is revolutionizing the UX on all kind of devices: voice interaction. Optionally, it can integrate seamlessy with Mycroft: an opensource project aimed to offer a completely free voice assistant.

Mycroft offers what are the so-called “skills”. Each skill, takes care of a particular voice interaction. there is an huge variety available, from the weather, do date/time and reminders, to online services clients such as soundcloud and Youtube, with comprehensive QML bindings to show a rich User Interface in a Plasma Environment.

We prepared a beta image for the Raspberry Pi4 for anyone to try, in which we pre-configured Plasma Bigscreen with Mycroft integration ready to go.

If you have a Raspberry Pi4, you can try this image immediately: you can find it here: just flash the file on a MicroSD and is ready to go on any Pi4.

Note that on that image, we point the device to the official Mycroft company’s “Mycroft Home” service, which internally uses the Google STT (Speech to text) which does use google but in a more anonymized fashion as this is not tied to your Google account (nor requires having one). Tough not ideal, it’s possible to configure the Mycroft core to use different services, even those you can self-host like Mozilla Deepspeech.

In here to emphasize voice controls, we prepared some skills (some preinstalled, some can come from the kde store) to fully show voice controls. In particular a Youtube client skill which is perfectly usable both from a remote control and voice only.

Youtube skill shipped in the image

As remote control, you can use the remote of your TV directly if it supports the CEC standard, or any of those remotes with an usb dongle which are “seen” as a keyboard with just few standard keys (some have also an integrated microphone, so giving voice commands is very easy).

In the next days, there will be more posts explaining the architecture and the technical choices we made, in the mean time, have a try and if you want to get in contact with us about the project, you can find directions here.

Fosdem and Plasma Mobile Sprint

BlaBlaSoftware

From January 31st to February 8th I went on a little tour, first at the two days of Fosdem in Brussels, then to Berlin for a KDE sprint about Plasma Mobile.

It was the first time i went to Fodem: it’s an awesome experience, even tough big and messy: which is the awesome of it… and the bad of it at the same time 🙂

Even tough there were 800 talks I didn’t attend that many, some about the Elixir language, some about retrocomputing, some about iot stuff. At Fosdem the best thing to do there.. is meeting a lot of interesting people, rather than attending talks, which are very interesting never the less, which you can find videos here.

I stayed most of the time at the KDE booth. in there we had as usual some hardware to show: besides the usual occasional “normal” laptops running Plasma, we had a PineBook pro running Plasma on Manjaro which runs impressively well for such a low resource machine and was in fact a favorite among the visitors.

Yep, there were a lot of people.

Besides that we had some Pinephones running Plasma Mobile, which they were very popular as well. The Plasma Mobile shell was very stable compared to last year fosdem which made me quite happy.

Looking people using it for the first time, was also a very precious feedback on what are the UI problems new users can encounter.

Feedback that turned out to be very useful the next week, when together other people working on Plasma Mobile and two perople from the UBports project few to Berlin for a Plasma Mobile sprint, graciously hosted by KDAB in their spaces.

There will be many points we will be able to collaborate with UBPorts, starting from background technology such as their telephony services, the content sharing infrastructure, and maybe push notifications. Tough in the end, we want their apps running smoothly on Plasma Mobile and our apps running smoothly on UBports as well, to have as many apps as possible.

Personally, the areas that i worked more at the sprint were the Plasma Mobile homescreen and fixes in Kirigami that were needed for some plasma mobile apps to look better.

On the Plasma Mobile homescreen, now dragging plasmoid thumbnails from the widget explorer to the homescreen works properly (fix in KWin, also drag and drop icons in your desktop wayland session should be fine now)

Plasma Mobile, Clean homescreen look

In order to make the homescreen look a bit more “modern” and less busy, I also removed most of the gray background rectangles that were there and in general simplified the homescreen code a bit, here is the result:

In general, it seems to me that Plasma Mobile is actually really start coming together 🙂