Author Archives: Marco Martin

Lazy loading QML

Software

What is one of the most important things while writing a QML UI? the answer is pretty easy: KISS.

There are several reasons: the first is of course a reason that is pretty valid in any UI in general: the more an interface is simple, the less visual elements there are, the less redundancy, the more elegant and easy it looks (attention: it does not mean less features, it means just better design
in their representation)

And second, of course the more objects are on the screen, the more memory is taken for their representation, sometimes a non negligible quantity.

Another thing that is very important is to actually load your interface only for the things that you are actually showing right now on the screen, anything that is hidden, or can be shown just eventually, should be instantiated only just before actually showing, otherwise you are paying in startup time and memory for some objects that may even never be actually shown to the user.

A typical example is a tabbar with maybe 10 pages, if those pages aren’t performing an important background task (like loading an html page) why bother loading them?

I’ve just added a new very simple QML component in org.kde.plasma.extras: ConditionalLoader.

It works pretty much like a standard loader: you specify a source component and it instances it, but just when a certain condition is satisfied, so for instance:

 import QtQuick 1.1
 import org.kde.plasma.components 0.1 as PlasmaComponents
 import org.kde.plasma.extras 0.1 as PlasmaExtras

 Item {
     PlasmaComponents.TabBar {
        PlasmaComponents.TabButton {
             text: "foo"
             tab: fooTab
        }
        PlasmaComponents.TabButton {
             text: "bar"
             tab: barTab
        }
        PlasmaComponents.TabButton {
             text: "baz"
             tab: bazTab
        }
    }

    PlasmaComponents.TabGroup {
        id: tabGroup
        PlasmaExtras.ConditionalLoader {
            id: fooTab
            when: tabGroup.currentTab == fooTab
            //source can be a path name as well
            source: Qt.createComponent("Foo.qml")
        }
        PlasmaExtras.ConditionalLoader {
            id: barTab
            when: tabGroup.currentTab == barTab
            source: Qt.createComponent("Bar.qml")
        }
        PlasmaExtras.ConditionalLoader {
            id: bazTab
            when: tabGroup.currentTab == bazTab
            source: Qt.createComponent("Baz.qml")
        }
    }
 }

In this example (simplified, without anchors and whatnot) the content of the tabs is in separated files, and they will get loaded only when the tab will become the current.

It may be used also to do other stuff, like in PopupApplets to load some things only when the popup gets open for the first time, or in delegates of listviews to load extra ui parts when the user clicks on an item, or whatever many other uses.

Active three

Software

Today we have a new release of Plasma Active: quite some time passed since the last release last year.

OGG version

Applications

Plasma Active 3 has several new applications out of the box, to reinforce the philosophy of “useful by default”.

  • Add Ons: an add ons store, where both the client and the server side are free software: right now are available a collection of wallpapers and all the thousands of ebooks from Gutemberg. Different kinds of content like more books and apps will come shortly.
  • Advanced file management: in mobile devices often the files are walled behind the single applications or cloud services: while is fine to have also that possibility, you should be more in control of your files. Plasma Active by default comes with an advanced file manager that lets you manage your files with timelines, tags and other advanced semantic features (thanks, Nepomuk 😉
  • Alarms: not much to say here, it was a must for a mobile device 😉
  • News reader: a convenient touch interface to browse your favourite rss feeds.

Incremental improvements

Compared to the Plasma Active Two release running on the same device, there is a significant improvement in stability, boot time, apps startup time and rendering performance.

Base OS

A quite visible one is that the official release is now based on a different OS: welcome to Mer.

Mer continues the effort of MeeGo, but in an more open, community-oriented way. It’s a lightweight Linux based OS with a high accent in offering the possibility to build customized final distributions, with adaptation for specific hardwares or specific end user software.

Many still remember what pain it was to set up correctly the old scratchbox based build environment for maemo, (or pain common for many embedded development environment for what matters) In Mer setting up a development environment is matter of probably 10 minutes or so.

The high customizability of the base OS of Plasma Active also means that is ideal to build highly customized verticals for the enterprise environment.

Plasma++ at Randa

BlaBla

Pretty busy days here at Randa. On the Plasma front some important decisions have been taken and a lot of work is now being done in this direction.

There is now a big traction in rewriting our current plasmoids in QML, such as the notifications I talked about some days ago, and many more that are coming.

The plan for the future is to have 100% QML-based workspaces, with as little changes as possible feature-wise compared to the current one, just with a smoother feeling 😉

This is only the first part of the road, and this is arriving already today, with each release of the Desktop workspace more complete, while Active is already 100% QML from day one.

What will happen next? Together with Qt5, there will also be the release of the next major revision of QML, that expands the language and changes the technology behind. Among the other things, everything will be on an opengl scene and the JavaScript bits will be managed by V8.

This comes at a price: only pieces of the user interface that are written in QML can be loaded in an environment based off QML2. hat means the old code based on QGraphicsView (and part of the old C++ API) has to go.

This wasn’t an easy decision, but it will make things dramatically easier in the future.

  • On QML2, we’ll have a lot more performance, guaranteeing a way smoother experience (with QML2 you can get 60fps in 1080p on an humble raspberry pi after all)
  • More stable, because most user facing code (especially most 3rd party) becomes scripted
  • Easier to write plasmoids: lowers the barrier for contributions and lowers the barrier in experimenting new innovative approaches for the UX
  • Also the internal library will be way easier to approach for new developers, because it will be way smaller and simpler
  • It will also be simpler to take and use the Plasma library for new things, like in application’s dashboards, or just using its QML components.

This is a long term plan: The work on the new Plasma library version started now, and we expect a working shell based on the new framewok and Qt5 in about 18 months or so.

I’ve just got a first preliminary version of libplasma building without QGraphicsView dependencies, and the binary is about two thirds of the size it had before, so this is an encouraging start 😉

It’s again an exciting time when we can design a new architecture, have crazy ideas, trying them out, and any input or help is very appreciated.

You can see the progress of the tasks that are added as the work proceeds here, and talk or claim some tasks for yourself at the usual channels of communication, plasma-devel@kde.org or the #plasma freenode IRC channel.

Notifications, let the redesign begin

Software

So, it seems it’s that time of the year again… the plasmoid used in KDE Plasma Desktop to display notifications and the progress of transfer jobs started to really show its age, due to some bad limitations in the old QGraphicsview code to handle complex layouts, so it appeared quite buggy and not so smooth to use. Yes, I’m aware that sometimes it started to dance resizing itself several times in a row, and there wasn’t much to be done on that regard… Until now 😀

The fact that there is some research/development being made to build a new backend for notifications that will support many new features, more “modern” to be actually useful with the applications that are so heavily “communication” oriented(both desktop clients and web stuff), that became essential part of out workflow.

The story begins more than a year ago: we needed a way to display notifications on Plasma Active, and obviously the desktop applet used back then wasn’t enough.

Since we would have to rewrite it in QML anyways, we started it, at the beginning as a thing used only in Plasma Active.

Here below is the status that will be shipped together Plasma Active 3, in about 2 weeks (more on that later 😉

WebM version

Then, as it became more and more feature complete, it was obvious that it could have replaced the one used in the desktop quite easily.

But wait, we have different input methods so we need two completely different things otherwise one won’t be usable on the other platform, right?

Wrong 😉 what we need is a different UI, and not even dramatically different: we need it adapted for a different input method and yet still familiar, the actual code difference needs to be very little.

In QML plasmoids, we can specify some files that will be used only in some platforms, so the system will automatically pick the most suited one. Also the basic component used (buttons, scrollbars etc) while having exactly the same API on all platforms, they can have a radically different behavior (or even, a completely different implementation).

Here is the plasmoid that is going to be merged and shipped with our next iteration of the desktop:

OGG version

While it looks very similar, there are some important differences:

  • Different theme/look (that was easy;)
  • The desktop version has scrollbars, the touch version works by flicking and has scroll indicators that automatically appear/disappear
  • The touch version has way bigger hit targets
  • The desktop version has mouse over effects, the touch version doesn’t
  • The text in the notifications can be selected with the mouse in the desktop version, and there is a right mouse button menu that allows to select/copy to clipboard
  • Instead, with an horizontal drag on a notification on the touch version, it will be moved, and if “thrown” outside the window, it will be deleted.

Difference in code? a couple 100-something lines QML files that are chosen either one or the other, that’s it.

The desktop version has more or less the same functionality as the old version shipped with 4.9, the changes are mostly about making the ui a bit simpler.

Also, this new notifications plasmoid is all about the user interface. It has been designed to be as easy as possible to change (or add a new one) the underlying notification system, so it will fit pretty well in the new system that is being researched right now.

Note about QML plasmoids and dataengines

BlaBla

There is quite a lot of work going on in plasma land at the moment.

For PlasmaWorkspace 4.10 you may expect an overhauled visual appearance and an overhauled user interface on some plasmoids.

The two weather plasmoids and the pager were rewritten in QML by Luis as part of his gsoc, while others are in the works.

The Notifications and jobs plasmoid is in the works, solving many behavioural bugs that were a limit of the previous architecture (more on that later;)

A Kickoff replacement is on the works as well, and if everything goes allright another important one will arrive for 4.10.

This is made possible by the contribution of the people, I’m seeing a renewed interest in working on plasmoids, also due the new architecture that significantly decrease the learning curve.

But, there is always a but 😉

There is a small problem I seen in some converted QML plasmoids: sometimes the needed data arrives from a Plasma dataengine. there is a nice system to map that data directly to a model usable in a QML ListView, Repeater and so forth, DataModel from org.kde.plasma.core

This is a (strong) simplification of the device notifier plasmoid:

PlasmaCore.DataSource {
    id: sdSource
    engine: "soliddevice"
    //hpSource is a datasource connected to the hotplug engine
    connectedSources: hpSource.sources
}

ListView {
    id: notifierDialog

    model: PlasmaCore.DataModel {
            dataSource: sdSource
    }
    delegate: PlasmaComponents.Label {
        text: i18nc("@info:status Free disk space", "%1 free", model["Free Space Text"])
    }
}

The keys of the dataengine are directly mapped to the ListView model, so directly accessible to the delegates via the “model” variable, or directly as their own variables.

However, due to a limitation in QML, if the dataengine doesn’t immediately do a setData() of a particular key, or a key is not always available, it won’t be mapped as a role in the delegates, if you encounter this roadblock, feear not, you can still access the data with something that in the above example would map sdSource.data[“DataengineSource”][“Free Space Text”]

When possible tough use the model mechanism, since some nice optimizations are planned for it 😉

One of the things I wish for plasma2 is a formal specification of dataengine available data in a way that would avoid situations like that and most important nice documentation may be extracted from 😉

Click here to lend your support to: KDE Randa Meetings and make a donation at www.pledgie.com !

The Randa platform meeting

BlaBla

Last year, a bunch of KDE hackers gathered together in the little lovely swiss village of Randa in a meeting that was called Platform 11 (actually the third KDE meeting held in Randa).

That quite large meeting turned out to have been one of the most productive ever. Developers from all areas of KDE, from libraries and platforms targeted to other developers to applications (built with those libraries) targeted to end users.

A lot of design and development work happened on the KDE Frameworks 5 happened. This is crucial for the future of the KDE applications and workspaces, because besides porting to the soon to be released Qt5, it will make the KDE libraries much more modular, making easier for any application developer to just cherry pick between the features offered by the the KDE frameworks and use them on any platform without significant dependencies, being Linux, Windows, Mac, Android…

You can help this meeting to happen, as usual the development is done by volunteers, but there are some fixed expenses, like food and travel. There is a fundraising campaign for this on Pledgie, of which we are already at a good point, so yeas, you can make the diffeernce 🙂

Click here to lend your support to: KDE Randa Meetings and make a donation at www.pledgie.com !

Workspace sprint

BlaBla

Last week I was among many other KDE fellow hackers at the Workspace sprint in Pineda de Mar, It was a really awesome location (and company, of course;).

But is not all fun and play: what we have produced while there? What it’s a bit different from other times, is that it wasn’t a coding sprint, but rather was a very needed moment of taking a step back and reflect about the future.

We are in a transition period from a technological point of view: Qt5 is coming and the KDE frameworks effort is helping to redefine the whole developer story around our tecnology… but this is another story 🙂

This is the ideal moment to see where we are now in term of what we offer to the user, and how the whole workspace is perceievd as a whole. The first important thing is indeed exactly realizing and knowing where we are now, having a big picture from different sides.

At this point we can have a defined and realistic vision on where the workspace should head in the next years, and how to get there.

To get to this comprehension on where we are and were we want to go we are starting to use several classic tecniques of design, such as personas (hello Carla and Raj) and role games that help to identify problems (speed boat) and current or future features (Product tree), of which Sebastian and Kevin already explained it in length.

Looking at the problems of the boats and the Product tree, among the other things two stick out: Activities are a feature that really stands out of the pack: nobody else has it, but in order to be really a game changer has to be expanded a lot and be way more “natural” to use.

Also, workflows that spread across multiple devices are becoming more and more common (they are very important for Carla and Raj); our device spectrum effort is the right thing to do, but has to be more integrated in the core development process of everything we do as a community, and has to look more integrated (communication and marketing goals).

Other wiki pages and documentation on what has been going on those days will come up over the next few days: we’ll keep you posted.

Artificial boundaries

BlaBla

This is the 4th and last part of a single post (1, 2, 3)

For all the things in our life, we constructed boundaries that sometimes are completely artificial with not much actual sense in the reality of things, presenting more and more false dilemmas

This series of posts is of course about user interfaces, and this last post is about two common beliefs about applications and UIs that I think are false distinctions (and particularly care of)

Workspace? Application?

The boundary between the workspace and the applications is somewhat artificial, as is the boundary between one application or the other. Is there a common function that most applications share?

If so whether it appears in the application window, in the workspace or as part of another application is just a matter of what are usability findings for that action in particular (see Share Like Connect in Plasma Active).

How much two applications have to be considered separated if they share most of their components? or are just two tools of the same procedure? the knife and the spoon of the dish application?

In the end how much corresponds the separation I see in windows? is what i see as different tasks in the taskbar actually a different task? We should think more about different semantic task rather than different process/window that is an implementation detail.

Everything is just part of the same tool, the same system.

But is system == the machine? No!

Desktop/phone/tablet/TV…

Even distinction between devices and type of devices is artificial: my work spreads trough multiple machines, some more capable than others.

I won’t be able to do the most complex data creation operations on a phone, but I can view that work and create something, so on a mobile device I don’t want a completely alien set of tools, and i want to be able to access the very same resources.

  • I want everything as much synchronized as possible
  • I don’t want to set up the very same mail accounts on every device
  • If I have a movie on my desktop I want it to be available on my tablet
  • And if I press “play” on my tablet I want to see it on my TV
  • And i want zero effort to do that, not having to play around with settings and shared folders for hours.

I also want as much as possible of the very same visual language shared between all my devices. There are differences in input methods and screen sizes (in few years differences between pixel densities will be nullified), but most of the aestetics, a part of the behavior and most of the applications should be shared between all the devices, even if with more faces.

I don’t want to hear anymore KDE mobile effort versus the desktop: they are the same system. They are an unity, a spectrum that scales smoothly without really distinct interruptions.

Devices used to be very different from one to another, but in the next years differences will be less and less, in 1-2 years probably 100% of the sold computers, desktop or not will have a touch screen, we will see as big X86 tablets as very small ARM laptops. the input method and the mobile-ness won’t be a boolean anymore, they will be a real, from the big desktop(that is definitely not going away) to the small phone there will be a million of in-between thinghies.

And this is a false dichotomy dying right now, a distinction that existed for technical reasons (small mobile processors not having enough power) is fading away, we better adapt or die as well.

We are seeing different approaches in software to be ready to this paradigm shift in hardware; both Windows 8 and OSX are trying to “mix” desktop and mobile paradigms. Here the problem (especially visible in Windows 8) is that the transition from different devices is too abrupt, you end up with an ui designed for a different kind of input method from what you are using (thik about Metro start screen on a 24″ monitor).

So:

  • One size does not fit all.
  • There aren’t anymore well defined boundaries for different devices.
  • The UI should be like lego, every little piece should be interchangeable to do a million different combinations: small tablet, large one, laptop with touchscreen, small laptop with a mouse, tablet dockable to a mouse/keyboard station, whatever we don’t imagine yet…

To do a synopsis of this last post of the series, a phone and a desktop are the same system.

There are the Bob’s and the John’s systems, not the tablet system and the laptop system, the only difference between two systems is belonging to different people, being tailored around different lives.

To conclude, by the way here is the complete version of the mockup that srarted this long brain dump:

Fictional Desktop?

Overdesign

BlaBla

This is the part 3 of 4.

As the popular saying goes, there is always an easy solution to every human problem: neat, plausible, easy to understand and wrong.

It’s now widely accepted that the graphical and behavioral presentation of an interface should be have some conformity to what we evolved to expect (as i talked about in the previous post: transitions, rounded shapes possibly textured, direct manipulation of objects are just some aspects of it), however this brought a very easy answer that introduces a whole new range of problems: skeuomorphism.

A skeuomorph design tries to mimic as much as possible real objects that are an analog version of the tool implemented by the interface, a typical example is an address book application that looks like a leather address book or an ebook catalog application that looks like an actual library.

Address book

This is the design direction Apple taken since a while, and now from mobile is starting to percolate in the desktop as well (and being Apple is influencing a whole lot of developers now).

This approach has several problems:

  • It kills consistency: the boundary between different applications becomes extremely evident, too evident, condemning to remain forever in the application based paradigm, while “application” is a technical detail, is not much really something that should be very evident in the user facing semantics. Every application has a completely different graphical language, even tough it was designed to ease the transfer of learning from the “real” world it ultimately hinders the transfer between learned part of the system.
  • Imposes artificial limits: by copying a real object you copy also its limitations. To stay in the example of the book browser that looks like a library, the mapping to the real object suddenly drops (and thus it starts to look unfamiliar, magical) when you perform functions like sorting or searching books. You can’t certainly make magically disappear from the library all books that aren’t from a certain author, with a snap of the fingers. This makes quite hard to create innovations, like in this case a more powerful intuitive way to browse books that leverages indexing and metadata extraction.
  • It’s uncanny: the application will mimic the real object, but not perfectly: it will always feel fake, there always will be something out of place (even if just because of the features it offers that are impossible in the real world just as a simple search), creating a cognitive dissonance: yes I am looking to something that looks like a leather address book, but I can’t use exactly like that, i have to use a completely different set of skills.
  • It’s expensive: last but not least, it’s extremely expensive and labor intensive to redo from scratch every single pixel of every application: not everybody that are not Apple can afford to deliver a complex product with quality good enough to be presentable. If the cost of this enormous amount of work was justified by a big benefit could be worth it, but as we seen causes more problems than what it solves.

Finding the balance

The debate between an hyper-realistic, application based design, and a more classical UI approach is very polarizing lately, with good arguments from both sides.

Problem is the detractors of skeuomorph UIs, as gets natural in every polarizing debate, advocate from stuff that looks like it comes out from some 80’s science fiction movie, with Windows 8 Metro or Android 4 Holo as examples (especially in the Android case, the similarity with Tron is quite again, uncanny)

As I said, I think Skeuomorphism is the easy and wrong answer to the need of interfaces that feels more natural (where natural doesn’t mean there isn’t need to learn it), easily learnable to make the machine being a desktop or a phone an extension of your arm rather than a weird machine that you have to dialog in a strange arcane dead language with.

There should be a natural looking language, natural looking (or even reality copying) materials to build the UI elements, with a correct lighting, but yet not trying to copy real objects in the end, what you have to construct is a new machine that looks realistic but yet doesn’t copy a library or an address book.

You want something that looks and behaves more “analog” than “digital” (even if is something quite hard to quantify) for the same reason the piano replaced the harpsichord very quickly.

The UI must “flow”: as square edges and shapes with spikes should be avoided, the movement of everything should be as smooth as well, nothing that just appears or disappears, everything that is obvious where it comes from.

Remote controls should be avoided, everytime you have to use a UI to configure another UI, rather than directly manipulating it, you are doing something wrong.

In one sentence, design a visual language that is new, consistent, rigurous, looks natural and stick to it.
A button that looks like a button, with the correct lighting, drop shadows to tell the brain what is the most important and secondary things are ok, replicating a fully functional rotary phone is not.
Also, small breaks from the visual grammar can sometimes enhance the value of a particular feature, but only if used extremely rarely.

Next and last, boundaries between applications and workspace, boundaries between devices (or, why they don’t exists)

Design and psychology of user interfaces

BlaBla

This is the part 2 of 4.

Some times an application, or its user interface, seem to exist more for themselves rather than around the purpose of being a tool to accomplish a task: it’s very important that always stays the first goal of the existence of any functionality the software offers, being an application, a particular screen of that application, or a piece of the shell.

  • The star of the user interface is the content, this has to always remain the center, what distracts from the content is something that is often not necessary and can be avoided.
  • You interact with objects represented on the screen with a particular input system, may be a mouse, a trackpad or the touch screen. Interaction should be direct: in the real world you interact with an object not with a remote control that remote controls the object. Objects represented on the screen are often quite abstract, from objects to “pure information”, so is not always easy avoiding levels of indirection, but they should be limited (one of the reasons you will see handles in Plasma and not things like text boxes that ask how many pixels you want your panel high).
  • Our mind is trained to recognize patterns, that’s why consistency in UI is extremely important: this is something that comes from our evolution: for our survival we have to memorize and then recognize anything new that we encounter, may be a menace or an opportunity. Once you learn what a snake is, when you see a very different type of snake you just run;). Same thing as learning to use a tool, and transferring this knowledge to the use of different and perhaps more advanced tools, and this is what interest us here. (see transfer of learning)
  • Organic user interfaces: same reason as the above point, our mind expect things that we have seen the past some million years: some things are hardwired in our brains: we expect that when something appears it comes moving from somewhere. Something appearing out of the blue without having being noticed it was coming it’s seen as a potential menace, if you just find yourself with a big spider in front of you all the sudden without having it seen coning from somewhere, is possibly even more terrifying.
    As important as UI element coming in from somewhere is “natural” shapes: even just using rounded corners when possible instead of perfect edges may do quite a lot of difference. There are two probable reasons here: a shape with sharp edges focuses the attention on the outside of the shape, where the edges are pointing, while a shape with rounded corners focuses the attention inside the shape. Moreover, in nature things with sharp edges again are a menace.
  • Finally, a good interface should be invisible. What? Again, the sole purpose of the interface is being a tool designed to do a particular task. Everything that is “more” that the strict use case of the particular UI hinders its learnability and efficience. Often most of an application UI can be seen as “Content”, “Document” that still is a UI artefact, but is a quite direct representation of what you are viewing or working on, with chrome as everything else, that is often necessary, maybe a necessary evil, but evil still.

Those are all concepts that gained quite a lot of traction over the last few years, and UI quality all around, from Windows to KDE to web apps improved a lot, most applications and environments we see around followed a very clear design procedure, but… there is a but 😉

There is also an ugly side about it.. in the next post.