I already extensively talked on this blog about the new QML declarative AppletScript that will be present in the upcoming 4.6 release of the KDE Platform and how is important especially in the light of the QtComponents project.
A little new feature got in some days ago: in technical terms, is a fallback chain for Plasma::Package
Wait, a what? wtfbbq?
Let’s see what this means with this video 😉 Here you see some further developments over the RSS reader we seen the last time, like a search as you type filter bar and a bookkeeping of read/unread feeds (done with Plasma::Storage, a projects of the last Google summer of code).
But wait, there are 3 rss readers shown in that video!
The other two, shown as standalone windows, they have a very similar and coherent behavior compared to the one on the desktop, but they look completely different and have some important differences in their behavior.
They are a version targeted to MeeGo Handheld and MeeGo Tablet (the one with a two column layout, that is possible in a larger screen)
Those two version, use a (very early and under heavy development) version of QtComponents for MeeGo, that in turn uses the MeeGo touch framework theming system to have applications with the same look and feel of native MeeGo touch apps, but with a QML declared interface.
What is really neat is that those are not 3 plasmoids: it’s only one, in a package that weighs less than 80Kb 😉
Also, the proper version is automatically chosen at startup by the Plasma package system (at the moment depends from an environment variable) in the code you won’t have to put explicit switches to discover what platform you’re on. You just have to provide the files for the proper platform in the proper directory, and they will be automatically chosen respecting the specified fallback chain.
Once this is in “full steam”, we will be able to have a set of plasmoids that are:
Traditional desktop widgets
more reduced widgets for the Plasma Mobile workspace
“Full apps” on standard MeeGo handset and tablet
“Full apps” on a different mobile system, for instance one completely based on Plasma Mobile.
How this can be done?
Plasmoids are distributed in packages, with a certain filesystem structure. In the javascript code you have, when asking for a certain resource, that can be an image, a javascript file, a QML file, you just ask for a particular include, you just have to ask for an “image” called “foo.svgz” for instance (plasmoid.file(“images”, “foo.svgz”) without having to worry about the actual path.
Now, (actually depending of environment variables) the actual path of the files depend from the device used, so you can decide for instance, that when you are working in a mobile environment to replace one of the package files with other one.
Files are searched from the more device specific to the more generic ones. In the example of the video there is the chain: MeeGo tablet (or handheld) -> “MeeGo generic” -> “completely generic” (where the Plasma-desktop case is treated as the most generic one right now since is the most common).
They can be qml files, images, ui files, svgs, whatever.
if the replacement is done carefully, you will be able to have even completely different user interfaces (in this example, MeeGo handheld and MeeGo tablet) but sharing the logic of the program, that can be Javascript, if the application is simple enough, or C++ modules if needed.
The image on the left is a new (example) plasmoid that marks an important change… this is going to be a quite massive blog post, with pretty technical details about what I think is going to be a feature of an higly important strategic relevance on the road towards Plasma as a mature and powerful platform.
The Plasma declarative bindings have landed into kdebase! This means it will be possible to use the new QML language part on Qt 4.7.
As Aaron noted some days ago, QML in conjunction with now in heavy development technologies like QtComponents (and in the future Qt cene-graph) will play a key role in the creation of user interfaces in the future, and gives us a very important puzzle piece we were still sorta missing.
So we deceided to be (as usual 😉 erly adopters of the technology and with the KDE Plasma workspace 4.6 the first pieces will be in place for the roadmap that will lead us basically to this path:
Separate the implementation of the logic from the user interface
Avoid C++ (and well, any imperative language) as much as possible for the UI
Be flexible, an application should be able to jump between different devices as easily as possible
do applications that can chose the proper ui amond different choiced depending on the device used, the screen size, the input methods and what not
With QtComponents, use what will be the native set of widgets/components for the given platform: on a MeeGo phone, use its own widgets, theme and UI paradigms, but a different interface somewhere else
Performance, perforance, performance, enter Qt scene graph
So, what will have the KDE Plasma platform 4.6 of all of this?
Use QML from C++ Plasma widgets
The first pass (needed to do the following one) is to have a way to easily load a QML file into a normal C++ plasmoid. Now, loading a QML file into a QGraphicsScene (and making it well behave inside the QGraphicsLayout based Plasma widgets) is a quite simple operation, however it is a bit long and repetitive amount of boilerplate code… enter Plasma::DeclarativeWidget.
This is a normal QGraphicsWidget just like the other Plasma widgets, all you have to do is to add it in the QGraphicsLayout of the applet, set the path of a QML file in it et voil
Have been quiet on the Plasma mobile land lately but this doesn’t mean there weren’t developments on that, as this small video, on the usual two ugly pieces of hardware, shows two quite foundamental those days features that recently got in.
Screen rotation, with a nice animation: so now screen rotation is supported both on devices that will rotate the screen in a “traditional” way, by actually rotating the screen resolution, and devices that will rely on the application painting itself rotated (such as MeeGo) in this case such kind of animation becomes possible.
Our on screen keyboard now has also a “compact” mode for mobile devices, you can see it there reduced on the smaller device and full on the bigger one.
More importantly, the keyboad knows when showing itself: when the user taps any editable text area.
Here it is also a smaller video that shows most of the things I’ve talked about in those months. (the nice musical background is by Nuno Povoa, made for the KDE4 launch)
When designing Plasma Mobile, it was immediately clear that wouldn’t have been possible to do a “one design fits all” application: mobile devices vould have come in pretty diffrent forms:
Different resolution
Different phisical size
That implies, different DPI
Different use cases: an internet tablet and a phone put the emphasis on very different primary functions
What we have seen right now, is the development of completely different code, from the ground up for different platforms.
In Plasma we always tried to avoid this, by having everything as a plugin, so it will be necessary to replace maybe the shell itself and just the components that really have to be changed.
With Qt 4.7 a new framework ha been introduced: the declarative UI, that permits to do quite fancy stuff in the QML language in a very short time.
So with the development of Plasma mobile we started an experiment, and here it is (usual VESA drivers disclaimer about the speed that applies to all screencasts done on that weird tablet):
Besides bindings to write plasmoids entirely in QML (or using QML bits into C++ plasmoids), the behaviour of the entire shell, how activities appear/disappear/are chosen is completely controlled by this language, so in theory should be enough to change the QML files set to achieve a completely different ui and behaviour.
In theory. But it gets interesting when this becomes actually practice: with just some adaptions to the C++ code of the shell to make it more flexible, last days I wrote a different QML set and a different default layout configuration aimed for bigger, but still touchscreen based tablets.
Those kind of devices always were for me one of the main targets of the netbook “Newspaper” containment, that’s why a quite big amount of attention has been put into the behaviour of “flicking” arouund things in it.
Here the pages concept is taken to the extreme: you can phisically change pace with a swipe of the finger, dragging the new one into the screen, as you would change real pages 😉
The search and launch interface is dragged in a similar way, but from the bottom, since it should always be quicly available with a single gesture, no matter on what page you are.
This is just a quick proof of concet, but that shows itself very promising for the future.
Bit after bit all the pieces come together: what’s impressing is how much the Plasma framework we have built into years is paying now.
Plasma Mobile still lacked a widget explorer: the particular formfactor really required a custom one since the one used in the desktop has a pretty different usage pattern and is targeted at a really different form factor (here we are targetting -really- small screen sizes).
I was expecting that writing one could take a quite long amount of time, after all getting there the firt time taken a fair amount of work…
I couldn’t have been more wrong. In less than two days I have now a working widget explorer, that works pretty well on small touch screens, using a mix of QML, Plasma Widgets and C++ models
The main interface is a big flicking icon grid, with a side panel that shows detailed informations on the widget (that can be scrolled with the finger as well).
To add a new widget into the screen, just click at the “+” icon positioned where the widget will be. Interaction wise is way simpler than the Plasma Desktop mechanism, because of both precision of the input device (where with device I mean a fat finger :p) and of screen size (where with size i mean actual phisical size, not pixel resolution)
This little video shows it in action on the usual device. ah, and a curiosity, at some point you see the screen switches to an activity with a big numeric keypad.. that’s not a mockup, as Artur announced during Akademy, if it runs on a N900 phone, that can do actual phone calls.
Sometimes we have to clone ourselves 5 or 6 times to manage to do everything. It’s a kind of magic we rarely succeed, but sometimes we do 🙂
Between Akademy and bugfixes for the SC 4.5 release, I managed to get some work done on the Plasma Mobile front, and a quick video of it can be seen there:
Now there is a new kind of activity for managing widgets there, a new addition to the Desktop (that you normally use on normal computers) and the newspaper (designed for netbooks and big tablets)
This one, that is called mobile desktop, is explicitly tought for very small screens (in phisical size, pixel resolution has nothing to do with size!). It’s based on a big flicking scroll widget, just as the newspaper, because the main method of interaction if a touchscreen, but shows only 2 widgets per screen, (at their minimal size to still make sense)
If you scroll everything around no interaction with the widgets will happen, but if you tap one, it will get maximized, getting a size big enough to be really interacted.
The behaviour of the shell changed a bit as well, since it was quite difficult to hit empty areas to make the whole thing flip, now dragging the panel of the activities, pulls the whole screen, making everything rotate in a 3d way (it has also considerably been optimized for speed)
In the video you can also see the progress of the mobile status indicator summer of code, by Yuen.
Memory is starting to be fuzzy already, so, since i promised a blog about the same topics on my talk about the netbook, here it is, mostly because the netbook project (but more important its implications in helping Plasma being ported and be optimized for a long spectrum of devices) has really soft spot in my hearth.
We started from the most familiar non-desktop platform: The netbooks, but…
Aren’t netbooks just underpowered little laptops? Indeed today are sold like that, with just still XP or Windows 7 preinstalled, but that’s the reason the interest in them is decreased recently. they are just not good general purpose machines.
But, they are pretty good as devices that can be used on the move to look at online content such as news, emails or social networks, and ultimately as a communication tool, rather that a content production workstation.
KDE SC and Plasma:why?
We still hear again and again that KDE is to heavy and too bloated to run on any modest hardware. Of course technically the situation can be improved and it will, for instance the platform profiles that are being cooked right now will be able to provide a law fat (as in Kevin words 🙂 you will find information about that in the upcoming future on the planet.
n the other hand, complaints are often not completely true, we need better communication about what the advantages of a KDE based solution are, and where the problems are: we pushes the edges of what all the layers of our platform can do, Being Qt, X, or graphics drivers, due to our hard beating the quality of the whole stack is really increasing (and this funningly enough is benefiting non KDE users as well).
About Plasma, it’s the primary UI on desktop, can be on completely different devices too?
Yes, because the Plasma library was designed with no assumptions on a particular form factor in mind. We started from netbooks to taste the waters, because was the less alien one and because we can provide a rich and complete user experience from day one.
What do you want?
Why Plasma can be good on a netbook, on a mediacenter, on a mobile phone?
because it can give people what they want, both to users that actually get to use it and developers, that want to create the most wonderful application in the world with it.
Users
So what users want? they want something useful. Now “useful” can mean different things in different form factors. On a netbook, and even more so on tablets and smartphones, is to be efficient instruments of communication and content fetching, perfect web clients, better than even just a browser.
They want to feel in control: the day for instance the facebook interface will change without them being able to choose they will get pissed, with rich clients there is always a slightly bigger degree of choice.
They want also something beautiful: between two systems that “just work” they will choose the one they find more aesthetically pleasing, even if it’s not technically the best, it’s irrational but it’s the human nature.
Developers
They want a powerful framework to build powerful applications as quickly as possible, Plasma offers a series of common components that makes the life of developers as easy as possible.
Plasma is heavily based on a strong separation between data and the visualization.
Everything in plasma is a plugin the data plugins are Data
Everything you see in Plasma is based on Svg, it’s theme engine makes easy to customize a potentially resolution independent widget set while maintaining pixel perfection.
Its widget set is based on QGraphicsView, like the other Qt widget sets conceived for the mobile, and is possible to integrate and use it together with them.
We have also a flick able scroll widget as well a flick able QtWebkit based one, that is very important, since the future on more and more devices is the touchscreen.
Plasmoids (or plasma widget, more generally), everything you see in a Plasma environment is one of those plugins. they can be specific for a certain formfactor, like kickoff or the taskbar, or they can be quite flexible, like the microblogging plasmoid.
They are done to work well in the desktop, panel, the netbook newspaper, as standalone little windows invoked from krunner (new from 4.5).
So, what about the netbook again?
First thing:window management: the desktop metaphor doesn’t work very well there…
KWin behaves slightly differently there, windows are maximized by default, maximize windows don’t have borders, and the panel is auto hide by default, so quite a lot of pixels are saved for the app
The boundary between primary UI and applications becomes blurred when some of the content can be fetched directly from the desktop, so it becomes a normal window like an application.
Search and Launch
After some usability tests done at Indt last year, what emerged is that Search is more intuitive than browsing trough many items in categories and subcategories. Search and launch uses all the search plugins of KRunner, even if is a completely different UI.
There is still a menu, but it’s flat, doesn’t have tree structure, that doesn’t work well on the human brain.
Of course uses flicking widgets and has an extensive support of drag and drop, that makes quite convenient to use it a mouse or even a stylus.
Netbook: Newspaper
Also the main widget view for the netbook is based on flicking widgets, this gives an infinite vertical space for our widgets, because the free layout of a desktop isn’t space-efficient enough for a small screen, so a layout that can be “browsed” like exactly an actual newspaper, feels more natural there.
Future
What the future reserves us? We need more widgets, especially PIM related, to make the “newspaper” really useful, a GSoc project is helping in this.
Also, the dataengines needs to cache their data for when the network connection is not available, there is a GSoc project working on this too.
Last two points, we need to expand on other devices, because I think netbooks won’t stay the way they are today, they will probably not converge with tablets, but for sure be strongly influenced by them, or become something even different.
Last but not least, the newspaper concept spawned some unintended at first but really interesting developments: two GSoc projects around Kontact and Skrooge, as well some work on KDevelop to provide a “dashboard” of useful widgets in standard desktop applications.
Sometimes when times are ready some ideas really want to come up and be used
Without calling the examples in the history of science, that is quite a bit exagerated here, in the recent history of computing, at all levels from low level stuff to the UI.
this happened over and over again, because of advancement in hardware that makes something possible, because a certain fraework matured enough, because some projects take inspiration from another one, maybe with changes that may improve over the original one or may get it worse (that’s all about the evolution in nature is btw) or some times they just pop out in the same moment even if one doesn’t really know about each others work.
As a result, the evolution follows very similar steps on several quite different platform.
I’ve tought about it quite a lot when i seen this video:
(no, don’t have other formats since that video isn’t mine)
This is a preview video of the user interface intended for the MeeGo distribution to run on tablet devices. The main “primary” UI shows two “pages”, one is a grid of icons with an optional search field that has been seen in several places already, from the very very old mobile devices such as the first Palms to things like The iPhone or the Plasma Netbook Search and Launch interface (where our search functionality is quite more powerful than just searching applications to launch)
The second interesting “page” is a space where is possible to have widgets that connects to webservices, list your media files, PIM data etc.. does this ring a bell?
This is quite similar to the netbook “Newspaper” ui, there is quite different, instead of being two columns that scrolls vertically, it’s one single row that scrolls horizontally, because when you have only touch as input, it’s indeed a way to use it quite more convenient.
Now, what’s interesting, that in prevision of adapting pieces of the netbook ui on devices like tablets, there is an hidden mode in the netbook ui that permits exactly this behaviour. In not suspect times (November 2009) i did shot a screencast that shown also that feature among other things (go to 2:39 to see it):
Yes, it’s quite funny to see this video now, since the project came quite a long way in the meantime and so it works way more smootly, but it shows the idea already basically working.
In KDE SC 4.5 the newspaper ui as well the other components received quite some love, so now the general usage feels much more smooth, even this hidden just-a-little-experimental feature of the newspaper, so for today I’m leaving you with a brand new video of this kind of interface as is in today’s trunk (and SC 4.5) running on the usual low powered VIA tablet:
Probably I should had written this little notice uhm, a quite while :), anyways, it’s quite technical but here there are a little hint on how to make the Netbook shell a garzillion times faster, if you’re hit by an unfortunate series of circumstances that makes it run sloooowly 🙂
Why on some netbooks it runs slow
As shown here, The Plasma Netbook shell can be quite fast on a standard netbook, however many are experiencing it going rather slow. Why?
Qt can use different graphics backends to render the graphics, now on X11 systems the default is the one that is called surprisingly enough, X11, that relies heavily on XLib and XRender. Now, this sounds allgreat, but two things tends to happen.
If the Xrender acceleration of the video drivers is pretty poor, things tend to slow down.
Second, if QPainter hits some code patch that aren’t directly implemented via the X11 graphics system it will have to fall back to another paint system: raster, that is totally software based, so pixmap-to-image conversion (and memory copy) is involved (and is necessary, to not actually lose features depending on what graphics system you’re on), and things slow down.
Now, with Intel drivers things seem to not go soo well, especially for items clipping: that’s unfortunately what’s used more heavily in Plasma Netbook 🙂 the newspaper and the search and launch result works as follows: a smaller QGraphicsWidget contains a bigger one, that contains the actual contents that could be the plasmoids of the newspaper for instance. the outer widget has a particular flag that enables clipping, so what overflows from the parent widget won’t be actually drawn.
with Intel drivers and the X11 graphics system this seems to be pretty fast until the parent widget is of a certain size, then performance degrades significantly pretty quickly.
The actual how to
So the alternative is to use the raster graphicssystem from the start, that is really fast (especially if one think about that is completely software based rendering) there are basically 3 ways to do it:
the application can be invoked with the graphicssystem commanline switch, so:
plasma-netbook -graphicssystem raster
But this has to be done every time, or Qt can be configured at build time to use this as default (some distributions such as Gentoo or MeeGo actually do this) but will have the effect on every Qt application.
So I’ve added a config file option in Plasma Netbook to use it as default even if Qt has X11 as default: with Plasma Netbook not running, edit the file
~/.kde/share/config/plasma-netbookrc (in some distributions could be ~/.kde4 instead)
And add the following line:
GraphicsSystem=raster
That’s it 🙂
Here is a video that compares before and after on the usual EeePC 1005ha, with a bright note: the difference isn’t anymore so noticeable like it was used to some months ago, even tough it’s still makes the difference.
Two reasons: it’s still quite unstable sometimes, overall with X11 Plasma (and other Kde apps as well) seems to (still) be more stable.
But most importat, as said it’s purely software based, so it could be not so gentle on the CPU (so with the battery life). I mean, it’s impressive how much it’s efficient, but will never belike a working gpu based solution. So if you aren’t running on the problem described before, it’s probably better you keep it as is.
So there is hope for the future?
Absolutely! a still experimental new graphics system is in the works, using this time OpenGL. The whole application (any Qt one actually) can be run completely with OpenGl drawing with the -graphicssystem opengl, or just a QGraphicsView can use it (by using a QGLWidget as viewport). That’s exactly what happens if you start Plasma Netbook with
plasma-netbook --opengl
This works in the new born Plasma Mobile too by the way. and is a great hope especially in those two sheels, since GPUs of small devices are designed to be not soo fast, but to be very low power, a bit less than the cpu, actually.
Right now, as in KDE SC 4.5 and Qt 4.6/4.7 is still not for everyday use, is still not so stable and there are some graphical glithces (this actually varies from a graphics driver/video card model like crazy) but what is encouraging is that since this feature was introduced (Qt 4.4) it came a very long way, it’s really sooo better than when it was originally out.
The image on the left is a new (example) plasmoid that marks an important change… this is going to be a quite massive blog post, with pretty technical details about what I think is going to be a feature of an higly important strategic relevance on the road towards Plasma as a mature and powerful platform.
