Speeding up startup for Hotfix 7!
While work for the next major revision of the firmware progresses (and if you'd like to lend a hand, be sure to speak up!), we're always still working on updates and changes in the current firmware, from actual code changes to just updating and patching existing applications. One piece of the system that many of you see is Minimenu -- the simple more console-like menu option we include alongside the Xfce desktop environment. (The system is designed to support multiple environments but of course NAND space is limited so we don't just through a dozen in there, as tempting as it is :) I recently made a forum post discussing some of the rationale for changes to Minimenu (which I call 'mmenu' for short) and thought it seemed a pretty good little topic of discussion.. so here I'm cleaning it up and expanding upon it a bit in hopes you find it interesting.
Without going into the pre-original-shipping history, suffice to say that Minimenu burst into existance rather quickly; we wanted a nice simple menu that was fast to operate, fast to start, really to the point with no fluff and extras.... and timeline was tight. We weren't really sure what direction things would take down the road (hindsight is great, but at the start of the journey you just hedge your bets and roll with it, right?) We were still designing the firmware and figuring out where we thought things would go, so weren't sure if we'd always have X11 (the graphics system Unix and Linux usually use) up all the time or not - drivers were still in flux, the PND idea and system was in flux, we were trying to guess what use cases people would want, and so on.
Anyway, so we tasked Minimenu with working with and without X, with being PND-file oriented so that it could run with or without the rest of the system and play nice with all our design goals, with being fast and configurable under the hood but 'just working' for the user. Do all that, and do it in a few days.. great! All told it turned out not too bad -- not flashy and swooshy which would've been fun, but we can leave that for teams who have the benefit of time, and seeing where the device is/has moved, etc. You don't usually look into the configuration, but it is reasonably configurable for certain critical points of behaviour, and its lightly skinnable. Some of the easier and more sensibly user definable options are presented in the UI itself if you look, but the ugly nasty stuff is buried in the textfile config files. There's a little documentation there for each line item, but if you want some additional discussion.. just ask; could always write up another article for another Inside the Box, or throw something up on the wiki!
So if you want to configure mmenu to be even lighter and leaner -- or even build a super tiny little firmware around it, you can.. that was some of the original goals. I have quite some time ago fiddled with just a kernel that boots into mmenu, and includes copies of all the shared libs the normal firmware has (to give applications a chance to load), and almost nothing else. No desktop or other apps or even X11, just Minimenu set up to scan for pnd applications and let you run them. Startup time measured in a few seconds, but an ugly hack. Maybe someday someone will pick up this idea again... :)
Recent mmenu startup changes
Aaaaanyway, so historically mmenu can run without anything else.. which means it had to do 'app discovery' for pnd files all on its own (using our standard 'libpnd' library), well and good, and that can take a few seconds as you see; but in practice we ended up building the firmwares always with X11 up and always with the rest of the pnd system fully active (pndnotifyd being the main daemon I'm referring to here.) 'pndnotifyd' is a daemon that sits quietly in background watching for the SD cards to be inserted or ejected, and takes care of mundane things like looking for applications, noticing threy've been added or removed, spitting out the icon artwork and .desktop files the various Unix systems can use. A .desktop file is a desktop standard for publishing applications (and other bits) to desktop environments, such as Xfce (or others, should you install them.) A nice little standard so any desktop you install (such as Enlightenment) will 'just work.' Since you can switch for mmenu to Xfce (or others) and back, we enabled pndnotifyd all the time.
This means of course that while mmenu is coming up in its turn, and firing up its libpnd based pnd discovery, pndnotifyd is performing the very same task. Now, the filesystem is cached so they're not both hitting the SD and so on and one scan is faster than the other (and happening more or less at the same time), its not as big a performance hit as you might think, but its a trifle annoying nonetheless. We've often thought about fixing it up -- caching in libpnd itself so that scans are much much faster (no need to do full scans, when a lot of the info hasn't changed since previous bootups!), or having pndnotifyd use shared memory or some comjunication handoff to mmenu directly, or just using minimenu alone when in the mmenu oriented desktop option.. lots of ideas. Of course, we don't want to build cryptic custom interfaces between applications when we can use simple and obvious and reusable - especially for code written at weirds times of the night and all, so bloodyminded and easy is the way to go :) Things worked, other priorities came and went, but we've always wanted to revisit this little nit. (And more to come in the future .. baby steps dear friends, baby steps!)
While it took awhile to get to, I've built in options so now mmenu can be configured to run either or both scans -- pnd scan, or .desktop scan and trust someone else to maintain the .desktops. I've not put in caching to libpnd itself yet (I've fiddled around and sometimes it can be a good speedup, but not always.. and there can be some risks, depending what 'keys' used to determine hits/misses across ejects/inserts of SDs) but you can still configure Minimenu how you like -- a pnd oriented solution or the .desktop solution.
So we've switched Minimenu to leverage .desktops for Hotfix 7, rather than performing its own full pnd discovery. Given in pracice mmenu is usually run with the rest of the system in place (certainly in all our firmwares to date), this is a reasonable assumption. But the code for both is in there.. if someone wants to build another firmware and have minimenu do a pnd scan, no problem, flip a config value and voila!
Now on boot, pndnotifyd does its app discovery nice and early as normal and spits out .desktop files (and icons); mmenu watches for pndnotifyd to finish up (you may have notied in early Hotfix7 alphas where Minimenu could pop up with most icons or even apps missing, due to mmenu coming up before pndnotifyd was finished :), and then picks up those .desktop files. That reduces boot load (one scan instead of two), and parsing the .desktop files is easier than parsing the pnd files .. so mmenu is general much sped up at end to end start time (for Minimenu desktop users. Xfce users won't notice a thing.) The minus side of course is now mmenu has to wait for pndnotifyd to finish up, but its pretty quick about its job, and overall things are still faster than minimenu doing the second scan on its own at the same time.
Also, I experimented with some timing tricks; UIs are funny since a lot of it is about 'perceived speed' versus 'real speeds', which factors design and where buttons are and the easier way to hold the device and all that .. and 'speed during use' which may be not what you expect; so until recently mmenu was loading all icons into memory cache at menu start up time, to try and keep 'usage speed high' with a hit to startup time, figuring that is what most people would prefer; all this behavious is configurable but few will fiddle with such settings. Problem here is that Minimenu wants to minimize its system impact, so it exits when you run a pnd -- so minimenu (unlike xfce) uses _almost zero_ system resources when a pnd is running; minimenu exits, runs the pnd, then comes back up. Nice! But as you can guess, this means Minimenu has been loading up all those icons to memomry every time you come back. It runs fast, paging and browsing is quick as intended.. but you have to load up all those icons, many of which you'll never see that usage session (unless you're hitting All all the time, I suppose .. which I do :)
So, here I got to thinking ... and ED is always thinking about user experience and pointed me in the right way :)
If you dig into mmenu's configs (which you don't want to ;), you can fiddle with a lot of stuff. Some stuff is exposed in the config UI but the real nasty fiddly bits are not shown there.. they're just in textfiles for serious spelunkers to edit. Anyway, you can turn on background loading of preview pics (if you use the detail side panel at all), or background app scanning, or background icon loading, etc. You can tweak the skins, fiddle with category naming, set up additional tabs for directory browsing, lots of crazy stuff I can talk about in another article. Turns out, since we're essentially single core, that doing things in background just chugs things up in user experience feel -- your main thread gets slower, and your behind thread isn't going fast, so cocks up the main thread for longer than it should -- Annoying :) (At least, doing threading in a simple brute force manner.. doing it very very carefully can works well, but thats work ;) Anyway, most background stuff is turned off, as it just didn't work out to be all that useful so early on in Minimenus life. Shelved, and a year or more goes by....
Now, with preview pics in the right side detail panel, if you want them are on demand loaded when you idle the selection -- lets you move the selection around fast, and only load preview pics when you pause/idle on an item (for a configurable amount of time..) - you idle and then mmenu decides to load the preview and makes you wait. (And loading previews is slow due to the whole 'mount', 'read pic', 'unmount' nature of fetching files from within a possibly compressed pnd, until we revise the pnd file layout a bit, sometime down the road. Anyway, thats a whole other discussion :)
For icons its always had the ability to background load icons, but it wasn't really efficient about it since it didn't seem to work over well for user experience, so I didn't work over that code much; recently I thought I'd bang on it some more and introduced a bunch of configurable options .. when to do background icon loading, how fast to do it, which ones to do -- visible apps only, go ahead so paging is fast if you're idling, or to scan all, or seek ahead for where you might be going next or what.. all that stuff. Loading icons is very fast, but I always liked the look of and the function of, background loading -- show you the generic icon then pop up the icons as they get loaded, while you're working. To my brain, it seemed you'd want to 'get to business' fast, not wait around for artwork -- maybe you just want to hit 'B' or 'Start' and run the selected app, who cares if the icon is loaded? So I banged around with a lot of options .. faster background icon loading slows down the main thread, so moving the selection feels clunkier; but it can load each image fast, just as long as it lets you move the selection around fast. The usual software balancing act -- how far to go and still feel good to the user. But in the end, it just didn't feel 'right' to do background icon loading, even if just background loading 'next page' and not too many icons, etc and so on. ED really didn't like the idea of showing generic icon and popping up random icons as they were retrieved. A little sluggish and not really professional.
Meanwhile, ED kept on with a suggestion that I misinterpreted, but when I 'got it' through my thick skull and built it out.. and it is what we're going with now. Sometimes someone will beat you around the head and you'll bloodywell try that idea last, and when it turns out to be a good idea, you've just got to grin and move on, right? :) Its all configurable of course, but right now its set to.... when it needs to show icons, make you 'wait' and just load them right away in foreground, then show the screen. Don't background anything, but also do not preload anything; when icons are needed, go get them right bloodywell then.. not earlier, not later, right when you want them. The reason being.. loading icons, even a whole page full, takes no time at all. But doing them _all_ in advance, that takes a bit of time; doing them in background, that cocks up everything or takes awhile. So just grab a 'bunch' when neededed, and turns out thats not too bad at all; a blink of time.
So now, you load up minimenu and it scans .desktop files instead of pnd files (faster) and doesn't compete with pndnotifyd for it (faster) and then jumps to the configured start page - be it All, or some tab/category, or previously selected app or whatever (perceived faster); then when it goes to render the page, it loads up the icons it needs right then and there (from cache if its already seen them of course, for free.) -- faster than before.
Sure turns out this is a pretty good startup improvement! At menu startup it picks up the app list fast, and then on demand loads icons fast; no extra work, nice and trim. Thats the minimenu philosophy.. and now instead of maybe 10 seconds of delay (say), you might get 2. Its not a big payoff for just afew apps, but some crazy people (again, ED!) like to suck down the entire Repo onto their pandora, and the payback is huge. Lots of apps, shaving off time per app and only loading whats visible.. huge.
Someday I'll revisit caching the whole app discovery and speed p that too, but we'll see how it goes. Got only limited time, right? :)
A quick note on usage speedups..
Then for user experience speedup, theres stuff like .. when you go into a subcat folder (such as Roleplaying under Games) and back up again, where does the selection land? First icon in the parent category? The last selected one (the subcat folder itself)? When you run a pnd app or shutdown and come back to the menu.. do we jump to your default startup tab, or go back to the app you had selected? "Simple" changes like these can make your general usage experience much more efficient, and 'feel snappier.' Sure, I didn't code swooshy fancy opengl graphics in with sliding panels .. I didn't have time when I designed this menu, and I wanted trim and sleek and fast. But we do try to make the device fun and fast to use :)
Much more to come.. lots of improvements and tweaks, and behavioural changes in where the selection goes when, and search filtering the app listing based on keys entered, and more efficient file management, and so on. Possible improved network management right in the menu, maybe even an option to check the Repo for updates to your apps .. lots to do; but wheres the line between simple, and getting 'too big' for a minimal menu?
Minimenu config files
The main operational config is /etc/pandora/conf/mmenu.conf in the device NAND.
A trick (since libpnd is pretty clever and will look in multiple places for config files..) is to put a copy of mmenu.conf into /pandora/mmenu/mmenu.conf on one of your SDs; this way if you pooch the config file and can't start minimenu any more, just eject your SD and reboor.
Alternatively, you could change to Xfce while hacking mmenu config files, permanent or temporarily; mmenu can be launched from the Terminal by invoking "mmenu" (which won't run pnds, then.) For full operation, invoke "mmwrapper -fmmenu" and then it'll work just like normal, right from within Xfce (say)!
A fast way to kill Minimenu is to enter ESCape, and it'll just die. This instruct it both to die, and not to restart again as normaly for mmwrapper behaviour. (ie: in case of crash, or after a pnd application completes, mmwrapper will bring mmenu back up; thats what the 'mmwrapper -fmmenu' line above tells it to do.)
Do keep copies of your working mmenu.conf ;)
A blow by blow discussion of every line in the conf file (and how skinning works, and other conf files) is out of scope for this article, but let me know if you'd like more details!
Enjoy, and keep hacking!
-- Jeff (skeezix)