Boutique modes

I’ve recently started a new job at Linaro which has been keeping me very busy. My role combines the low-level fun of Dynamic Binary Translation that I so enjoyed at Transitive and the fact Linaro is a fully Open Source company. I work directly on the upstream projects (in this case QEMU) and with the project community. In many ways it’s my ideal job.

Of course I’ve quickly built up a reputation as the Emacs guy in the office surrounded by a sea of VIMers although one of the guys does profess a desire to learn Emacs “one day”.

One of the first things I did was move all my email into Emacs. I’d long been dissatisfied with the state of Thunderbird (and previously Evolution) that I took the opportunity to migrate to mu4e. I spend my days slinging patches and going through mailing lists so I really appreciate being in my editor while I do that.

I have also started streamlining some of my work-flow with a few specialised extensions to Emacs. I think I’m finally comfortable enough with elisp to have a pretty good stab at solving most problems. I’m also appreciating the ability to leverage the mass of Emacs code under the hood to make incremental tweaks rather than solve everything at once. I thought I might give you a tour of the code I’ve written so far.

First up is risu-mode. RISU is the code testing tool we’ve been using to verify our aarch64 ARM work in QEMU. The .risu file is a template that’s used to specify instruction patterns that the tool then uses to generate random code sequences with. risu-mode is really just a bunch of regex expressions wrapped in the mode machinery that highlights the elements on the page. It doesn’t sound like much but when you are working through a bunch of patterns looking for bugs it’s easier on the eye when the different elements are coloured.

Next thing I wrote was my own QEMU mode which is a simple comint-mode based mode for launching QEMU system emulation. It’s still very rough and ready as I’m mostly working on user emulation but I suspect it will be handy once I start on system emulation stuff.

Finally there is lava-mode. LAVA is Linaro’s automated test and validation framework. Although it already provides command line and web interfaces I thought it would be nice to launch and track test jobs from within Emacs itself. The job control files a JSON based so I built on the existing json-mode and a slightly patched version of xml-rpc.el to add job submission. I’ve started a simple job tracking mode that uses the tabulated-list-mode framework and eventually I’ll link it into the tracking library so job completion will be as seamless as my IRC work-flow.

So there you have it, a bunch of code that may well not interest anyone else but shows how Emacs provides a very rich base of functionality on which to build up tools that are useful to you.

Does anyone else want to share an example of their esoteric extensions? What’s the most obscure thing you’ve built on top of our favourite text editor?

Edit with Emacs v1.13 now available

I’ve just pushed the latest version of Edit with Emacs to the Chrome App Store. Hopefully most people are already tracking the latest edit-server.el via MELPA but this does introduce a few minor fixes to the extension itself. A new piece of functionality is the ability to trigger bringing Emacs to the foreground from a key-stroke within Chrome. I added this to support running Emacs on ChromeOS which together with my chromebooks.el package gives me a rather nice development environment without having to dump ChromeOS.

So new for v1.13


* Change the handling of hidden elements (fix bug #78)
* Add debugging for erroneous hidden text areas (#93)
* Add keyboard shortcut to bring Emacs to foreground
* Pass clipboard contents to foreground request


* add advice to save-buffers-kill-emacs to avoid prompting on shutdown
* add autoload cookies
* fix bug with format chars in url (#80)
* don’t call kill buffer hooks twice (#92)
* don’t set-buffer-multibyte on process buffer
* support the “foreground” request with optional clipboard contents

Get the latest from the Chrome Webstore.

Travelling with the truffler

We’re are about halfway through our family holiday to the remote ends of the earth. It has been the first time we’ve taken Ursula on a plane so we thought we’d make it a big journey while we are at it.

To her credit she was mostly fine with the 24 hours on a plane required to get to the other side of the world. Most of the tears were during take-off and landing when it was hard to explain pressurisation to a 16 month old child. There were a few other snatches of complaint due to tiredness but otherwise it went well. It helps that she is a very cute child who instantly won over the cabin crew who were keen to help keeping her amused. She even had a freshly prepared meal of Salmon Fried Rice cooked for her by the First Class cabin crew.

Before we left the UK I had left Ursula playing in the kitchen while I sorted something out in the living room. When I came back into the suspiciously quiet kitchen I found the following example of toddler OCD:

OCD much?

An interesting aspect of having psychologist for a mother-in-law is the wonderful insight she gives me on how the mind works. It had only been few weeks earlier that we had been talking about a common behaviour that often precedes a spurt in language development. It seems as children start getting their heads around the concept of things belonging to categories they will start sorting their toys (or anything else) into organised piles. Obviously an understanding of the fact things can exist in categories is a prerequisite for understanding a lot of things about language.

I make the distinction between language and speech because the two are very different skills. Language is primarily a cognitive ability to map communicated ideas to abstract concepts. Speech is the vocalisation of that communication and involves fairly precise control of a dizzying array of muscles in our mouth and vocal chords. The mastery of this physical skill takes a lot longer so often the distinction between words is only recognisable to parents and others who spend a lot of time with the child.

Ursula has been understanding basic instructions for some time now and it’s now possible to send her off to fetch or carry things with a reasonable degree of success. Combined with her deistic pointing there has been genuine two way communication for some time. However perhaps due to fluke or stimulated by the new environment she’s in on holiday we are starting to see an explosion in words. She’s had the basic “Dadadada” and “Momamama” for sometime although it’s hard to distinguish from the baby babbling she’s been doing for a long time. We have long joked about her generic use of “dat/cat” for the cat and then pretty much any other object she was pointing at. Just before we left she had started associating “NaNa” with bananas (a favourite food of hers). We now have distinct sounds for birds, cats, dogs and my favourite “papa” for the Nexus 7 which we call the PadPad so as to avoid confusing it with the Apple brand product 😉

My mum found my description of all this behaviour very amusing as I swing between proud Dad and scientific curiosity. I will put on the record that I’m not treating my daughter as a lab experiment but I do find the whole development of language and mind fascinating. I understand now why watching your kids grow and develop is so often cited by parents as one of the main joys of parenthood.

On the death of Google Reader

You can probably tell the sort of on-line company I keep from the deluge of noise on the social networks regarding Google’s decision to shut down Reader. However we shouldn’t be that surprised. In fact some companies that source content from Reader have anticipated the need to collect content themselves.

I of course will have to make a decision at some point. However I’ll not do it today like a lot of Reader users have. The rush to try out alternatives has over-whelmed some open source based projects who were quietly growing organically. I don’t envy those that have to suddenly gear up their back-end systems because an Internet behemoth gave us 2593 hours notice to sort out a replacement.

I’m mulling over the difference between self-hosting and having someone else do it. I’m not overly worried about going for convenience if I know I can get my data back if I need to. In fact the knowledge that you can theoretically self-host might be enough. To be fair to Google their Data Liberation team made exporting all my Reader data easy.

Before I make a choice I need to decide what my priorities are. Currently I subscribe to 250+ RSS feeds. Obviously I don’t read every single post but I make extensive use of tags to quickly process through stuff I do need to see when I need to see it. Aside from news, blog posts, funny cat pictures I also subscribe to other data feeds like bug trackers, code repositories, and other data sources. I of course want access to all of this data at any point on one of a number of devices. This makes a web hosted solution pretty much a must. There is no point having the data on my desktop when I’m somewhere else. From my point of view I want it to be open source compatible because if the company hosting now decides it no longer wants to I’ll only have to move the data and not break my work-flow.

It would also be very useful if it had a public API so others can interact with the data. I don’t need the solution to be all provided by one company. It’s perfectly fine to have multiple 3rd parties sorting out the Android integration. I might even look to doing something to integrate it with my favourite editor (the name of which even my non-geek readers probably know by now). So far my experiment with moving all of IRC and IM into Emacs seems to be working well and should be a subject of another post.

Are you a Reader user? What are your criteria for it’s eventual replacement? Is RSS just a dying protocol or is the need to aggregate and sift through data becoming more important?

There may well be a much better way of solving this problem around the corner. I certainly am open to persuasion. But don’t take away my current preferred solution until I’m convinced I’m ready to switch 😉

The future is inching its way onto our set-top boxes

I have from time to time lamented the tardiness of media producers in getting content in-front of my eyeballs. It’s not a case of being tight with money, although giving Murdoch £30/month for a glut of programming I’ll never watch is *too much* for me. I’ll happily pay for the stuff I want to watch as long as I can watch it now on whatever device I happen to have to hand. Until it’s made that easy for consumers the pirates will keep winning by default.

There was an interesting quote from the boss of Netflix in a recent interview: “The goal is to become HBO faster than HBO can become us”. Netflix is moving from being just a content distribution company to a producer of content. And if they can create content as good as their new remake of House of Cards hopefully the days of production companies being tied to restrictive bundle contracts are numbered.

The fact that shows can be disconnected from the broadcast schedules also offers other compelling opportunities. The BBC recently aired a number of comedy sketch show pilots which we only caught on iPlayer. Indeed they are now going to start trialling iPlayer net first runs of new shows over the next 12 months. No longer does the success of a show have to hope that it was picked up by enough of the 11500 viewers that happened to catch a random scheduled slot. Each and every viewing can be accounted for and counted even if the show is a slow burner like Firefly which was cruelly cancelled before it built up it’s cult following.

So the balls now in HBOs court. How exactly to I pay you so I can watch the next season of Game of Thrones when it comes out?

2012 a year in review

In past years when I would review the year I’d flip back through my blog. For 2012 however I managed on average one post a month and the majority of those were geek related ones which in some ways are a lot easier to write.

The big change has been becoming a parent of course but at the same time I also took back the reigns of management as I became the head of the software team at work. The combined effect has been a year that flew by very quickly leaving a slight sense of bewilderment about where it has all gone.

I no longer feel quite as out of my depth as I did half way through last year. The first 6 months are pretty gruelling in terms of missing sleep and trying to interpret the needs and desires of something that can basically sleep or cry. However Ursula has grown up so fast you soon forget about what she was like and try and re-focus on what’s she’s doing now.

For starters she’s now a fully mobile and operational battle-truffler able to move under her own direction. Next to that she’s moved from simply reporting state (happy/unhappy) to communication. Granted every object she sees currently seems to be a cat (or possibly “that”) but I suspect her vocal cords just need to catch up.

She’s now pointing to objects she wants to interact with including multiple objects she’d like to combine (e.g. Bonjella and teething ring) like some sort console RPG game. She laughs at silliness, enjoys playing around and being chased and basically has a personality that is uniquely hers.

I used to think that it wouldn’t get interesting until she learnt to talk but of course my preconceptions about what it would be like are constantly being shattered. Sure it still gets frustrating at times especially for Ursula. She understands more and more of what her parents are saying without being able to respond the same way. As she gradually realises she’s an independent being able to manipulate the world around her she’s very quickly having to learn that she can’t control everything. I suspect that would put teenage angst in perspective if only we could remember our really formative years.

Well this post seems to have turned into yet another one about my child growing up. Other things have happened to me that are not Ursula related but really there not as interesting. Instead I shall try and look forward to what’s coming up. The annual February walking holiday is just round the corner and I’m looking forward to catching up with all the other new (and slightly more experienced) parents at that. We’ll be up in Manchester (without Ursula) for P&K’s wedding which we are really looking forward to. And we’ll be off to the land down-under for a much delayed holiday in April. Hopefully the next post will not be too far away 😉

Edit with Emacs v1.12 now available

Apologies for the extended delay in getting out a new release of Edit with Emacs. Real life has been doing a good job keeping me busy that spare hacking time is in short supply. A number of people have submitted pull requests and I’ve also done a bit of clean-up on the code.

The JavaScript in the extension has had the hacky change tracking code replaced with a modern funky 21st century Mutation Observer implementation. This should solve some of the more obvious performance problems on highly dynamic pages.

I’ve also implemented the iterative save method for edit-server.el. Unfortunately it doesn’t quite work as intended if the edited region uses encoding (e.g. longlines). In theory the buffer you have saved should have it’s contents replaced with the new data from the browser but it just doesn’t seem to work. For now I’ve just ensured the kill-ring has the real data in it so it’s not overly painful to restore formatting. Patches and/or explanations are of course welcome!

So grab it now from webstore. The changes are as follows:


* Upgraded manifest to version 2 for future releases of Chrome
* Added an “Enable Debug” flag to control logging to console
* Use Mutation Observers to better track changes in dynamic DOMs
* Fake a keypress when updating the textarea
* Bug fix for handling editable DIV elements


* Now available from the MELPA package archive!
* Beta support for iterative C-x C-s saving (it works but is ugly, see kill-ring)
* Default to using UTF-8 coding for process communication
* Raise existing frame consistently on Mac
* Tweak the portion of the URL used to name buffers

Many ways to skin a GNU

It’s time for another examination of Emacs LISP and a selection of ways to solve the same problem. In this case it’s the behaviour of the etags-select package. This handy function will present a selection table when it finds multiple definitions of the same tag. This is handy if your global TAGs file contains reference to multiple binaries that might have a common code heritage. It even provides a handy function etags-select-find-tag-at-point which will check point to see if that is a tag. However if point is on a blank line I’d prefer it to just prompt me for a tag name.

First version:

(defun my-naive-find-tag ()
  "Find at point or fall back"
  (unless (etags-select-find-tag-at-point)

Unfortunately this fails rather badly. I naively assumed etags-select-find-tag-at-point would return ‘nil on failure. Instead it bombs out with an error because etags-select-find expects a parameter and when find-tag-default fails it errors out.

Second version:

(defun my-working-find-tag()
  "Find tag and call etags-select-find-tag-at-point if it will work"
  (if (find-tag-default)

This works by checking find-tag-default will work before calling etags-select-find-tag-at-point. Of course there is some duplication here because find-tag-default will get called again once I know it will work. Dissatisfied I asked the stackoverflow community for suggestions. The first solution is to simply trap the error case.

Third version:

(defun my-try-except-find-tag()
  "Find at point or fall to etags-select-find-tag on error"
  (unless (ignore-errors (or (etags-select-find-tag-at-point) 't))

This works by utilising deep lisp black magic to stop the error propagating and returning a ‘nil if it does. The (or (etags-select-find-tag-at-point) ‘t) line is to ensure a successful call returns something so we don’t then fall through. Interestingly the comments around subr.el mentions some of the keywords used may be redefined by common lisp.

Forth version:

(defun my-efficent-find-tag()
  "Find tag at point, caching find-tag-default"
  (let ((ftd (find-tag-default)))
    (flet ((find-tag-default () ftd))
      (if (find-tag-default)

While I expected something like ignore-errors to exist this demonstrates the flexibility of dynamic languages like Emacs Lisp. The key is the use of flet to redefine find-tag-default so when it gets executed again inside etags-select-find-tag-at-point it simply returns the cached value.

So as usual with these posts I try to invite feedback. Which of these forms do you prefer? Would you solve the problem another way? Have you just learnt something new about Emacs Lisp?

Living la vida ELPA

I’ve been running Emacs 24 direct from the version control tree (technically a git mirror, bzr still confuses me) for some time now. As many people have mentioned ELPA is one of the big features that helps de-clutter an Emacs users ~/.emacs.d directory. I thought it might be a useful exercise to discuss which packages I’m now loading from ELPA and which I still track directly.

First and formemost is my my dotemacs collection. Weighing in at around 2700 lines of elisp it’s small by some standards. I’ve toyed with moving my config across to things like the Emacs Starter Kit but I’m not sure if it’s worth the transition pain. There is certainly a lot of cruft in my code but there is also quite a lot of muscle memory now invested in it. I have been trying to modularise it a little more but to be honest most of that was driven by a desire to get autoload working nicely which is no longer much of an issue as my session tends to stay up for weeks at an end.

I’ll gloss over edit-server.el surfice it to say I’d hope your not surprised I keep the development version running given it’s for my extension 😉

After that we have Johnathan Rockway’s eproject. I work with lots of code bases during the day and some sort of sensible project type structure is a must. I’ve tried a number of different solutions and this one stuck as it was fairly lightweight and easy for my smaller lisp brain to extend. I’ve even managed to contribute some changes back. As a result I find tracking the bleeding edge of development useful.

The same reason applies to Stephen Bach’s lusty-emacs. While ido-mode and ibuffer work well for speed nothing matches lusty’s lazy file and buffer matching. It can get a little too much once you start dealing with hundreds of buffers at a time (something midnight-mode is trying to keep on top of). It’s still seeing some activitiy on the repository hence the local checkout.

Finally the biggie but generally unused cedet. There has been some work to make it comparitively easy to run out of a src checkout and as long as it’s included early enough it won’t conflict with the built in cedet shipping with Emacs 23+. I still haven’t really made much use of it although I have managed to get a completion out of it when editing some C. It’s basically sitting there until I can commit enough time to figuring out how to use the beast.

Finally a quick review of the ELPA packages that I’ve got loaded. Obviously there is the latest org-mode which I’m spending an increasing amount of time in. I also have my favourite zenburn-theme for easy on the eyes goodness. The popular GIT interaction mode magit also sits there which I use every day. js2-mode and ssh_config_mode completes the list of modules that I actually use.

I have a number of additional things in there which I don’t actually use at the moment but I plan to try out including nose (for python unit test), jsshell (for JavaScript coding) and another one I see a lot of the wizards using yasnippet but again is awaiting time to play with.

So what ELPA packages have you discovered and what hidden gems to you track directly in your .emacs.d?

Trials and Trufulations

One of the most challenging things about managing small bundles of fun is the fact their neural circuitry is brand new and developing very quickly. A key part of development at this stage is bonding with their parents. The last few weeks have been quite hard when Ursula has spent large chunks of time in my sole presence being very un-happy and wanting Mum (who I’ve been trying to let have some sleep). Understandably at this point she is most familiar with Fliss as well as having a fairly base urge for the milky goodness only she can provide. It hasn’t helped that I’ve been working hard so not always been able to get home in time for the evening bath and bed ritual.

This combined with lack of sleep has seen me reach a fairly low ebb of late. In fact it’s possible I’ve had a glimpse of the lack of emotional state that someone with depression has to deal with on a more regular basis. I was certainly hard pressed to remember the last feeling of emotion I’d had for some time.

It was fortunate then the jubilee weekend celebrations saw us visiting Leominster for M&T’s wedding anniversary. Being able to spend time with both Ursula and so many of my old friends from Manchester was an excellent tonic for my depressed mood. What was even more useful was being able to chat to fellow parents who had already been down this particular road and were more than reassuring. I didn’t realise quite how much I’d been bottling up worries about my parental abilities and it was good talk about it.

Of course now with those worries put aside for now we are tackling the next emotional roller coaster of trying to instill a regular sleep pattern in our dear daughter. Much as it is satisfying to successfully rock a previously fractious baby to sleep it’s not a long term viable solution. Ursula will have to learn to “self-sooth” and get herself to sleep without her parents. The process however is fraught and there is a lot of conflicting theories on the best way to achieve this. Suffice it to say it’s not been overly popular with Ursula herself and the resulting wobbly lipped tantrums have been taking their toll on us. We are trying to maintain perspective and keep our eyes on the long term goals. Wish us luck!