Alex's Adventures on the Infobahn - emacs

dired-rsync 0.7 released

2023-07-14T16:04:00+01:00

There haven't been many changes in the last few years but Liu Hui did submit a new package that allows you to use dired-rsync with a transient interface. Aside from that there have been a few minor bug fixes for a few issues and I spent some time cleaning up the bit rotted CI system by migrating from the broken Travis setup to using GitHub actions.

Enjoy!

v0.7

truncate process buffer output to last status line
raise minimum Emacs version to 25.1
new dired-rsync-transient frontend
various code clean-ups
new customisation hook dired-rsync-success-hook
moved the project CI to GitHub Actions
now understands alternate port syntax in tramp URIs

My Setup (2022 Edition)

2022-11-01T09:30:00+00:00

It's been a while since I last blogged about anything personally (although I have written a few things for work). In a struggle to come up with things to talk about I finally decided it might be worth documenting my current working setup. It's probably a function of my aging but as I've gotten older I've slowly tweaked and tuned my environment to minimise the friction of doing things. I'll briefly talk about the hardware but most of the environment is software and therefor fairly malleable.

Starting from the top and working my way down.

Desktop (sway/i3/Gnome/ChromeOS)

I spend by far and away the most time in Sway and previously it's X11 reference i3 which are both tiling window managers. I don't really use the tiling feature much as in reality my main workflow is to have full maximised single application screens in workspaces. However I do have hotkeys for the two most important applications I open (terminal and editor) and with a narrowing launcher for the rest. I use Gnome on one of my laptops because I share it with other people and it pays to keep track of the mainstream Linux desktop. I have remapped some keys to closer match my tiling experience as well as tweaking the default applications.

ChromeOS is the weird case where I accept a slightly different desktop paradigm because when travelling I like the ChromeOS feature set of secure and lightweight hardware. Any development I do on the road tends to use the Crostini terminal. More often than not it's just a shell for a ssh or mosh session to a bigger development box.

Key Desktop Apps (Emacs, Firefox, Modern Terminal)

It won't be a surprise to anyone that my main application is Emacs which provides the main interface into the rest of the system for pretty much everything except browsing the web. For web browsing I prefer Firefox which is synced up between my desktop and mobile. I occasionally have to use Chrome to access Javascript heavy web apps like Atlassian or some video call solutions.

I currently use a range of terminals depending on the system. My daily driver terminal on my main box is foot which is simple and minimal. It has a client/server setup so there is really only one foot binary running for all my terminals. I don't need anything fancy because all my terminal multiplexing is handled by tmux which also handles shell persistence.

Hardware

I now run a single desktop machine which is a ~9 year old Intel brought for work which has seen upgrades to the SSDs and thats about it. When I do serious building I have a range of powerful servers with lots of cores and disk space which I can use. I have an electric standing desk although I don't tend to move it during my work day. I do put it in the upright position at the end of the day to stop me sitting down if I'm just checking something in the evening. I also own a personal XPS15 laptop which I can run Steam on although mostly on the built in Intel graphics as Nouveau can't clock it to get any reasonable performance. I don't really mind because for most of my intensive high frame rate gaming I use my PS5 on a big flat screen TV.

For my NAS I have a Synology DiskStation with around 8TB of storage and a Raspberry Pi 4 in a cute metal case running Kodi connected to the main TV. Finally there is a 24 core Arm server which draws a miniscule 5w which acts as a permanent point of presence and eventually may take on some home automation tasks. This is all supplied with 300Mbs of fibre optic broadband and a mesh network of Eero's supplying WiFi to the rest of the house. My office network has a hard link to the main router and comes through a Turris Omnia.

dired-rsync 0.6 released

2020-12-24T22:18:00+00:00

Given the slowly growing collection of fixes I thought it was worth cutting a new release for the benefit of those that get their dired-rsync via the stable MELPA channels. The biggest user visible change is dired-rsync no longer tweaks mode-line-process itself. You can now decide if to add dired-rsync-modeline-status in your modeline (or anywhere else if you wish).

Merry Christmas ;-)

v0.6

use tramp functions to decompose URIs (fix #22)
use username from dired URI if we have it in rsync
fix escaping of 's for remote-to-remote copies
better detection of running jobs by parsing buffers
increment the reverse proxy port based on active connections
don't set mode-line-process, leave it to user to put dired-rsync-modeline-status somewhere

Edit with Emacs v1.16 released

2020-12-24T16:10:00+00:00

Despite what I said last time it seems that this is still a useful tool for me. I had some free time to peruse some of the open issues and managed to work out what was breaking the GMail interaction and work out a fix. While Atomic Chrome provides a nicer interactive experience with direct feedback the release of Crostini keeps the Emacs flame alive on my Chromebook. I suspect I still need someone to step up who is more conversant in the web languages of Javascript and CSS to improve the front end experience and maybe tackle the frequently requested feature of interaction with fancy javascript editors. In the meantime I'll continue to bumble along and look at what pull requests do come in and leave the pile of feature requests slowly growing.

Merry Christmas ;-)

v1.16

Extension

allow disabling of switch to settings behaviour
whitelist penguin.linux.test for edit server (ChromeOS/Crostini)
fix context menu on newer Chromes (#158)
honour edit_server_host instead of hard-coding 127.0.0.1
use the Chrome/Firefox extension UI to set keyboard shortcuts.
fix handling of spellcheck=false nodes for Gmail (#171, #162)

edit-server.el

use make-frame (see updated docs for edit-server-new-frame-alist)

magit-file-dispatch

2020-12-14T16:10:00+00:00

magit-file-dispatch is my new favourite command since the recent removal of magit-file-mode. Rather than paper over the change and stick to the old way I tried the suggestions of launching with the dispatch commands. I discovered a few cool things including the ability to trace an individual functions changes over time which has completely replaced my old method of piping git log through less. In the end I found the fall-back of going to magit-dispatch a little too much friction given I've a lot of memory muscle to launch the status of git from various non-file backed yet still project orientated buffers. compilation-mode is one example but I also have my mail client set-up so I can quickly get to the code from the relevant mailing list. As everyone can have their own special snowflake settings in Emacs I finally went with:

(with-eval-after-load 'magit
  (defun my-magit-file-bindings ()
    "Setup my file bindings"
    (local-set-key (kbd "<f5>") 'my-counsel-git-grep))
  (add-hook 'magit-find-file-hook 'my-magit-file-bindings))

(defun my-magit-dispatch (&optional prefix)
  "My personal preference for magit-dispatch.
While magit-file-dispatch is cool, falling back to magit-dispatch is
not, I'd rather just go to magit-status. Lets make it so."
  (interactive "P")
  (if (or prefix
          (not (buffer-file-name))
          (not (functionp 'magit-file-dispatch)))
      (magit-status)
    (magit-file-dispatch)))

(use-package magit
  :bind (("C-x g" . my-magit-dispatch))

Migrating again

2020-05-25T17:53:00+01:00

It's been some time since I last blogged but I have been doing some spring cleaning amongst my servers. While moving all my web content onto a new box I though I would take advantage of the fresh start to migrate away from Wordpress. While it is a fine FLOSS project it really needs quite a bit of time to keep it up to date and secure. I've decided it's much better to have a static site so I've imported all my exiting blog content into a bunch of rST and Markdown files and re-generated the site with a Python based content generator called Pelican.

So far it seems easy enough to tweak. The main challenge is ensuring the various feeds for different planets are in the same place as they would have been with Wordpress. One difference with the change is the lack of comments although I think I could probably re-enable them with some 3rd party solution if I wanted to. We shall see.

dired-rsync 0.5 release

2019-11-10T19:42:00+00:00

If you've been running from MELPA you will have had all these features already but it was certainly overdue a stable tagging. The commit history shows a lot of faffing around getting the CI working again. However there are a bunch of customisation tweaks exposed as well as support for remote-to-remote rsync support (with a very ugly port-forward hack). From the NEWS file:

Big CI clean-ups (dropped emake, added Cask)
smarter modeline status (show R:n% when one job running)
make dired-rsync process dest with expand-file-name
we now support remote to remote with an ugly port forward hack
prompt the user for a passphrase if we detect stall in process
add some basic ert tests
new customisation option dired-rsync-source-files
new customisation hook dired-rsync-failed-hook

Happy file syncing ;-)

dired-rsync 0.4 released

2018-06-01T18:12:00+01:00

I started hacking on this a while back but I've finally done the house-keeping tasks required to make it a proper grown up package.

dired-rsync is a simple command which you can use to trigger an rsync copy from within dired. This is especially useful when you want to copy across large files from a remote server without locking up Emacs/Tramp. The rsync just runs as an inferior process in the background.

Today was mainly a process of cleaning up the CI and fixing any issues with it. I'd still like to add some proper tests but the whole thing is interactive and that seems to be tricky for Emacs to test. Anyway I've now tagged 0.4 so it will be available from MELPA Stable once it rebuilds. You can of course grab the building edge from MELPA any time ;-)

Working with dired

2018-04-07T11:12:00+01:00

I've been making a lot more use of dired recently. One use case is copying files from my remote server to my home machine. Doing this directly from dired, even with the power of tramp, is a little too time consuming and potentially locks up your session for large files. While browsing reddit r/emacs I found a reference to this post <https://vxlabs.com/2018/03/30/asynchronous-rsync-with-emacs-dired-and-tramp/> that spurred me to look at spawning rsync from dired some more.

:: r/emacs: http://reddit.com/r/emacs

Unfortunately the solution is currently sitting in a pull-request to what looks like an orphaned package. I also ran into some other problems with the handling of where rsync needs to be run from so rather than unpicking some unfamiliar code I decided to re-implement everything in my own package <https://github.com/stsquad/dired-rsync>.

I've still got some debugging to do to get it to cleanly handle multiple sessions as well as a more detailed mode-line status. Once I'm happy I'll tag a 0.1 and get it submitted to MELPA.

While getting more familiar with dired I also came up with this little helper:

(defun my-dired-frame (directory)
  "Open up a dired frame which closes on exit."
  (interactive)
  (switch-to-buffer (dired directory))
  (local-set-key
   (kbd "C-x C-c")
   (lambda ()
     (interactive)
     (kill-this-buffer)
     (save-buffers-kill-terminal 't))))

Which is paired with a simple alias in my shell setup:

alias dired="emacsclient -a '' -t -e '(my-dired-frame default-directory)'"

This works really nicely for popping up a dired frame in your terminal window and cleaning itself up when you exit.

Solving the HKG18 puzzle with org-mode

2018-03-26T11:19:00+01:00

One of the traditions I like about Linaro's Connect event is the conference puzzle. Usually set by Dave Piggot they provide a challenge to your jet lagged brain. Full disclosure: I did not complete the puzzle in time. In fact when Dave explained it I realised the answer had been staring me in the face. However I thought a successful walk through would make for a more entertaining read ;-)

First the Puzzle:

Take the clues below and solve them. Once solved, figure out what the hex numbers mean and then you should be able to associate each of the clue solutions with their respective hex numbers.

Clue	Hex Number
Lava Ale Code	1114DBA
Be Google Roe	114F6BE
Natural Gin	114F72A
Pope Charger	121EE50
Dolt And Hunk	12264BC
Monk Hops Net	122D9D9
Is Enriched Tin	123C1EF
Bran Hearing Kin	1245D6E
Enter Slim Beer	127B78E
Herbal Cabbages	1282FDD
Jan Venom Hon Nun	12853C5
A Cherry Skull	1287B3C
Each Noun Lands	1298F0B
Wave Zone Kits	12A024C
Avid Null Sorts	12A5190
Handcars All Trim	12C76DC

Clues

It looks like all the clues are anagrams. I was lazy and just used the first online anagram solver that Google pointed me at. However we can automate this by combining org-mode with Python and the excellent Beautiful Soup library.

from bs4 import BeautifulSoup
import requests
import re

# ask internet to solve the puzzle
url="http://anagram-solver.net/%s" % (anagram.replace(" ", "%20"))
page=requests.get(url)

# fish out the answers
soup=BeautifulSoup(page.text)
answers=soup.find("ul", class_="answers")
for li in answers.find_all("li"):
    result = li.text
    # filter out non computer related or poor results
    if result in ["Elmer Berstein", "Tim-Berners Lee", "Babbage Charles", "Calude Shannon"]:
        continue
    # filter out non proper names
    if re.search("[a-z] [A-Z]", result):
        break

return result

So with :var anagram=clues[2,0] we get

Ada Lovelace

I admit the "if result in []" is a bit of hack.

Hex Numbers

The hex numbers could be anything. But lets first start by converting to something else.

Hex Prompt	Number
1114DBA	17911226
114F6BE	18151102
114F72A	18151210
121EE50	19000912
12264BC	19031228
122D9D9	19061209
123C1EF	19120623
1245D6E	19160430
127B78E	19380110
1282FDD	19410909
12853C5	19420101
1287B3C	19430204
1298F0B	19500811
12A024C	19530316
12A5190	19550608
12C76DC	19691228

The #+TBLFM: is $1='(identity remote(clues,@@#$2))::$2='(string-to-number $1 16)

This is where I went down a blind alley. The fact all they all had the top bit set made me think that Dave was giving a hint to the purpose of the hex number in the way many cryptic crosswords do (I know he is a fan of these). However the more obvious answer is that everyone in the list was born in the last millennium.

Looking up Birth Dates

Now I could go through all the names by hand and look up their birth dates but as we are automating things perhaps we can use computers for what they are good at. Unfortunately there isn't a simple web-api for looking up this stuff. However there is a project called DBpedia which takes Wikipedia's data and attempts to make it semantically useful. We can query this using a semantic query language called SparQL. If only I could call it from Emacs…

select distinct ?birthDate {
  dbr:$name dbo:birthDate|dbp:birthDate ?birthDate
}

So calling with :var name="Ada_Lovelace" we get

"birthDate"
1815-12-10

Of course it shouldn't be a surprise this exists. And in what I hope is a growing trend sparql-mode supports org-mode out of the box. The $name in the snippet is expanded from the passed in variables to the function. This makes it a general purpose lookup function we can use for all our names.

There are a couple of wrinkles. We need to format the name we are looking up with underscores to make a valid URL. Also the output spits out a header and possible multiple birth dates. We can solve this with a little wrapper function. It also introduces some rate limiting so we don't smash DBpedia's public SPARQL endpoint.

;; rate limit
(sleep-for 1)
;; do the query
(let* ((str (s-replace-all '((" " . "_") ("Von" . "von")) name))
       (ret (eval
             (car
              (read-from-string
               (format "(org-sbe get-dob (name $\"%s\"))" str))))))
  (string-to-number (replace-regexp-in-string "-" "" (car (cdr (s-lines ret))))))

Calling with :var name="Ada Lovelace" we get

18151210

Full Solution

So now we know what we are doing we need to solve all the puzzles and lookup the data. Fortunately org-mode's tables are fully functional spreadsheets except they are not limited to simple transformations. Each formula can be a fully realised bit of elisp, calling other source blocks as needed.

Clue	Solution	DOB
Herbal Cabbages	Charles Babbage	17911226
Be Google Roe	George Boole	18151102
Lava Ale Code	Ada Lovelace	18151210
A Cherry Skull	Haskell Curry	19000912
Jan Venom Hon Nun	John Von Neumann	19031228
Pope Charger	Grace Hopper	19061209
Natural Gin	Alan Turing	19120623
Each Noun Lands	Claude Shannon	19160430
Dolt And Hunk	Donald Knuth	19380110
Is Enriched Tin	Dennis Ritchie	19410909
Bran Hearing Kin	Brian Kernighan	19420101
Monk Hops Net	Ken Thompson	19430204
Wave Zone Kits	Steve Wozniak	19500811
Handcars All Trim	Richard Stallman	19530316
Enter Slim Beer	Tim Berners-Lee	19550608
Avid Null Sorts	Linus Torvalds	19691228

The #+TBLFM: is $1='(identity remote(clues,@@#$1))::$2='(org-sbe solve-anagram (anagram $$1))::$3='(org-sbe frob-dob (name $$2))

The hex numbers are helpfully sorted so as long as we sort the clues table by the looked up date of birth using M-x org-table-sort-lines we are good to go.

Workbooks for Benchmarking

2018-02-21T20:34:00+00:00

While working on a major re-factor of QEMU's softfloat code I've been doing a lot of benchmarking. It can be quite tedious work as you need to be careful you've run the correct steps on the correct binaries and keeping notes is important. It is a task that cries out for scripting but that in itself can be a compromise as you end up stitching a pipeline of commands together in something like perl. You may script it all in a language designed for this sort of thing like R but then find your final upload step is a pain to implement.

One solution to this is to use a literate programming workbook like this. Literate programming is a style where you interleave your code with natural prose describing the steps you go through. This is different from simply having well commented code in a source tree. For one thing you do not have to leap around a large code base as everything you need is on the file you are reading, from top to bottom. There are many solutions out there including various python based examples. Of course being a happy Emacs user I use one of its stand-out features org-mode which comes with multi-language org-babel support. This allows me to document my benchmarking while scripting up the steps in a variety of "languages" depending on the my needs at the time. Let's take a look at the first section:

1 Binaries To Test

Here we have several tables of binaries to test. We refer to the

current benchmarking set from the next stage, Run Benchmark.

For a final test we might compare the system QEMU with a reference

build as well as our current build.

Binary title

/usr/bin/qemu-aarch64 system-2.5.log

~/lsrc/qemu/qemu-builddirs/arm-targets.build/aarch64-linux-user/qemu-aarch64 master.log

~/lsrc/qemu/qemu.git/aarch64-linux-user/qemu-aarch64 softfloat-v4.log

Binary	title
/usr/bin/qemu-aarch64	system-2.5.log
~/lsrc/qemu/qemu-builddirs/arm-targets.build/aarch64-linux-user/qemu-aarch64	master.log
~/lsrc/qemu/qemu.git/aarch64-linux-user/qemu-aarch64	softfloat-v4.log

Well that is certainly fairly explanatory. These are named org-mode tables which can be referred to in other code snippets and passed in as variables. So the next job is to run the benchmark itself:

2 Run Benchmark

This runs the benchmark against each binary we have selected above.

import subprocess
import os

runs=[]

for qemu,logname in files:
cmd="taskset -c 0 %s ./vector-benchmark -n %s | tee %s" % (qemu, tests, logname)
    subprocess.call(cmd, shell=True)
    runs.append(logname)

    return runs

So why use python as the test runner? Well truth is whenever I end up munging arrays in shell script I forget the syntax and end up jumping through all sorts of hoops. Easier just to have some simple python. I use python again later to read the data back into an org-table so I can pass it to the next step, graphing:

set title "Vector Benchmark Results (lower is better)"
set style data histograms
set style fill solid 1.0 border lt -1

set xtics rotate by 90 right
set yrange [:]
set xlabel noenhanced
set ylabel "nsecs/Kop" noenhanced
set xtics noenhanced
set ytics noenhanced
set boxwidth 1
set xtics format ""
set xtics scale 0
set grid ytics
set term pngcairo size 1200,500

plot for [i=2:5] data using i:xtic(1) title columnhead

This is a GNU Plot script which takes the data and plots an image from it. org-mode takes care of the details of marshalling the table data into GNU Plot so all this script is really concerned with is setting styles and titles. The language is capable of some fairly advanced stuff but I could always pre-process the data with something else if I needed to.

Finally I need to upload my graph to an image hosting service to share with my colleges. This can be done with a elaborate curl command but I have another trick at my disposal thanks to the excellent restclient-mode. This mode is actually designed for interactive debugging of REST APIs but it is also easily to use from an org-mode source block. So the whole thing looks like a HTTP session:

:client_id = feedbeef

# Upload images to imgur
POST https://api.imgur.com/3/image
Authorization: Client-ID :client_id
Content-type: image/png

< benchmark.png

Finally because the above dumps all the headers when run (which is very handy for debugging) I actually only want the URL in most cases. I can do this simply enough in elisp:

#+name: post-to-imgur
#+begin_src emacs-lisp :var json-string=upload-to-imgur()
  (when (string-match
         (rx "link" (one-or-more (any "\":" whitespace))
             (group (one-or-more (not (any "\"")))))
         json-string)
    (match-string 1 json-string))
#+end_src

The :var line calls the restclient-mode function automatically and passes it the result which it can then extract the final URL from.

And there you have it, my entire benchmarking workflow document in a single file which I can read through tweaking each step as I go. This isn't the first time I've done this sort of thing. As I use org-mode extensively as a logbook to keep track of my upstream work I've slowly grown a series of scripts for common tasks. For example every patch series and pull request I post is done via org. I keep the whole thing in a git repository so each time I finish a sequence I can commit the results into the repository as a permanent record of what steps I ran.

If you want even more inspiration I suggest you look at John Kitchen's scimax work. As a publishing scientist he makes extensive use of org-mode when writing his papers. He is able to include the main prose with the code to plot the graphs and tables in a single source document from which his camera ready documents are generated. Should he ever need to reproduce any work his exact steps are all there in the source document. Yet another example of why org-mode is awesome ;-)

FOSDEM 2018

2018-02-06T09:36:00+00:00

I've just returned from a weekend in Brussels for my first ever FOSDEM - the Free and Open Source Developers, European Meeting. It's been on my list of conferences to go to for some time and thanks to getting my talk accepted, my employer financed the cost of travel and hotels. Thanks to the support of the Université libre de Bruxelles (ULB) the event itself is free and run entirely by volunteers. As you can expect from the name they also have a strong commitment to free and open source software.

The first thing that struck me about the conference is how wide ranging it was. There were talks on everything from the internals of debugging tools to developing public policy. When I first loaded up their excellent companion app (naturally via the F-Droid repository) I was somewhat overwhelmed by the choice. As it is a free conference there is no limit on the numbers who can attend which means you are not always guarenteed to be able to get into every talk. In fact during the event I walked past many long queues for the more popular talks. In the end I ended up just bookmarking all the talks I was interested in and deciding which one to go to depending on how I felt at the time. Fortunately FOSDEM have a strong archiving policy and video most of their talks so I'll be spending the next few weeks catching up on the ones I missed.

There now follows a non-exhaustive list of the most interesting ones I was able to see live:

Dashamir's talk on EasyGPG dealt with the opinionated decisions it makes to try and make the use of GnuPG more intuitive to those not versed in the full gory details of public key cryptography. Although I use GPG mainly for signing GIT pull requests I really should make better use it over all. The split-key solution to backups was particularly interesting. I suspect I'll need a little convincing before I put part of my key in the cloud but I'll certainly check out his scripts.

Liam's A Circuit Less Travelled was an entertaining tour of some of the technologies and ideas from early computer history that got abandoned on the wayside. These ideas were often to be re-invented in a less superior form as engineers realised the error of their ways as technology advanced. The later half of the talk turns into a bit of LISP love-fest but as an Emacs user with an ever growing config file that is fine by me ;-)

Following on in the history vein was Steven Goodwin's talk on Digital Archaeology which was a salutatory reminder of the amount of recent history that is getting lost as computing's breakneck pace has discarded old physical formats in lieu of newer equally short lived formats. It reminded me I should really do something about the 3 boxes of floppy disks I have under my desk. I also need to schedule a visit to the Computer History Museum with my children seeing as it is more or less on my doorstep.

There was a tongue in check preview that described the EDSAC talk as recreating "an ancient computer without any of the things that made it interesting". This was was a little unkind. Although the project re-implemented the computation parts in a tiny little FPGA the core idea was to introduce potential students to the physicality of the early computers. After an introduction to the hoary architecture of the original EDSAC and the Wheeler Jump Mary introduced the hardware they re-imagined for the project. The first was an optical reader developed to read in paper tapes although this time ones printed on thermal receipt paper. This included an in-depth review of the problems of smoothing out analogue inputs to get reliable signals from their optical sensors which mirrors the problems the rebuild is facing with nature of the valves used in EDSAC. It is a shame they couldn't come up with some way to involve a valve but I guess high-tension supplies and school kids don't mix well. However they did come up with a way of re-creating the original acoustic mercury delay lines but this time with a tube of air and some 3D printed parabolic ends.

The big geek event was the much anticipated announcement of RISC-V hardware during the RISC-V enablement talk. It seemed to be an open secret the announcement was coming but it still garnered hearty applause when it finally came. I should point out I'm indirectly employed by companies with an interest in a competing architecture but it is still good to see other stuff out there. The board is fairly open but there are still some peripheral IPs which were closed which shows just how tricky getting to fully-free hardware is going to be. As I understand the RISC-V's licensing model the ISA is open (unlike for example an ARM Architecture License) but individual companies can still have closed implementations which they license to be manufactured which is how I assume SiFive funds development. The actual CPU implementation is still very much a black box you have to take on trust.

Finally for those that are interested my talk is already online for those that are interested in what I'm currently working on. The slides have been slightly cropped in the video but if you follow the link to the HTML version you can read along on your machine.

I have to say FOSDEM's setup is pretty impressive. Although there was a volunteer in each room to deal with fire safety and replace microphones all the recording is fully automated. There are rather fancy hand crafted wooden boxes in each room which take the feed from your laptop and mux it with the camera. I got the email from the automated system asking me to review a preview of my talk about half and hour after I gave it. It took a little longer for the final product to get encoded and online but it's certainly the nicest system I've come across so far.

All in all I can heartily recommend FOSDEM for anyone in an interest is FLOSS. It's a packed schedule and there is going to be something for everyone there. Big thanks to all the volunteers and organisers and I hope I can make it next year ;-)

Edit with Emacs v1.15 released

2018-01-17T16:47:00+00:00

After a bit of hiatus there was enough of a flurry of patches to make it worth pushing out a new release. I'm in a little bit of a quandary with what to do with this package now. It's obviously a useful extension for a good number of people but I notice the slowly growing number of issues which I'm not making much progress on. It's hard to find time to debug and fix things when it's main state is Works For Me. There is also competition from the Atomic Chrome extension (and it's related emacs extension). It's an excellent package and has the advantage of a Chrome extension that is more actively developed and using a bi-directional web-socket to communicate with the edit server. It's been a feature I've wanted to add to Edit with Emacs for a while but my re-factoring efforts are slowed down by the fact that Javascript is not a language I'm fluent in and finding a long enough period of spare time is hard with a family. I guess this is a roundabout way of saying that realistically this package is in maintenance mode and you shouldn't expect to see any new development for the time being. I'll of course try my best to address reproducible bugs and process pull requests in a timely manner. That said please enjoy v1.15:

Extension

* Now builds for Firefox using WebExtension hooks
* Use chrome.notifications instead of webkitNotifications
* Use

with style instead of inline for edit button * fake "input" event to stop active page components overwriting text area

edit-server.el

* avoid calling make-frame-on-display for TTY setups (#103/#132/#133)
* restore edit-server-default-major-mode if auto-mode lookup fails
* delete window when done editing with no new frame

Get the latest from the Chrome Webstore.

checkpatch-mode

2017-02-09T15:30:00+00:00

A couple of weeks ago I mused that I should really collect together the various hacks to integrate checkpatch into my workflow into a consistent mode. Having a quick look around I couldn't find any other implementations and went to create the said mode. It turns out I'd created the directory and done the initial commit 3 years ago. Anyway I polished it up a bit and you can now get it here. I hope it's useful to the wider community and as ever patches welcome ;-)

Anyone interested at an Emacs BoF at LCU14?

2014-04-29T16:00:00+01:00

I'm fairly atypical as a Linaro employee because I have a desk in a shared office. This means I get to participate in technical banter with the other employees based in the Cambridge office. However it has become quite clear I'm surrounded on all sides by VIMers with only one potential convert who wants to try Emacs out "one day". As a result I thought it might be nice to have an Emacs Birds of a Feather (BoF) session at LCU14?

BoF sessions are basically an informal gathering where people with a shared interest who come along to swap tips and stories about their area of interest. In the context of LCU it would be an opportunity to network and meet fellow Emacers. So any interest?

Boutique modes

2014-03-05T10:12:00+00:00

I've recently started a new job at Linaro which has been keeping me very busy. My role combines the low-level fun of Dynamic Binary Translation that I so enjoyed at Transitive and the fact Linaro is a fully Open Source company. I work directly on the upstream projects (in this case QEMU) and with the project community. In many ways it's my ideal job.

Of course I've quickly built up a reputation as the Emacs guy in the office surrounded by a sea of VIMers although one of the guys does profess a desire to learn Emacs "one day".

One of the first things I did was move all my email into Emacs. I'd long been dissatisfied with the state of Thunderbird (and previously Evolution) that I took the opportunity to migrate to mu4e. I spend my days slinging patches and going through mailing lists so I really appreciate being in my editor while I do that.

I have also started streamlining some of my work-flow with a few specialised extensions to Emacs. I think I'm finally comfortable enough with elisp to have a pretty good stab at solving most problems. I'm also appreciating the ability to leverage the mass of Emacs code under the hood to make incremental tweaks rather than solve everything at once. I thought I might give you a tour of the code I've written so far.

First up is risu-mode. RISU is the code testing tool we've been using to verify our aarch64 ARM work in QEMU. The .risu file is a template that's used to specify instruction patterns that the tool then uses to generate random code sequences with. risu-mode is really just a bunch of regex expressions wrapped in the mode machinery that highlights the elements on the page. It doesn't sound like much but when you are working through a bunch of patterns looking for bugs it's easier on the eye when the different elements are coloured.

Next thing I wrote was my own QEMU mode which is a simple comint-mode based mode for launching QEMU system emulation. It's still very rough and ready as I'm mostly working on user emulation but I suspect it will be handy once I start on system emulation stuff.

Finally there is lava-mode. LAVA is Linaro's automated test and validation framework. Although it already provides command line and web interfaces I thought it would be nice to launch and track test jobs from within Emacs itself. The job control files a JSON based so I built on the existing json-mode and a slightly patched version of xml-rpc.el to add job submission. I've started a simple job tracking mode that uses the tabulated-list-mode framework and eventually I'll link it into the tracking library so job completion will be as seamless as my IRC work-flow.

So there you have it, a bunch of code that may well not interest anyone else but shows how Emacs provides a very rich base of functionality on which to build up tools that are useful to you.

Does anyone else want to share an example of their esoteric extensions? What's the most obscure thing you've built on top of our favourite text editor?

Edit with Emacs v1.13 now available

2013-11-08T14:05:00+00:00

I've just pushed the latest version of Edit with Emacs to the Chrome App Store. Hopefully most people are already tracking the latest edit-server.el via MELPA but this does introduce a few minor fixes to the extension itself. A new piece of functionality is the ability to trigger bringing Emacs to the foreground from a key-stroke within Chrome. I added this to support running Emacs on ChromeOS which together with my chromebooks.el package gives me a rather nice development environment without having to dump ChromeOS.

So new for v1.13

Extension

* Change the handling of hidden elements (fix bug #78)
* Add debugging for erroneous hidden text areas (#93)
* Add keyboard shortcut to bring Emacs to foreground
* Pass clipboard contents to foreground request

edit-server.el

* add advice to save-buffers-kill-emacs to avoid prompting on shutdown
* add autoload cookies
* fix bug with format chars in url (#80)
* don't call kill buffer hooks twice (#92)
* don't set-buffer-multibyte on process buffer
* support the "foreground" request with optional clipboard contents

Get the latest from the Chrome Webstore.

On the death of Google Reader

2013-03-14T22:17:00+00:00

You can probably tell the sort of on-line company I keep from the deluge of noise on the social networks regarding Google's decision to shut down Reader. However we shouldn't be that surprised. In fact some companies that source content from Reader have anticipated the need to collect content themselves.

I of course will have to make a decision at some point. However I'll not do it today like a lot of Reader users have. The rush to try out alternatives has over-whelmed some open source based projects who were quietly growing organically. I don't envy those that have to suddenly gear up their back-end systems because an Internet behemoth gave us 2593 hours notice to sort out a replacement.

I'm mulling over the difference between self-hosting and having someone else do it. I'm not overly worried about going for convenience if I know I can get my data back if I need to. In fact the knowledge that you can theoretically self-host might be enough. To be fair to Google their Data Liberation team made exporting all my Reader data easy.

Before I make a choice I need to decide what my priorities are. Currently I subscribe to 250+ RSS feeds. Obviously I don't read every single post but I make extensive use of tags to quickly process through stuff I do need to see when I need to see it. Aside from news, blog posts, funny cat pictures I also subscribe to other data feeds like bug trackers, code repositories, and other data sources. I of course want access to all of this data at any point on one of a number of devices. This makes a web hosted solution pretty much a must. There is no point having the data on my desktop when I'm somewhere else. From my point of view I want it to be open source compatible because if the company hosting now decides it no longer wants to I'll only have to move the data and not break my work-flow.

It would also be very useful if it had a public API so others can interact with the data. I don't need the solution to be all provided by one company. It's perfectly fine to have multiple 3rd parties sorting out the Android integration. I might even look to doing something to integrate it with my favourite editor (the name of which even my non-geek readers probably know by now). So far my experiment with moving all of IRC and IM into Emacs seems to be working well and should be a subject of another post.

Are you a Reader user? What are your criteria for it's eventual replacement? Is RSS just a dying protocol or is the need to aggregate and sift through data becoming more important?

There may well be a much better way of solving this problem around the corner. I certainly am open to persuasion. But don't take away my current preferred solution until I'm convinced I'm ready to switch ;-)

Edit with Emacs v1.12 now available

2012-12-03T21:58:00+00:00

Apologies for the extended delay in getting out a new release of Edit with Emacs. Real life has been doing a good job keeping me busy that spare hacking time is in short supply. A number of people have submitted pull requests and I've also done a bit of clean-up on the code.

The JavaScript in the extension has had the hacky change tracking code replaced with a modern funky 21st century Mutation Observer implementation. This should solve some of the more obvious performance problems on highly dynamic pages.

I've also implemented the iterative save method for edit-server.el. Unfortunately it doesn't quite work as intended if the edited region uses encoding (e.g. longlines). In theory the buffer you have saved should have it's contents replaced with the new data from the browser but it just doesn't seem to work. For now I've just ensured the kill-ring has the real data in it so it's not overly painful to restore formatting. Patches and/or explanations are of course welcome!

So grab it now from webstore. The changes are as follows:

Extension

* Upgraded manifest to version 2 for future releases of Chrome
* Added an "Enable Debug" flag to control logging to console
* Use Mutation Observers to better track changes in dynamic DOMs
* Fake a keypress when updating the textarea
* Bug fix for handling editable DIV elements

edit-server.el

* Now available from the MELPA package archive!
* Beta support for iterative C-x C-s saving (it works but is ugly, see kill-ring)
* Default to using UTF-8 coding for process communication
* Raise existing frame consistently on Mac
* Tweak the portion of the URL used to name buffers

Many ways to skin a GNU

2012-07-24T15:09:00+01:00

It's time for another examination of Emacs LISP and a selection of ways to solve the same problem. In this case it's the behaviour of the etags-select package. This handy function will present a selection table when it finds multiple definitions of the same tag. This is handy if your global TAGs file contains reference to multiple binaries that might have a common code heritage. It even provides a handy function etags-select-find-tag-at-point which will check point to see if that is a tag. However if point is on a blank line I'd prefer it to just prompt me for a tag name.

First version:

(defun my-naive-find-tag ()
  "Find at point or fall back"
  (interactive)
  (unless (etags-select-find-tag-at-point)
    (etags-select-find-tag)))

Unfortunately this fails rather badly. I naively assumed etags-select-find-tag-at-point would return 'nil on failure. Instead it bombs out with an error because etags-select-find expects a parameter and when find-tag-default fails it errors out.

Second version:

(defun my-working-find-tag()
  "Find tag and call etags-select-find-tag-at-point if it will work"
  (interactive)
  (if (find-tag-default)
      (etags-select-find-tag-at-point)
    (etags-select-find-tag)))

This works by checking find-tag-default will work before calling etags-select-find-tag-at-point. Of course there is some duplication here because find-tag-default will get called again once I know it will work. Dissatisfied I asked the stackoverflow community for suggestions. The first solution is to simply trap the error case.

Third version:

(defun my-try-except-find-tag()
  "Find at point or fall to etags-select-find-tag on error"
  (interactive)
  (unless (ignore-errors (or (etags-select-find-tag-at-point) 't))
    (etags-select-find-tag)))

This works by utilising deep lisp black magic to stop the error propagating and returning a 'nil if it does. The (or (etags-select-find-tag-at-point) 't) line is to ensure a successful call returns something so we don't then fall through. Interestingly the comments around subr.el mentions some of the keywords used may be redefined by common lisp.

Forth version:

(defun my-efficent-find-tag()
  "Find tag at point, caching find-tag-default"
  (interactive)
  (let ((ftd (find-tag-default)))
    (flet ((find-tag-default () ftd))
      (if (find-tag-default)
          (etags-select-find-tag-at-point)
        (etags-select-find-tag)))))

While I expected something like ignore-errors to exist this demonstrates the flexibility of dynamic languages like Emacs Lisp. The key is the use of flet to redefine find-tag-default so when it gets executed again inside etags-select-find-tag-at-point it simply returns the cached value.

So as usual with these posts I try to invite feedback. Which of these forms do you prefer? Would you solve the problem another way? Have you just learnt something new about Emacs Lisp?

Living la vida ELPA

2012-07-10T12:45:00+01:00

I've been running Emacs 24 direct from the version control tree (technically a git mirror, bzr still confuses me) for some time now. As many people have mentioned ELPA is one of the big features that helps de-clutter an Emacs users ~/.emacs.d directory. I thought it might be a useful exercise to discuss which packages I'm now loading from ELPA and which I still track directly.

First and formemost is my my dotemacs collection. Weighing in at around 2700 lines of elisp it's small by some standards. I've toyed with moving my config across to things like the Emacs Starter Kit but I'm not sure if it's worth the transition pain. There is certainly a lot of cruft in my code but there is also quite a lot of muscle memory now invested in it. I have been trying to modularise it a little more but to be honest most of that was driven by a desire to get autoload working nicely which is no longer much of an issue as my session tends to stay up for weeks at an end.

I'll gloss over edit-server.el surfice it to say I'd hope your not surprised I keep the development version running given it's for my extension ;-)

After that we have Johnathan Rockway's eproject. I work with lots of code bases during the day and some sort of sensible project type structure is a must. I've tried a number of different solutions and this one stuck as it was fairly lightweight and easy for my smaller lisp brain to extend. I've even managed to contribute some changes back. As a result I find tracking the bleeding edge of development useful.

The same reason applies to Stephen Bach's lusty-emacs. While ido-mode and ibuffer work well for speed nothing matches lusty's lazy file and buffer matching. It can get a little too much once you start dealing with hundreds of buffers at a time (something midnight-mode is trying to keep on top of). It's still seeing some activitiy on the repository hence the local checkout.

Finally the biggie but generally unused cedet. There has been some work to make it comparitively easy to run out of a src checkout and as long as it's included early enough it won't conflict with the built in cedet shipping with Emacs 23+. I still haven't really made much use of it although I have managed to get a completion out of it when editing some C. It's basically sitting there until I can commit enough time to figuring out how to use the beast.

Finally a quick review of the ELPA packages that I've got loaded. Obviously there is the latest org-mode which I'm spending an increasing amount of time in. I also have my favourite zenburn-theme for easy on the eyes goodness. The popular GIT interaction mode magit also sits there which I use every day. js2-mode and ssh_config_mode completes the list of modules that I actually use.

I have a number of additional things in there which I don't actually use at the moment but I plan to try out including nose (for python unit test), jsshell (for JavaScript coding) and another one I see a lot of the wizards using yasnippet but again is awaiting time to play with.

So what ELPA packages have you discovered and what hidden gems to you track directly in your .emacs.d?

Edit with Emacs v1.11

2012-05-23T15:57:00+01:00

I haven't really had much time for hacking at home unless programming my daughters wetware counts. However a few enterprising users have been sending me in some pull requests that fix a few minor bugs that slipped through the extensive pre-release testing I did for v1.10 so I thought it was worth pushing out a bug-fix release:

Extension

* Fixed some breakage to the "edit focused area" feature

* Fixed broken link to embedded edit-server

edit-server.el

* A couple of fixes for Emacs 24 compatibility

* Globalize the minor mode so it persists through major-mode changes

Now a Windows user!

2012-05-01T13:50:00+01:00

I've finally buckled and now have a Windows laptop. It was mainly forced on me by the need to have some sort of access to the intranet during the large number of meetings I'm now involved in at work. I have to say the experience has been enlightening, especially seeing all the hacky stuff that has to be done to get things working under Windows. For example by default I couldn't connect to any https (SSL) pages. Luckily I can just hand the laptop back to IT to fix it so I don't have to scratch my head too much.

I've had only sporadic luck getting Emacs up and running on it though. Having been running the tip-of-tree release on my Linux workstation so long going back to Emacs 23 has been a bit of a retrograde step. It doesn't help there are multiple suggestions for installation. I've been trying to get EmacsW32 working but I've run into problems on start-up. So far I've been unable to fix the issue as the paths Emacs references don't seem to show up in the system file browser. This seems to be the only avenue by which I can fix the permissions it's complaining about.

Getting my init file onto the system and in the right place has also proved to be more complex than it should have been (there seems to be two "HOME" directories, one under a Roaming title). The Windows shell has finally gained completion but it's still a shadow of a decent Unix shell. On the positive side I can already run eshell from within Emacs which provides a nice alternative to the command shell. I've yet to get tramp working though but I suspect that's just a case of getting ssh keys sorted out.

If there are better solutions to getting a decent Emacs set-up on Windows I'm all ears.

Switching buffers and Google+

2012-01-24T08:36:00+00:00

When you do so much of your work in one text editor the efficiency of switching between buffers becomes more important. For a long time I've had two bindings "C-x b" and "C-x C-b" which in days of yore I had bound to bs-show and a hacked up list-buffers that opened another window. These are broadly the "quick switch between working buffers" and "show me all the buffers".

For some time I've relegated bs-show to the longer binding and now use Stephen Bach's excellent Lusty Explorer which works really well when you know the name of the buffer and it's fairly unique. However when you've been going a while it can get un-manageable with a large number of open buffers, especially if you've opened second copy of a file from another source tree. This is what I would use the old classic bs-show for.

A few days ago I discovered ibuffer-mode with it's ibuffer-bs-show buffer navigator. Looking back through the Planet Emacsen history I can see it has been mentioned before and given it's been in Emacs since version 22 I'm surprised I hadn't cottoned on to it earlier.

One thing that might have put me off is the initial buffer list can be quite sparse. By default you only see buffers with files associated which misses out IRC, Edit with Emacs and *scratch* buffers. However hit "h" and you'll see there are a plethora of quick keys for chaning the view. A quick "//" and all filters are removed and you can quickly filter by different criteria. To get the most out of the mode you'll probably want to set up some custom filters ("/r<completing filter name">) to make quickly switching to groups of buffers easy. I currently have "work", "remote", "irc" and "logs" as filters. You can filter by name as well as major-mode.

ibuffer-mode does have the concept of Filter Groups although I'm not sure what they add on top of having normal filters which as far as I can tell can be arbitrarily complex. It also has some quite handy sorting and selection modes e.g. "sv" - sort by last viewing time. Given the amount of space the wiki devotes to the topic I wish I'd re-examined my buffer switching habits sooner. The change is already paying dividends for my productivity.

Before I go I thought I'd put in a quick mention of Google+. There is growing community of fellow Emacs users starting to post on it. One thing that attracts me to Google+ over Facebook (too data-miney) and Twitter (too short) is the concept of "Circles". It makes sharing geeky Emacs posts with people that might actually care easy while sparing them the flood of baby pictures I share with friends and family. If you'd like to follow me or my alter-ego please do mention Emacs in your profile or in a message so I can assign you to the correct circles.

Getting organised

2012-01-13T10:38:00+00:00

What with becoming a parent and getting promoted I suddenly find myself needing to become a lot more organised. Although I've been using org-mode for a bit I need to get a lot more organised with it.

Previously I had two sets of org notes. My personal set where sitting on my server which I could access via the terminal. I generally accessed this at home on the odd occasion when I was doing things like the annual round of insurance quote gathering. The second set was a fairly simple time sheet type affair that I was using at work to keep a vague track of where all my time was spent. The big missing part of this is when I'm on the move.

I've just recently upgraded my phone to the latest Galaxy Nexus which is a fine Google enabled device. I make no apologies for using Google's calendering and shared document services. They work very well and importantly allow me to share things with my wife who doesn't quite share my desire to run everything from a text editor. However for my personal task lists on the move and remembering what's coming up at work it doesn't quite cut it. Besides I like org-mode and I'd heard about MobileOrg so I endeavoured to set it up.

MobileOrg has been around some time for the iPhone but the mechanisms it uses for integrating with org-mode are fairly well documented. As a result there is a couple of Android implementations for it. Matthew Jone's mobileorg-android was the first version I tried.

The original sync method for MobileOrg was to use a service like Dropbox to sync files. Given the history of Dropbox's security I wasn't about to move my files into the proprietary cloud. The alternative is to enable WebDAV on my web-server and therefor enable two way communication via HTTP. It was a little concerning to see self-signed SSL wasn't supported as this does open up a potential attack vector on my machine. I've mitigated it a little by using digest authentication instead of basic-auth but I'd still prefer to be conducting these read-write operations over something more secure.

Initial results were a little underwhelming. After some messing around with the format of org-links I eventually got a basic outline summary up. Unfortunately I can't seem to sync notes created on my phone to the server. This seems to be a Apache problem which I shall have to dig into later.

After perusing the market some more I noticed there is a new project in town. Konstantin's MobileOrgNG was forked some time ago from Matthew's code and on installing I found it looked an awful lot better. I've still be unable to post any locally added notes (due to previously mentioned Apache config issues). However it's presentation is a lot slicker and it shows a lot of potential for being a good MobileOrg client.

I'm now stuck with a classic open source fork dilemma. The code bases look to have diverged enough that these two projects are essential going their own way. Looking at the two impact graphs it looks like they diverged around August 2011 and since then MobileOrgNG looks pretty much like a solo effort albeit with an impressive commit rate of new features.

So the questions for my readers. Which code base should I jump on? Has anyone got experience with the two different code bases and the reason they split? Are there any other Android clients for org-mode I should be looking at?

Edit with Emacs v1.10 released

2011-11-06T20:25:00+00:00

I received a bunch of feedback and patches from my last announcement but I think all the outstanding bugs are now squashed. The edit-server.el has seen some love to make it more idiomatically correct for elisp. The main change is new code to handle editable DIV tags beloved of such sites as Google+ (which you are welcome to follow me on, maybe I should have an elisp circle?).

A big cosmetic change is a brand new settings page which looks less like a web-form from the early 90's and more like part of Chrome. Alas I can take no credit for this but can thank Frank Kohlhepp's fancy-settings library. In fact a lot of the credit should go to third party libraries like jQuery and of course the growing list of contributors who have submitted code for merging.

So the final changelog for 1.10 is:

Extension

* Ignore textareas marked as read only
* Don't tag areas that are not visible
* General clean-up to use jQuery to find elements
* Explicit CSS for edit button to override page settings
* Handle editable DIV blocks (e.g. Google+)
* Optimise the finding of text areas for highly dynamic pages
* Revamp the settings page with "Fancy Settings"

edit-server.el

* Allow customisation of edit-server-default-major-mode
* Allow edit mode to be set by matched URL
* Tweak detection of MacOS X Emacsen
* Change behaviour of C-x C-s to save to kill-ring
* Persist the buffer-local variables beyond mode changes
* Setup keymap within defvar
* Clean-ups to code to be more idiomatic.

Call for testing for Edit with Emacs

2011-10-30T22:55:00+00:00

I've been doing a bunch of house-keeping on Edit with Emacs recently in preparation for a new release. I can only apologise to those people who have submitted patches and merge requests for my tardiness. I'm afraid Real Life ^tm has been taking precedence.

As regular readers of the non-emacs sections of my blog will be aware my recreational coding time is about to be severely curtailed. As it's been a while since the last release and a number of new features have been added it would be nice to get some wider testing. I therefor am hoping to elicit the help of the Emacs community to check out the code and give the tires a bit of a kick before I push out the final release to "the cloud". I'd rather not regress behaviour for the 1917 users who get Edit with Emacs from the Chrome store just before I disappear into a haze of nappies and parental responsibilities. The current changes over the last release are:

Extension

* Ignore textareas marked as read only
* Don't tag areas that are not visible
* General clean-up to use jQuery to find elements
* Explicit CSS for edit button to override page settings
* Handle editable DIV blocks (e.g. Google+)
* Optimise the finding of text areas for highly dynamic pages

edit-server.el

* Allow customisation of edit-server-default-major-mode
* Allow edit mode to be set by matched URL
* Tweak detection of MacOS X Emacsen
* Change behaviour of C-x C-s to save to kill-ring

org-mode and clocking in

2011-09-20T17:06:00+01:00

I've recently started using org-mode's time tracking to keep track of what I spend my time doing at work. This was in response to being asked by one of my managers what I spend my time doing and basically being forced to guess.

Setting up a clock page is fairly simple. It's then just a case of C-c C-x C-i and C-c C-x C-o on the appropriate sub tasks. Dynamic blocks can then be added to your org-document to generate weekly, monthly or annual reports based on the clock lines in the document.

So far the results have been illuminating and certainly shows how optimistic I can be in predicting how much time I spend doing my core job hacking on the code. However one thing that is captured but hard to summarise is interruption cost. I've taken to switching task every-time I'm interrupted in person or by phone call (I'm not counting IM/IRC as it's less disruptive). I can eyeball the raw data and see that some weeks are exceptionally bad for task switching. However what would useful is a break-down of mean and median clock lengths against each task to give some sort of indication of how much straight line hacking I've gotten done. I have a feeling the :formula and :formatter options could be used for this but I've been struggling to find any example. Does anyone do a similar analysis with their org-mode clock data?

UPDATED: fixed keystrokes.

Perils of bleeding edge

2011-07-27T16:55:00+01:00

I've taken to running the latest emacs from a source tree install. It works well enough and additional modes I use have been liberally ${VC} fetched into my .emacs.d. However there are still a number of packages I'd like to use from Debian's emacs version agnostic site-lisp directories. I came up with this:

 ;; Add site-lisp to search path
;
; This is a work-around function for when I'm running bleeding
; emacs from the source tree but still want Debian's developer
; tools. I'd caution about having too many extra packages about that
; have been merged into the source tree (cedet etc) lest it get
; confused.
(defun load-debian-site-lisp()
"Attempt to load Debian's site-lisp if it's there"
  (interactive)
  (when (and (not (member "/usr/share/emacs/site-lisp" load-path))
             (fboundp 'normal-top-level-add-subdirs-to-load-path))
          (let* ((default-directory "/usr/share/emacs/site-lisp"))
      (normal-top-level-add-subdirs-to-load-path))))

(load-debian-site-lisp)

Which seems to work well enough to give me my debian-changelog-mode back. However it's still not seamless as I have to manually (require 'debian-changelog-mode) before loading a changelog which forces the issue with local variables. I suspect I'll have to replicate the boilerplate that /usr/share/emacs/site-lisp/debian-startup.el does but I can't use because it doesn't degrade gracefully if no debian-emacs-flavour is defined. Suggestions for making this behaviour neater would be useful....

completion-ignored-extensions

2011-04-13T16:27:00+01:00

I've been using the rather spiffy Lusty Explorer for some time for my buffer and file finding. However it (I thought) had a rather annoying bug where I could never tab directly into some of the repositories I was hacking on. Eventually I figured out that the problem was down to the way I name them.

My naming scheme for any repository is generally to add the version control extension to the directory. This is more for my benefit than emacs as it's quite capable of working this out for itself in any given repo. However it turns out that Lusty was being a good emacs citizen and using the variable completion-ignored-extensions which included the pattern ".git/" and duly hiding my repo directories from completion. In fact there are a number of patterns in there which should probably be more specific. I wrote this to fix the problem:

(setq completion-ignored-extensions
      (mapcar '(lambda (ext)
                 (if (string-equal "/" (substring ext -1 nil))
                     (concat "^" ext)
                   ext)) completion-ignored-extensions))

It still seems a little ugly to me so given the last time I berated elisp's style gained so many useful suggestions I'd welcome improvements.

Edit with Emacs v1.9

2011-03-09T22:22:00+00:00

It's been a while so I thought I'd push out a new version of Edit with Emacsto the interwebs. Unfortunately I wasn't able to wrangle the edit-server.el to reliably handle keeping the current frame open for iterative editing. For now this feature is only available via the python server. Patches are of course welcome!

v1.9

Extension

* Support for iterative editing (python server only currently)
* Add context menu while in text area box
* Fix reference to Alt-E, it's actually Alt-Enter
* Trigger a DOM change() event when we update the text area

edit-server.el

* Actively encode responses as UTF-8

* Make edit-server-edit-mode a minor mode instead of a derived mode

2010 In Memorium

2011-01-02T15:25:00+00:00

2010 has been a pretty good year for us, it seems like is has packed a lot in.

Professionally I've been doing really well. The company I work for has had yet another record breaking year which has triggered my bonus again (although I don't find out how much until I get back). It continues to grow in both revenue and size. I finally feel comfortable with the code base I'm working with. At the same time the ever growing feature list means I've still got plenty of interesting things to do with it. Although the internal code is proprietary to the company I've got a fairly wide latitude to work with FLOSS code and it makes a significant portion of the NMS product. Pretty much anything that I hack on that is useful to the wider community outside the application specific task is fed upstream which is good for both the company and the wider world.

Aside from my work based hacking 2010 was also the year I became a real open source project maintainer. I've been publishing code I write and use for a long time and have done maintenance work on some niche projects as well as contributing to other projects as normal. However at the start of 2010 I released Edit with Emacs to an unsuspecting world. It started as a simple exercise in learning some Chrome Javascript and kicking me to delve deeper into elisp coding. However since then it has grown to something that gets regular contributions and is used by over a thousand people. To quote Blur it gives me an enormous sense of well being :-)

This year has also been an interesting year politically. While there is plenty of debate about the economics of some of the choices made by the coalition I'm personally fairly happy with the approach being taken given the fairly dire conditions they inherited. From my geek point of view it's heartening to see the pre-election commitment that was made to open data seems to be being lived up to. I'm hoping the changes to openness will get embedded into the way government does business permanently.

The big thing that dwarfed all other things this year of course was getting married to the wonderful Fliss. Saving up all my holidays for the honeymoon meant we didn't have many extended breaks and of course as the logistical tasks approached there was much running around and controlled panics. In the end everything went like a dream and the wedding itself was a blur of happy memories. It was a fantastic day and made all the better for all the friends and family that came along to share in the celebration. Getting married to Fliss is quite possibly the best thing I've ever done.

I do occasionally have pangs of guilt considering how lucky we have been especially when so many are struggling with the uncertainty of the economic situation and the stress that causes. All I can do is wish the best of lucky to everyone for the next year. I hope it exceeds the positive expectations and under-performs on the negative ones!

Almost there

2010-11-19T22:48:00+00:00

I thought I might just mention the current state of the development for Edit with Emacs. A number of useful contributions have come in but I want to be able the "incremental edit" feature in the elisp edit server. I had a go but got stuck (marvel at hack in progress). So if anyone fancies having a go be my guest ;-)

The rest is as follows:

Ryszard Szopa (1):
      Allow the server to send correct Unicode to Chrome.

gfxmonk (4):
      trigger a DOM change() event when textarea content has changed
      clarify that the shortcut key is Enter, not "E"
      only re-request edit when "x-open" header is "true", not merely present
      Made indentation consistent across all source files

A step towards reducing Emacs Pinky

2010-11-03T15:33:00+00:00

As someone who spends most of the day attached to their keyboard I'm fairly sensitive to RSI inducing things. One thing I do suffer from is what's colloquially known as "Emacs Pinky". As the Ctrl is the start of practically every editing command in Emacs it's the left little finger that spends most of it's time stretched out and after a long heavy coding session can be more than a little sore.

I've made adaptations including re-binding the useless Caps Lock key to being an alternative to Ctrl (although my use of it is sporadic). However one other culprit is the incredibly useful GNU Screen.

I've been running with the default bindings for some time which use the Ctrl-A sequence as the command sequence. This is especially troubling when you have multiple nested screens. Today I finally cracked and made the decision to break out of this bad habit. So starting today I've chosen the backtick (`) character for the escape sequence. This makes the normal (and most frequent) screen action of switching screens a two stroke press without having to hold any control key down. We shall see how long it takes to retrain my muscle memory.

Does anyone have any tips for reducing strain on the little finger inside Emacs itself? Sometimes I wonder if it would be easier to replace Ctrl actions with a double tap on the space bar before going for the modifier key. Is that even possible?

I found this entry on superuser useful for other screen config tips.

More on TRAMP

2010-10-29T12:10:00+01:00

I've mentioned TRAMP before. It's a very handy way of editing remote files. However I've been having a few problems with it today.

The most common access method used by TRAMP to access files these days is via ssh. It basically uses a remote terminal session to navigate the remote file system and fetch files back and forth. However it does take a few things for granted and one of those is the shell environment on the far end. For most *nix users this will be Bash. Long time command line hackers like myself usually have a rather heavily customised shell environment which can cause problems for TRAMP, especially if you pull tricks with customising the command prompt. In an effort to handle this a little better I modified my .bashrc to treat 'dumb' terminals as non-interactive. It works but it still seems a little hacky. I would welcome any better suggestions for detecting TRAMP sessions my start-up files.

I was still experiencing a problem navigating to one of the directories on my remote machine. Whatever I did when I got to the root of my code tree I couldn't navigate into the correct directory. The prompt would only offer me an old (since moved away) directory. Tricks like opening dired mode and navigating in would cause the session to spin consuming 100% cpu. In the end I tracked it down to IDO and the old directory existing in my ~/.ido.last file. Once removed and emacs restarted the problem went away. I'm hoping this is just a peculiarity of the interaction between ido and TRAMP but is was certainly annoying.

Edit with Emacs v1.8

2010-08-19T11:10:00+01:00

After a relatively quiet period a number of patches have flowed my way so I thought it was worth pushing out a new version. Perhaps the most "important" feature is the edit box flashing and fading from yellow after being updated (like It's All Text). It wasn't that hard to do given jQuery and the colour animation plugin do all the heavy lifting.

We've added a new hook to the edit-server for pre-edit customisation. If anyone has some nice examples of using the various hooks it would great if you could add examples at the emacs wiki.

As ever the extension can be found at the Chrome Extensions site. Development versions are hosted at github.

Full Change Log

v1.8

Extension

* Added option to enable/disable visual edit boxes
* Improved feedback as editable elements come in and out of focus
* Updated text box will now fade from yellow after an update

edit-server.el

* Added edit-server-start-hook for additional customisation when edit starts

Looping in LISP

2010-08-10T09:04:00+01:00

Loops are a fairly important part of any programming language and fairly fundamental to a language that is purported to be all about manipulating lists. However it's not something I use that often in my .emacs code so I thought it might be useful to discuss the various options with some examples.

Problem: I run emacs on a number of machines, each with a different set of sound sets. I want to set up set up a valid sound for erc but I don't want an overly verbose set of cases depending on what machine I'm on. Instead a given a list of sound files I want a function that will return the first one that actually exists.

This problem can be easily generalised into return the first valid path from a list of paths.

First version: pure emacs lisp

; the 'elisp' way
(defun find-valid-file-elisp-way (list-of-files)
  "Go though a list of files and return the first one that is present"
  (let (r '())
    (mapc '(lambda (f)
             (if (file-exists-p f) (add-to-list 'r f)))
          list-of-files)
    (car r)))

First impressions aren't good. The lisp parenthesis do seem to get in the way of making what is happening clear. However it's using one of common mapping functions you see a lot of in lisp. A mapping function essentially takes a list, applies a function to each element of the list and eventually returns a result. The most common of the mapping functions is mapcar which returns a modified list as a result. In this case that isn't what we want so we use mapc where the only value that is built up is the result r as we identify each valid file. The final return value is just the first entry in that list. This does mean we have processed the whole list of alternatives which is sub-optimal.

Second version: Common Lisp Version

(defun find-valid-file-clisp-way (list-of-files)
  "Go though a list of files and return the first one that is present"
  (loop for path in list-of-files
        until (file-exists-p path)
        finally return path))

This version probably is the easiest to read for people familiar with other programming languages. The intention of the code jumps out at you. However the actual implementation is done with a macro. If you look at the help for loop you'll see it can take a number of different forms - follow that to the code and you'll see a fairly complex elisp implementation. However to my mind still easier to follow than the pure elisp version with mapc.

Third Version: Using the dolist macro

; using 'cl-macs
(defun find-valid-file-dolist-way (list-of-files)
  "Go though a list of files and return the first one that is present"
  (dolist (f list-of-files)
    (if (file-exists-p f)
        (return f))))

This is yet another version using an LISP macro but this one has considerably less potential forms to cause confusion. It's fairly comprehensible what is going on and even follows the traditional parenthesis happy form. It also takes advantage of the common LISP return to early return from the loop when we detect a valid file. If it makes it to the end of the list it evaluates the 3rd optional form to calculate the result which in this case will be 'nil.

So what do you think? What version do you prefer? Where does the balance lie between writing code is LISPy ways and for code comprehension? Are there any other ways to solve this particular problem? I'll be looking forward to your comments.

Edit with Emacs v1.7

2010-07-29T15:31:00+01:00

It's been a while so I released a new version of Edit with Emacs for Google Chrome(ium). To be honest most of the changes are to edit-server.el. The most major change is moving all the frame configuration options into a single edit-server-new-frame-alist which might cause confusion if people don't read the change log which is:

Extension

* New icon state. Blue=Waiting, Green=In Progress, Red=Error

* Improved mouse-over text for icon to give more useful feedback

edit-server.el

* Move all frame customisation into edit-server-new-frame-alist
* Don't ask user before closing emacs and network process
* Just skip creating new network process if it's already running
* Make sure edit buffer is selected on new frames
* Prompt window manager to bring new frames to the top of the stack (X windows only)
* Enable multi-byte mode on edit buffers for better unicode handling
* Explicitly fail on XEmacs if make-network-process isn't found (XEmacs patches welcome)

The power of Tramp

2010-07-29T08:06:00+01:00

The project I've been working on at work the last few weeks has involved bringing up a brand new embedded system. As part of that I've been tweaking a lot of start-up scripts and configuration files on the new system.

For part of that task I've installed GNU Zile which provides a nice Emacs-like experience for quick edits where you would usually fire-up vi. Despite the fact the new platform has a considerably larger backing store I'm not going to install the full-fat Emacs just to satisfy my editing needs.

Fortunately the full-fat Emacs has long carried a feature called TRAMP (standing for "Transparent Remote (file) Access, Multiple Protocol") which is incredibly useful in this and other use cases. TRAMP allows you to edit files on remote machines in your normal home Emacs environment. find-file and friends all behave as if you are editing local files while behind the scenes Emacs pulls directory listings and files from the remote machine with ssh (or ftp or whatever actually works). I use TRAMP all the time to edit files on the rather ancient build machine rather the using the relatively ancient Emacs it has installed on it.

There was one minor wrinkle getting TRAMP working with my new embedded system. As some of the methods used to move files back and forth are designed for transferring text files Emacs goes to great pains to ensure the files are moved across un-mangled. Typically this involves some sort of encode/decode utility. The tramp code is actually quite a good example of using elisp in an extensible way and if you look at tramp-remote-coding-commands you'll see it will even fall back to running remote in-line perl if it has to. As it happens we don't have perl running on the embedded platform and as it would be a little overkill to put perl on the remote machine so I added the recode utility.

I consider TRAMP to be one of Emacs' killer features. Next time you sigh at the prospect of being consigned to an inferior editor on some remote box consider visiting the file with TRAMP straight from your current Emacs session.

Emacs Daemon and Handling Projects

2010-07-05T11:16:00+01:00

All my machines have been running persistent Emacs daemons for some time now. Every time I edit a web-page text area or a tweak a configuration file the Emacs client whizzes me to the file with efficient enthusiasm.

However I'm still running individual sessions for each "project" I work on. For example at work I currently have one for the main source tree, one for the standalone Java project and one for a kernel I'm working on. This is in no small part due to the rather basic project support I hacked up some time ago.

While conceptually switching my various project variables to be buffer locals would solve a lot of problems there would be a mess of smarts to improve on and I currently can't justify the diversion. I've attempted to get things like CEDET and JDEE working before but with limited success. They all seem overly heavy-weight solutions to the things I'd like it to achieve:

Associate new open files with the "project" they are in
Make M-x compile do the right thing:
- Select compile command from per-project compile history
- Separate compile window for each project
Make M-x find-tag behave in the project context

As I say simple needs. Once I have all that working then current pain points like having multiple IRC channels over multiple instances of Emacs can go away and everything is literately only a few key-strokes away.

Dear lazyweb, what lightweight project management tools exist out there and which are really good?

EDIT TO ADD: I'm going to have a look at eproject. Other suggestions still welcome though :-)

On the nature of Planets

2010-07-01T16:26:00+01:00

I upgraded my Wordpress install yesterday which inadvertently broke the 'emacs' tagged feed to Planet Emacsen. The aim of having a sub-feed from my blog was so I didn't pollute the planet feed with my rambling life story which is probably of more interest(?) to my friends and family. They tend to read the other syndicated feed on LiveJournal which has no filtering and as a result often generates confusion from my less technically inclined friends when I start talking about esoteric editor features.

However one of my non emacs related posts did generate a number of useful replies pointing me towards the useful autossh tool. This made me wonder if sticking to purely emacs related posts was being a little too limiting. I'd be interested in hearing from Planet Emacsen subscribers what they actually expect from the feed? I'll assume non-techie posts should be off limits but is there value in reading technical posts by emacs users that are not actually related to emacs? Or do people prefer the planet feed to be all about our favourite editor and the things we can do with it? Are we just a community of like-minded individuals with a penchant for octipedal key-strokes?

Learning to love the snake

2010-03-31T10:47:00+01:00

Yesterday I finished of what is hopefully the last testing release of the product I'm responsible for, assuming no major problems it will go gold soon. This means I can start on doing some new development work. The thing I'm working on next will be done in a scripting language as basically the task of ensuring configurations are correct and properly setup is a) not performance critical and b) a hell of a lot easier than wrangling strings in C.

While most of the scripting stuff on the product is written in perl I've been contemplating doing something a bit more major with Python. While the perl I write (example) probably looks more like C with extra punction marks compared to the code of a seasoned perl wrangler I still occasionaly get tripped up by it. In contrast my experience with Python so far has been much more pleasent, no doubt helped by the ability to experiment in an interactive python shell. So far it's behaviour has been unsurprising. We shall see how I feel about it after this piece of work :-)

Of course the first thing to do before starting out on this course is to see if my editor is configured and ready. I've immediately walked into the python.el and python-mode.el schism. I'm going with python-mode.el for the time being but at the moment I'm not doing much more than run iPython as a inferior shell within Emacs.

Adding Google Juice to mutt

2010-03-17T07:35:00+00:00

As I've been mailing out invites I discovered a minor problem with my data.

My main email client is the fantastically functional mutt. It's terminal based but incredibly flexible. When it comes to mass sorting/searching your email it leaves GUI based clients standing. However now I'm a roving around with a Google Phone the majority of my contact data is in the cloud. While I have a small address file used by mutt it only has a few oft-mailed addresses in it.

Luckily thanks to Google's data APIs your information is only a few RESTful requests away. The goobook program provides a handy mutt compatible address book interface to this cloud data.

There is one wrinkle however. The configuration of the script involves putting some rather valuable login details in a plain text file on your home partition. While I like to think my machines are pretty secure and maintained you can always do more. Good security is defence in depth. A quick patch later and I can store those details in an GPG encrypted file that can be decrypted on the fly when required.

The final piece of the puzzle is creating these encrypted config files in the first place. Although you can do this by hand from the command line I find the best method is using EasyPG (now part of Emacs 23). This will automatically cause any files with a .gpg extension to be encrypted. You can control the Emacs mode selection and default encryption key to use by using file variables in the header comments of the file.

It's not all perfect though, when enabling EasyPG I had to do the following:

(if (maybe-load-library "epa-file")
    (progn
      (setenv "GPG_AGENT_INFO" nil) ; gpg-agent confuses epa when getting passphrase
      (epa-file-enable)))

The problem seems to be that when GPG agent runs in terminal mode it confuses Emacs/EasyPG. By suppressing the GPG_AGENT_INFO environment variable EasyPG will fall back to requesting your passphrase in the mode line. While it takes care to flush the value as soon as possible it does open a small window of attack if an attacker can cause emacs to crash and dump core.

Thoughts on Java

2010-02-12T11:18:00+00:00

I've been spending the last week hacking around in Java. One of the components of the product I develop is the open source OpenNMS. As I've mentioned before it's been lightly modified by myself to blend in better with our code mainly in the JSP department for it's web interface. Last week I discovered I needed to make a slightly deeper change to the code to export some more data to it's scripting interface.

While I was exploring the code base and scratching my head at some of the exceptions being thrown I thought I'd have yet another go at making my Java development environment a little more integrated than binding the build script to "C-c c".

So far I have a couple of observations about Java code. The first is that it never seems to do very much. As you navigate the code base you tend to find a lot of simple skeleton classes, usually working as simple adaptors between one framework and the next. Thanks to Java's strong support for generics and interfaces you often find yourself looking at a class wondering what else it might do. None of this is helped by Java's standard/deep/directory/naming/strategy/of/doom. My usual tools of a decent programmers editor and grep start to show their limitations. Java is a language that is designed to be serviced by a full IDE to help you make sense of the whole system.

Having previously tried and failed to get JDEE working I thought I would capitulate and try Eclipse. I blithely thought that given the fact it's the standard Java developers tool with a long development history it would Just Work (tm). I was to be sorely disappointed.

I tried several different versions on both my home Gentoo machine as well as various PPA based versions on my work Ubuntu setup. A lot of times it failed to start up due to some Mozilla dependency issues and when I did finally get it started I couldn't get any of the plugins to install. The entire Eclipse stack is designed around the concept of plugins and it seems every Java framework comes with it's own additional plugin for Eclipse. Without the plugins I wasn't able to get a working build or any sort of source level debugging. Eventually I had to concede defeat and bin the IDE approach and return to some tedious grep work and piece together the structure of the software by hand so I could finally crank out what in the end was a fairly simple patch. I doubt I shall return to Eclipse.

I did have a brief tinker with JDEE which at least installed better from it's SVN repo (needing a one line patch). However the documentation is a little sparse on how to import an existing large project into it's view of the world. Annoyingly potentially useful links like the FAQ referenced on the main site lead to dead ends. I never got to the point of seeing if I had a working Java GUD integration. I hope to revisit JDEE in the future when I'm a little less pressed for time at work. It's hard to work up the enthusiasm for plumbing in support for a language you never intend to use for fun in your own time.

Finally public

2010-01-25T12:41:00+00:00

After much faffing about with repeated sending of faxes to multiple numbers I finally proved I was human and accountable enough for "Edit with Emacs" to appear on the Chrome Extension Gallery. This almost immediately showed up some documentation and usage usability issues so I spent some of my spare time at the weekend creating an options page. I've also got a growing number of feature branches coming in from other github users so I merged some more contributions into the v1.4 release.

I have to say the usage of git as an SCM tool as well as github's non-fussy functional website makes managing contributions a lot easier. It's nice to see the world has moved on since the venerable SourceForge was the only option for those who didn't want to bother maintaining their own project infrastructure.

One of the feature branches being proposed is changing the interface to the "Edit Server" to pass a richer set of information about the text area being edited. This would allow the server to do clever things like position the frame near the position of the text area on the browser and possibly manipulate fonts. So far I'm trying to keep the current edit server calling conventions similar to those offered by other similar extensions that need to do a similar thing. Although for Emacs users edit-server.el will surely be the default and most used method it seems churlish to break compatibility for those that prefer to run/hack other servers. I've been documenting the URL conventions so it would be nice if I got some feedback how to maintain a useful extensible "API" for the broadest range of solutions.

Today's Ubuntu Emacs PPA tip

2010-01-19T10:56:00+00:00

Like many Emacs users I use the excellent Ubuntu Elisp PPA to get a more recent Emacs than the main distro repos package. However I'd been puzzled as to why emacs-snapshot hadn't been updated in ages. The was doubly confusing as the orebokech packages on which they are based and I use on my server had seen a couple of version bumps.

Turns out emacs-snapshot is no longer packaged by the PPA, but emacs23 is. Once I'd installed that my little (make-frame-on-display) problem went away.

First elisp patch for emacs chrome

2010-01-09T12:40:00+00:00

One of the nice things about putting your coding experiments up early and under a permissive license is people can submit patches. I'd been trying to get the native elisp edit server working but I'd fallen back to the working python script as I've been busy at work. However along comes Riccardo Murri who cleaned up the code and got it working. I've pushed the changes to the repo and with a little extra glue spawn it when emacs is in --daemon mode:

;; Do we want an edit-server?
(if (and (daemonp) (maybe-load-library "edit_server"))
    (edit-server-start))

And bingo you save the cost of 2 unnecessary fork/execve's to get emacs editing your Google Chrome(ium) hosted <textarea>. It seems to have an odd interaction with longlines-mode though which I need to investigate, perhaps longlines mode does magic stuff with the buffer text on save which gets skipped when the edit server does it's thing?

I've submitted the extension to the Google extension gallery although it's currently held up in actual physical paperwork. As my content script needs fairly liberal permissions to work I have to at least demonstrate I'm not an evil hacker/spammer intent on creating browser malware. I may take advantage of the delay to some clean ups to the browser feedback as we will likely be housebound sheltering from the weather. That is of course unless anyone else beats me to it :-)

Update on Emacs Chrome

2009-12-21T13:49:00+00:00

In my spare time I've been mostly bashing away at trying to implement an edit server for the Emacs Chrome extension in Emacs Lisp. It's taking longer than I hoped mainly as it's the first time I've ever tried to use the Emacs Lisp Debugger and it's fairly alien compared to the usual functional GDB's of the world which I'm used to. The current version should be pushing canned response back to every edit request but for some reason it's not working. Once the basics are working the rest is just cleanup :-)

In the meantime I've pushed a few additional updates to the repo. The first makes the extension usable on multiple tabs by passing the "port" back to the XmlHttp handler. You would think being an Emacs user I'd be used to the first class function paradigm by now but it's not something I really get a chance to use much in the day job. I'm not sure if:

function contentTalking(port)
{
    port.onMessage.addListener(function(msg, port) {
        handleContentMessages(msg,port);
    });
}

is the correct way way to pass parameters from the local function's scope to the listener function's scope but I'm not sure how else you'd do it. Certainly the pattern of declaring functions in-line seems to be very common in the world of Javascript (as well as Emacs Lisp via the lambda () directive). If you look at the earlier version of the code you'll see my C habits come through, after all it's just a pointer :-).

I've also tweaked the pycl.py edit server code to handle running on Python 2.5 as my work box is running a fairly old Hardy Heron. Anyway the latest results of my hacking can, as always be seen on github.

EDIT TO ADD: Well that seems to be working now. I was getting confused in my use of car/cdr which is probably a result of too much Forth when I was younger. They are not equivalent to grabbing the first and next bits off the stack. Specifically cdr returns a list, so you had better munge it to what you want if it's not a list.

End to End Connectivity

2009-12-16T23:22:00+00:00

We had a slight degree of excitement this evening when we got home. All the power was out and the refrigerating appliances were slowing defrosting. Due to the randomness of the breakers tripping we thought it was the boiler. As it happened it was a earth-neutral leakage that was causing the craziness. Once that was all sorted out I sacked off doing the washing-up to do a little more tweaking.

I'm almost there with the Chromium extension. I based it on the work done by David Hilley and basically further hacked it about to make it work more like It's All Text on Firefox. I'm still using David's Python 2.6 shim layer to handle the edit requests until I can get around to implementing an elisp one for Emacs. Don't let my intention to do write one stop any readers from doing so in the meantime should you wish :-).

It works if you stick to one page, currently it gets confused if you have multiple pages open which have sent edit requests. This is because the xmlcomms.js function which handles farming out the edit requests has a single reply_port parameter which will point to the port of the last page to connect to the master extension. I'll need to come up with a slightly neater solution but I'm fairly crusty on Javascript.

Hopefully I'll get around to packaging it up properly in the next few days. In the meantime you can run it from a git checkout by pointing the Extensions page at it via "Load Unpacked Extension".

Still it's my first Chromium extension so I feel quite proud of myself :-)

Chrome and Emacsclient

2009-12-12T13:24:00+00:00

I had a number of things planned to do today, one of which was to go pick up some parcels that the local delivery service failed to do. However some workmen started digging up the road opposite my house. As the un-controlled single lane alternating between inbound and outbound traffic is right outside my drive I decided I couldn't face the horror and stayed in to tinker.

I've switched my default browser settings to Chromium on all my machines. It's just so much faster to launch and get around. I've sacrificed the security of NoScript but I'm less worried about Flash on my netbook now as each tab kills any CPU sucking plugins when deleted. The only major problem left is finding an alternative to It's All Text so I don't have to fire up Firefox every time I post something to the web (like this blog entry).

I decided to have a look into what was involved in getting a Chrome extension running that can call an external editor. I checked the group before I started and stumbled on Dave Hilley's rough cut solution. It's almost exactly what I need so I've started toying with it.

Due to the way Chrome works it doesn't seem possible to spawn additional processes from Chrome itself (unless you want to use the NSAPI sledgehammer). Dave's solution is a simple server script that the Chrome extension can make calls to using XMLHttpRequest's. The server is written in Python but my first thought is there is no reason why the local emacs --daemon couldn't terminate the calls directly. Certainly the example echo server on the Emacs Wiki doesn't look too complicated to hack up.

The other thing that could be improved is the detection of text areas. Currently the Javascript just searches the DOM for text areas and selects the first one as a candidate for editing. I'm fairly sure it would be possible to detect either which text area currently has focus or do something more akin to It's All Text and wrap unique <div>'s around each text area and have a edit content button.

Unfortunately I seem to be having trouble getting the Chrome side to work. Trying to set breakpoints in the extension scripts keeps causing the extension to "crash" so it's hard to follow the sequence and tweak it. I also have to wrap my mind around Chrome's distinction between "content" and extension scripts. Still at least it's something to work from. Of course if anyone else wants to jump in with ideas (or even better a pointer to a better piece of emacs server code for handling XMLHttpRequests) then don't let my stumbling hack attempts hold you back :-)

Post Travel Slump

2009-12-07T13:51:00+00:00

I got back from my Hungarian adventures at a slightly more reasonable time than Wednesday mornings departure. Never the less I find travel still zaps the energy out of me. On Friday night I managed to eat dinner before crawling upstairs and passing out.

Saturday has been ably documented elsewhere and basically involved a large amount of DVD watching. I was twiddling with my nascent Android application in the background but until I can make sense out of JDB and it's associated emacs GUD bindings it's going to be slow progress. Once I've figured out how to make breakpoints actually work as well as show position in the Android base code hopefully things will progress faster.

Sunday involved much de-construction as we took down the metal fencing at the back of the property. We are planning on replacing the boundary with more aesthetically pleasing (and fruitful) bushes. Once that was done I was minded to have a final push at getting my playlist patch into Rockbox. I also did my bit testing another outside contributors better solution to my stale hacky m4a mdat patch. I can appreciate Juliusz' frustrations with getting things into Rockbox. It's a great project and it's amazing we have truly free firmware we can install on a variety of bits of hardware. However it's by far and away one of the hardest communities I've had to deal with as a casual contributor. I don't know if the project suffers as a result, it's one of those hard to quantify things. I suspect the bar is pretty high anyway as the embedded nature probably rules out a lot of people used to desktop development unless they are really keen to dig into and learn the code.

What's in your Browser?

2009-11-26T11:16:00+00:00

The official Chrome Extensions page is coming and soon I'll be seriously considering making a switch to my default browser setting. While Chromium (the open source component of Chrome) is a fantastically speedy, low memory and nimble browser it's lack of extensions if the main thing holding me back from making the fully committed switch. I already use it to access my various Google applications and it's also useful for accessing last.fm radio as when the Flash plugin goes mad I don't have to kill my entire browsing session. With the discovery of the Greasemonkey based AdSweep I can even move my Google Reader browsing across to Chromium without suffering from animated banner induced madness.

However Firefox still has a number of killer Add-On's which would need to be ported before I can throw the switch. These include:

Firebug

Firebug has become an indispensable tool as I've done more JavaScript based hacking. I literally would not attempt to develop anything client side scripting based without this tool by my side.

It's All Text

This simple tool allows me to spawn an Emacs client to edit any text area on a web-page. It's what I use when I'm making blog posts and pretty much any non-trivial amount of web based text entry. While it might be tempting to do the Greasemonkey treatment to something like Ymacs to get something close to the Emacs experience there is really no substitute for the real thing. Especially, as you may have noticed, when you want to insert code snippets into your posts.

<h3>It's All Text</h3>
This simple <a href="http://trac.gerf.org/itsalltext">tool</a> allows
me to spawn an Emacs client to edit any text area on a web-page. It's
what I use when I'm making blog posts and pretty much any non-trivial

No Script

The Web 2.0 experience is certainly a good thing, browsers have certainly moved from being a presenter of static pages to being a platform where all sorts of fantastic stuff can be done. However I still like to have control over who gets to run code in my browser.

I used to have Bug Me Not in my list of must-have extensions but to be honest I just skip stuff that requires registration these days. I'm looking forward to Murdoch's content dropping off the search engines radar when he finally learns to use robots.txt.

So are there any extensions essential to your Firefox experience? Or have you already made the jump to Chrome(ium)? Or do you still see something in Opera that I never could?

Thinking about email

2009-11-17T12:49:00+00:00

I saw a post on my feeds today discussing a new email client called notmuch which set me thinking about my own use of email.

I have three primary ways of accessing email at the moment. On my domain I run Mutt in a screen session. At work I have the GUI behemoth that is Evolution and most of my mailing list subscriptions run through a Gmail account I have set aside for the very purpose.

I'm quite happy with Mutt for my main personal email. As I don't use my personal email to subscribe to mailing lists the volume is low, it's fairly quick and snappy to write email (in an emacsclient naturally) and searching is only a "/" away. However it is intrinsically a folder based email client which works fine as I only have two folders: Inbox and Oldmail. I periodically (~annually) move email into Oldmail when Mutt starts to slow down on start-up.

The main bugbear is my work email. I mainly run evolution through inertia and 2 features:

It connects to the work Exchange server, although calendering doesn't really work properly as I can't Accept/Decline meeting invites.
The vFolder concept allows me to easily separate company emails from the bug tracker and CVS commits and the odd work related mailing list while still offering a quick eyeball indication of what pending emails there are.

Other than those features Evolution sucks balls in every other way. It takes an age to compose a new email, I have to shut it down every night to avoid swap death and it's certainly not keyboard operable.

The GMail account I have to live with as there isn't really anywhere else I can easily index and search the 4gb (and growing) of mailing lists I subscribe to. Of course it is accessible via IMAP so it would be nice if I could access it from the same client, especially as the web interface tends to munge in-line patches.

The obvious choice would be to look into one of the many many modes for handling email in emacs. However the plethora of choices is one that makes selecting one to try even harder. Some people at work have fetchmail tasks setup to mirror the email on the server which sounds a little clunky but I guess must work well enough. I don't think it would be practical for Gmail though, I usually just browse through various tags for mailing lists, including the very handy 'mythreads' which marks any email list thread I'm involved in.

I think the key feature I need will be something like Evolutions vFolders for the quick eyeball along with a decent way to interact with Gmail. Any suggestions?

Avoiding RSI one keypress at a time

2009-11-04T12:04:00+00:00

Much as I love emacs some of the key combinations it expects of you to a) remember b) use are don't help when your trying to ameliorate the effects of RSI. As I spend an lot of time in compilation mode the following quick win helps:

(define-key compilation-mode-map (kbd "n") 'compilation-next-error)
(define-key compilation-mode-map (kbd "p") 'compilation-previous-error)

If you try executing (self-insert-command) the buffer would complain it's read-only anyway. I took this single keypress approach to the next level when I was doing a lot of patch merging and tweaked diff-mode to be a little less multi-finger.

Migrated to Wordpress

2009-11-02T11:34:00+00:00

I've finally gotten around to migrating the blog to Wordpress. While I'm still very happy with Personal Weblog it was getting a little long in the tooth and given Livejournal's ongoing problems with syndicated feeds I thought it would be easier to host everything (including comments) on my site. You can see the very ugly migration script on github.

The one remaining thing to do is code up some re-directs in the original news.php page so the syndicated feed still slurps up something, but there is no rush I guess seeing as LJ isn't currently paying attention.

EDIT TO ADD:Well LJ may not be paying attention but I've implemented the redirect for both normal pages and RSS feeds. Old permalinks should continue to the old page.

Should have been obvious...

2009-09-15T13:04:00+01:00

I recently turned on longlines-mode for my It's All Text buffers as that is generally what I want (especially for this blog). However when I'm editing email for GMail I want to be able to switch between modes as I may be writing a simple message or inserting a patch which shouldn't get mangled.

It took me a bit longer than it should because I was scratching my head trying to work out how to tell if a mode was on. Once I actually read the code it was obvious there is both longlines-mode the variable and (longlines-mode) the function, hence this tweak. I guess it's an idiom enforced by the define-minor-mode macro but I have to admit there is some unfamiliar syntax which I didn't follow. I shall put exploring the elisp macro syntax on my list of things to do once the house is sorted.

Updates and clarifications

2009-08-26T13:19:00+01:00

My previous post regarding the frustrations of a slow frame update when emacsclient was spawning frames wasn't helped by a gotcha I've walked into a couple of times regarding the if function. In my haste to debug problems I'd added a message in some code:

 (if I-am-emacs-22+
+    (message "loading color-theme at %s" (current-time-string))
     (if (maybe-load-library "color-theme")

As my eyes are used to scanning indentation and considerably sparser use of {}'s I never twigged that I'd completely changed the meaning of the code. As C-f h if will tell you:

if is a special form in `C source code'.

(if cond then else...)

If cond yields non-nil, do then, else do else...
Returns the value of then or the value of the last of the else's.
then must be one expression, but else... can be zero or more expressions.
If cond yields nil, and there are no else's, the value is nil.

If we ever want to have multiple line of functions in the "true" case you always need to wrap them up using something like progn:

(if "true"
    (progn
      (message "It's true")
      (message "I wanted to say something else"))
  (message "No it's not"))

Of course this makes things a little messier (even more parenthesis!) and you'll notice a lot of LISP code strives to be as dense as possible. However it's a worthwhile gotcha to keep in mind if your just starting with LISP.

In the meantime I've made a few changes to my .bashrc_emacs when I realised having emacsclient called with -n makes no sense in EDITOR and VISUAL environment variables. These variables are typically used by tools such as crontab or git which get confused when the editor they spawned for a user input returns straight away.

I also solved (or at least mitigated) the slow drawing of new frames by caching the colour theme so it doesn't get set every time I bring up a new frame. Notice the use of progn in the code ;-)

(defvar my-last-theme 'nil
  "Last color theme we set")

(defun my-color-theme-set (theme)
  "Set colour theme but don't bother if we already have"
  (if (eq my-last-theme theme)
      (message "my-color-theme-set: theme already set")
    (if (fboundp theme)
        (progn
          (funcall theme)
          (setq my-last-theme theme)))))

(defun my-set-tty-colours ()
  "Set the colours for tty mode"
  (my-color-theme-set 'color-theme-midnight)
  ; some tweaks
  (set-face-attribute 'show-paren-match-face nil :weight 'extra-bold)
  (set-face-background 'region "blue"))

(defun my-set-x-colours()
  "Set the colours for X windows mode"
  (my-color-theme-set 'color-theme-gnome2))

(defun my-new-frame-colours(frame)
  "Set the colour scheme of a new frame"
  (if (frame-parameter frame 'tty)
      (my-set-tty-colours)
    (my-set-x-colours)))

; Lets hook into the frame function
(add-hook 'after-make-frame-functions 'my-new-frame-colours)

The curious case of (make-frame) delays

2009-08-22T16:32:00+01:00

I had a problem on my home machine with the way emacs created new X frames. New frames would appear but then pause for 4-8 seconds before I could type any text. My question to the mailing list was met with a comment to get a faster graphics card. This seemed a little silly so this morning I put on my debugging head and tried to figure out what was going on.

First up was OProfile which confirmed that the problem wasn't CPU bound anywhere in the system. Then I threw the Emacs Lisp Profiler at the problem:

(load-library "elp")
(elp-instrument-function 'make-frame)
(elp-instrument-package "frame")
(make-frame)
(elp-results)

This confirmed that the "make-frame" function was indeed taking between 8-12 seconds to complete. However none of the other frame functions where taking that long. Time for another tack and breaking out strace. This did show that emacs was getting a lot of -EAGAIN's while reading from a Unix socket. Unfortunately it seems impossible to find out what the other end of the socket is connected to on Linux. I had some guesses but couldn't narrow it down.

Then I started emacs with "-q --daemon" so I could use Dope to profile by .emacs. A quick test showed that new frames popped up straight away before my init file was loaded. This really should have been the first thing I checked as often configurations can break things.

After some radical M-x comment-region surgery on the .emacs I found that unsurprisingly my display hacks seemed to be the culprit. I wasn't sure why so I started adding some (message) calls to the various functions to see if I could track down the delay. After instrumenting I found the problem had gone away which was very unexpected. In fact I tracked down the minimum diff to the following:

diff --git a/dotemacs b/dotemacs
index 55609aa..6948e3c 100644
--- a/dotemacs
+++ b/dotemacs
@@ -511,9 +511,11 @@ on the command line"
 ; First we need the colour-theme package

 (if I-am-emacs-22+
+    (message "loading color-theme at %s" (current-time-string))
     (if (maybe-load-library "color-theme")
        (if (fboundp 'color-theme-initialize)
-           (color-theme-initialize))))
+           (color-theme-initialize)))
+    (message "loading color-theme done at %s" (current-time-string)))

 ; Define the default frames sizes, shouldn't apply to a tty invocation
 ; TODO: handle remote sessions better, probably by probing remote X

This doesn't make any sense to me. For starters the colour-theme library is only loaded on start-up not for every frame. Besides why would an additional message change this behaviour? Have I just triggered some sort of weird racyness? And what an earth could this have to do with all the -EAGAIN's I saw emacs suffering from when talking to the unknown Unix socket?

One additional interesting data point from my investigation. Without the change the syscall count for a new frame looks like this:

11:22 alex@danny/x86_64 [elisp.git] >strace -c -p 6369
Process 6369 attached - interrupt to quit
^CProcess 6369 detached
% time     seconds  usecs/call     calls    errors syscall
------ ----------- ----------- --------- --------- ----------------
 42.11    0.000435           0      6692           writev
 40.08    0.000414           0     13674           poll
  9.68    0.000100           0     16940     10341 read
  4.55    0.000047           0      6637         1 rt_sigreturn
  3.58    0.000037           0      3700      3696 open
  0.00    0.000000           0         4           close
  0.00    0.000000           0         4           stat
  0.00    0.000000           0         2           fstat
  0.00    0.000000           0        17           brk
  0.00    0.000000           0        72           rt_sigprocmask
  0.00    0.000000           0       269         1 select
  0.00    0.000000           0         1           setitimer
  0.00    0.000000           0        41           kill
  0.00    0.000000           0         1           uname
  0.00    0.000000           0        10           getdents
------ ----------- ----------- --------- --------- ----------------
100.00    0.001033                 48064     14039 total

And with the change we get this:

11:24 alex@danny/x86_64 [elisp.git] >strace -c -p 6441
Process 6441 attached - interrupt to quit
^CProcess 6441 detached
% time     seconds  usecs/call     calls    errors syscall
------ ----------- ----------- --------- --------- ----------------
100.00    0.000040           0      1189           poll
  0.00    0.000000           0      1135       706 read
  0.00    0.000000           0       268       264 open
  0.00    0.000000           0         4           close
  0.00    0.000000           0         4           stat
  0.00    0.000000           0         2           fstat
  0.00    0.000000           0        16           brk
  0.00    0.000000           0       114           rt_sigprocmask
  0.00    0.000000           0       433         2 rt_sigreturn
  0.00    0.000000           0       445           writev
  0.00    0.000000           0       140         2 select
  0.00    0.000000           0         3           setitimer
  0.00    0.000000           0        18           kill
  0.00    0.000000           0         1           uname
  0.00    0.000000           0        10           getdents
------ ----------- ----------- --------- --------- ----------------
100.00    0.000040                  3782       974 total

It's all very curious.

EDIT TO ADD: Doh! It's all the fault of color theme. I was spending so much time being confused why the (message) patch changed everything and it's obvious when you look at it cleanly. The first (message) passes the (if I-am-emacs-22+ stanza and the else leg which loads color-theme is never executed. It then become very clear where the delay is coming from when you profile color-theme:

(load-library "elp")
(elp-instrument-package "color")
(elp-instrument-function 'modify-frame-parameters)

Function Name                                   Call Count  Elapsed Time  Average Time
==============================================  ==========  ============  ============
color-theme-gnome2                              1           6.949076      6.949076
color-theme-install                             6           6.9489930000  1.1581655000
modify-frame-parameters                         28          6.8929370000  0.2461763214
color-theme-install-frame-params                6           6.8865859999  1.1477643333
color-theme-blue-gnus                           1           1.707156      1.707156
color-theme-salmon-font-lock                    1           1.06505       1.06505
color-theme-blue-erc                            1           1.045608      1.045608
color-theme-salmon-diff                         1           1.041382      1.041382
color-theme-blue-eshell                         1           1.040324      1.040324
color-theme-install-faces                       6           0.062013      0.0103355
color-theme-spec-compat                         218         0.0012549999  5.756...e-06
color-values                                    21          0.000656      3.123...e-05
color-theme-filter                              18          0.000298      1.655...e-05
color-theme-install-variables                   6           9.7e-05       1.616...e-05
color-theme-plist-delete                        58          9.099...e-05  1.568...e-06
color-theme-alist-reduce                        12          3.9e-05       3.25e-06
color-theme-canonic                             6           3.700...e-05  6.166...e-06
color-theme-alist                               12          1.100...e-05  9.166...e-07
color-theme-variables                           6           9e-06         1.5e-06
color-theme-frame-params                        6           5.999...e-06  1e-06
color-theme-faces                               6           5.999...e-06  1e-06
color-theme-spec-filter                         1           2e-06         2e-06

And it's very obvious that the extra eye candy is expensive, especially when applied to every frame.

Using daemon mode

2009-08-09T13:23:00+01:00

Now the Fedora 11 has pushed Emacs 23 into it's testing repository I can now have 23.1 goodness on all the machines I regularly interact with (although having checked my Gentoo box is still 23.0.96.1, must fix that). I've previously implemented rather painful hacks to make a --server mode work well but now the mainline supported --daemon mode is the new shizzle. I had a quick play with my .bashrc shell scripts and came up with:

#!/bin/bash
#
# Alex's .bashrc_emacs
#
# Emacs Specific setup
#
# There is only one editor (although I can get to it in different ways).
# For most stuff I want to use emacsclient to spawn a quick shell and
# for emacs 23 I want to ensure the daemon is always running for the user.
#
# Luckily this is covered by specifying -a '' which will spawn a daemon if
# one is not running
#

if [[ "$DISPLAY" == "" ]]; then
    # Can we use muti-tty?
    emacsclient --help | grep "\-\-tty" > /dev/null
    if [[ "$?" == "0" ]]; then
        # Thats a yes
        EMACS_CMD="emacsclient -a '' -t"
    else
        # Hmmm, opening in another pane would be a pain?
        EMACS_CMD="emacs -nw "
    fi
else
    # otherwise don't wait and open a new frame
    EMACS_CMD="emacsclient -n -a '' -c"
fi

# Set the environment variables for the editors
export EDITOR=${EMACS_CMD}
export VISUAL=${EMACS_CMD}
export ALTERNATE_EDITOR=emacs

# shortcut
alias ec="${EMACS_CMD}"

# And finally lets get the status of the emacs server
DT=`emacsclient -a '' -e "(server-running-p)" 2> /dev/null`
echo "loading .bashrc_emacs (server-running-p)=${DT}"

The "emacs -nw" fallback is a little redundant but I suppose it's worth keeping for the day someone gives me shell access without the latest goodies :-)

longlines-mode

2009-06-16T15:06:00+01:00

I use a fantastic little Firefox add-on called It's all Text to spawn an Emacs client when editing text boxes. However up until recently I kept having problems with formatting of my posts. Even though HTML doesn't really care about white space most blog software tends to treat them as though they mean something. This can lead to some ugly formatting because hidden HTML elements still count towards auto-fill-mode when flowing text. Yes I can turn off auto-fill mode but then lines wrap at the full width of my emacs window.

Enter longlines-mode which still wraps the text pleasingly on my display but only puts the real "hard returns" in the final text. As a bonus it even shows me the real returns as special glyphs.

; I don't want all text-mode stuff to be auto-fill as editing text
; boxes can screw with the formatting (especially if html is involved)
(add-hook 'text-mode-hook
          '(lambda ()
             (if (string-match "itsalltext" (buffer-file-name))
                 (progn
                   (message "enabling long lines for itsalltext")
                   (longlines-mode 1)
                   (longlines-show-hard-newlines))
               (turn-on-auto-fill))))

I can finally remove the horrible /r /n gobbling hack in my blog script and post code snippets formatted the way I want them :-)

Getting it

2009-06-08T22:50:00+01:00

Apparently according to href="http://c2.com/cgi/wiki?LispUsersAreArrogant">some learning
Lisp is a path to programming enlightenment. Although I'm not in that
school (elisp is about a better editor for me, not a way of life) I
did have a slight light-bulb illuminating moment today.

One of the jobs I have at the moment is maintaining some href="http://en.wikipedia.org/wiki/Simple_Network_Management_Protocol">SNMP
agent software. It's written in C with a hand hacked GTK GUI and very
sparsely documented. One of the tasks has been converting a bunch of
the code to abstract some of the SNMP tables into proper structures
which is a massive improvement on the dense mixture of sparsely commented OID
parsing/GUI update code. However I caught myself Copy & Pasting a lot of the
boiler plate code because most of it is identical apart from the
actual object parsing which looks like:

if (snmp_oid_compare(oid, oid_entry_A))
{
    obj->valueA = oid->val->integer;
}
else if (snmp_oid_compare(oid, oid_entry_B))
{
    obj->stringB = g_memdup(oid->val->string)
}

Clearly something has gone wrong when you find yourself copying and
pasting code. Really it should be possible to describe the mapping in
a data structure and have one common set of code for iterating a
table. I started wondering how such a structure would work:

myMagicMappingTable[]= {
    {oid_a, INT, offset(&magicObj->objA)},
    {oid_b, STRING, offset(&magicObj->objB)},
}

Well that would work OK, except the munging code isn't quite all that
boiler plate. The existing code isn't all straight mapping of SNMP
types to the C structure. Also using offsets into anonymous structures
is the sort of old-school C trick that one should try to avoid in
maintainable code.

As I thought on I considered a formulation involving C function
pointers in the table. This would have involved a lot of typing to
declare each function snippet before some shady casting tricks could
be pulled. "What I really need", I thought, "is some way to add
anonymous functions to my data structure". That's when the light bulb
went off and I recalled the meaning of href="http://en.wikipedia.org/wiki/First-class_function">first class
functions and the world made a little more sense.

So I can't recode this application into Lisp (or even elisp) but as an
exercise I coded up an approach to the problem in emacs to see if
things are making more sense. I ended up with:

(setq snmp-results (list
                    (list ".1.2.3.4.5" "value A")
                    (list ".1.2.3.4.6" "value B")
                    (list ".1.2.3.4.7" "value C")
                    (list ".1.2.3.4.8" "value D")))

(setq magic-object (make-hash-table :test 'equal))

(setq magic-map   '(
                    (".1.2.3.4.5" . (lambda (val obj)
                                      (puthash "thingA" val obj)))
                    (".1.2.3.4.6" . (lambda (val obj)
                                      (message "skipping thing B")))
                    (".1.2.3.4.7" . (lambda (val obj)
                                      (puthash "thingC"
                                               (format "value is %s" val)
                                               obj)))))


(let ((results snmp-results))
  (while results
    (let ((pdu (pop results))
          (magic))
      (setq magic (assoc (car pdu) magic-map))
      (if magic
          (funcall (cdr magic) (cdr pdu) magic-object)
        (message "Couldn't map:%s" pdu)))))

(maphash (lambda (k v) (message "k:%s v:%s" k v)) magic-object)
=>
 k:thingA v:(value A)
 k:thingC v:value is (value C)
 nil

And that code makes sense to me. I wonder if that means I'm starting to "get lisp"?

Messing about with emacs server

2009-06-02T17:11:00+01:00

I've been messing around with some weird-ass behaviour with my emacs-server. I had some code in it to deal with automatically opening a fresh frame (while in X) which I had plugged from the emacsclient wiki page. In one of my more recent updates the bleeding edge emacs I run on most (but not all) of my boxes gained -c option to create a new frame for editing.

This caused all sorts of confusion and I started seeing inconsistent behaviour between invocations of emacsclient causing new frames to maybe appear or multiple frames being created. A bunch of (message)'s latter I started making the code more defencive and detecting the case when the current frame was the original server frame.

It was at this point the inconsistent behaviour disappeared and I was able to reliably bring up a fresh focused frame for everything I was editing. Calls without the -c parameter will cause the code to spawn a new frame automatically although it doesn't benefit from correct workspace focus -c will give you. I can only think that the code as it was would break at some subtle point (I'm guessing the server-edit-hook) and at that point not be able to recover. Still the changes I've made do seem to have improved things for now. Now all I need to do is test it on the other boxen with emacsen.

Desktop Save Mode

2009-06-01T18:56:00+01:00

While browsing stackoverflow I (re)discovered Desktop Save Mode which I must of played with in an earlier emacs version and then disabled. My only gripe at the moment is watching emacs load everything in it's old face style before my magic new frame starts up. I suspect this is probably a hangover from the funny hacks I have to ensure the default frame appears as I want it.

New Feed: Emacs Stuff

2009-05-29T12:59:00+01:00

I've created a new feed for my emacs related posts (there are more) with an eye to getting it syndicated on Planet Emacs and joining the wider emacs blogging community. Of course this means I'll have to blog about emacs more often which may throw the "all topics" syndicated livejournal feed people, but they are probably use to the wild swings between general trivia and geek overload already :-)

As soon as I have a working solution for my "emacsclient frames keep jumping to the wrong workspace" problem I'll be sure to post it up here.

Bunker mentality

2009-02-06T09:58:00+00:00

True to Hofstader's law it took a few extra days to finish of my (unpaid) work at Essex University. You can see the final results here. It was essentially tweaking a psychology experiment to work in a different colour space and with some funky high dynamic colour range hardware. After the first day of struggling with Python IDLE editor I caved in and installed a Windows aware copy of Emacs and fiddled with getting python-mode working in Windows. It removed a lot of the pain apart from every time the keymap randomly switched and whenever I had to use the browser which didn't have an awesome bar. And I certainly don't miss the constant reboots and bad crashes.

The last few days we have been hunkered in at home as the weather decides how much snow it will dump each day. The trains to London are basically broken so Fliss has been working from home. I'm also working from home and onto my next project of generating smart playlists for Rhythmbox. Luckily it seems someone has already done some work on this so I can concentrate on plumbing the existing code into Rhythmbox and cleaning up the UI.

More emacs server support

2008-11-03T19:05:00+00:00

I found that one of the Debian developers packages emacs-snapshot for Debian stable. A little bit of shell script later and my Mutt is firing off a muti-tty emacsclients whenever I need to edit a mail. It shaves literally seconds off the time it takes me to compose an email …

Lisp Service

2008-08-29T11:49:00+01:00

I finally figured out the correct magic to pass custom command line arguments to my emacs. This has allowed me to get the emacs-server running in a way more tuned to the way I work.

A lot of emacs people only run one instance of emacs and therefor take the start-up hit and leave it at that. I however tend to run several (one per project) so I can keep a certain amount of state in each session. All the solutions involving shell scripting to optionally use emacs client weren't quite what I wanted. Instead I want to start up one emacs at the start of my windows session that handles all the emacsclient stuff (like editing web-forms) and leave everything as is. It will be no surprise to emacs users that there is a hook for processing custom command line options so now my emacs is "-server" enabled. The only wrinkle is the command line options are processed after you .emacs is loaded so I can't define a "-skip-stuff-for-mutts-edits" flag for invoking from mutt. I don't really want to start messing with the Muti-TTY stuff just yet so I suspect rather than custom flags I may well start parsing the command line to detect mutt temp files.

UPDATE: My emacs commit is mirrored here.