One of the many deficiencies of Digital Restrictions Management is the fact it makes doing legal authorised things hard for paying customers. For example my brother still can't play Blu-Ray disks on his laptop despite it having the requisite a) drive, b) power and c) display resolution. Because the OS on his machine (Vista?) can't see a verifiable encrypted path to his big screen display it refuses to play his legally purchased disks.
This is one reason why I haven't brought any Blu-Ray disks since the couple I got with my PS3 (to test the Hi-Def goodness). I refuse to buy them while I have no devices under my control* that can play them. Still the conversation with my brother and the fact I had a Blu-Ray drive in my desktop prompted me to see if it was indeed now possible.
My desktop system runs Gentoo which makes running bleeding edge code very easy. I duly enabled the "bluray" and "aacs" use flags and recompiled mplayer. This brought in libbluray and the command line utility aacskeys. Unfortunately mplayer was still unable to play the disk, complaining the disk was still encrypted.
After a bunch of searching the interwebs and drawing blanks on finding any explicit "HOWTO" instructions I resorted to running mplayer through a debugger (one advantage of an Open Source system is there are no "black boxes" to reverse engineer) to see how libbluray was being invoked. It turned out it attempts to dlopen another library libaacs which wasn't on my system. Once I added it's ebuild to my overlay I was one step closer.
Unlike the libdecss (which works around the rather anaemic crypto on plain old DVDs) the libaacs library doesn't crack the considerably stronger keys used on BluRay. It relies on a database of Volume keys which can be placed in ~/.config/aacs/KEYDB.CFG which are used to calculate the final decryption keys for the content. Confusingly the format is different from the key databases you may find searching the internet. However the libaacs file format documentation provides instruction on convert the old format into a new format file.
With all of this in-place I was finally able to play my legally owned copy of Black Hawk Down. Yay!
Of course this does require having the appropriate keys for the disc you want to play in your key database. This is where aacskeys comes in. This program allows you to extract the appropriate volume ID key from the disc you want to play. However there are some potential caveats. For starters the program is a little unfriendly to use as it requires certain files to be in the directory your running from. Also some BluRay drive firmwares will refuse to supply the needed keys if the disks are a newer generation than the drive (due to the AACS key revocation model). I'm not sure if a drive will self revoke under Linux if presented with a newer disk but this is the reason there are patched drive firmwares out there.
In the event I was able to verify that aacskeys generates the same volume ID for my copy of Blackhawk Down as was in the KEYDB.CFG database so I know it works for now.
In summary this does show it is possible to play a Blu-Ray disk on a Linux system however there was a fair bit of hoop jumping involved. If I can find a source repository for aacskeys I might start by cleaning it up and making the process of updating the key database a little less painful. I doubt I'll be buying any more Blu-Ray disks until the process of playback is a lot simpler.
* My PS3 is very much Sony's device, a Faustian bargain I accept because I like playing games every now and again. Witness the removal of OtherOS from working systems in the wild by a mandatory firmware update.