The code relies on an XML node class, which is presumably one that you can find in Python’s library. Perl 6’s library is rather sparse right now, so I’m going to partially specify a Node class which is basically the same as the bits of it used by Paul’s Python code, just with some Perl 6-ish naming conventions.

class Node {
  enum NodeType <TextNode>;
  has $.data;
  has Node @.child-nodes;
  has NodeType $.node-type;
  method get-elements-by-tag-name(Str $name) { ... }
}

This, then, is a more or less direct translation of Paul’s Python code into Perl 6. The main difference is that Perl 6 has type annotations, we don’t have to import anything to get partial function application (that’s given by .assuming) and we don’t have to do any tricks to get string reduction, because of Perl 6’s [] metaoperator.

# Given a list of nodes, return the .data member of all nodes
# which have node-type of TextNode
sub get-text-data(Node @nodelist) {
    @nodelist.grep: { .node-type ~~ Node::TextNode }.map: { .data }
}

# Given a list of nodes, extract the text data and return it as
# one long string
sub extract-text(Node @nodelist) {
    [~] get-text-data(@nodelist);
}

# Given a node tree and an element name, find the first text
# element of that name and get the text from all its child nodes
sub get-one-text-element(Node $node, Str $name) {
    my @elements = $node.get-elements-by-tag-name($name);
    @elements.elems == 1 or die;
    extract-text(@elements[0].child-nodes);
}

# Given a node tree and a list of element names, extract all the
# text from each named node and return as a hash of name => text
sub make-text-element-dict(Node $node, Str @namelist) {
    return @namelist Z @namelist.map(
      &get-one-text-element.assuming(node => $node)
    );
}

This could of course be shorter – most of these subs don’t need to be defined here, we could turn it into a very few lines indeed, but I’m assuming that the subs might be needed elsewhere in the form presented here.

A few things to point out for Perl 6 novices:

As in Perl 5, variables starting with @ are arrays (if you want to get precise, it means there’s something in there which does the Positional role, so you can expect to be able to use [] element accessors and suchlike).

The [~] construct is actually two things – first, the infix operator ~, which in Perl 6 is string concatenation, surrounded by the [] metaoperator which turns it into a list reduction. Thus the effect is to concatenate all the list elements together and return one big string.

Perl 6 identifiers can have hyphens in them. The parser’s smart enough to distinguish between those and use of infix -.

Perl 6 subroutines can have formal parameter lists. These parameters, like Perl 6 variables, can have type information explicitly declared for strict checking.

You can pass blocks of code to methods like .map and .grep using :{} syntax, which saves some nested brackety things.

You can produce a curried version of a function using its .assuming method.

The infix operator Z works like the zip function in Python and Haskell.

I haven’t checked if this would compile, but I think it’s pretty close.

Rakudo already ships with a fairly basic Test.pm which already had some support for planless testing, although you still had to explicitly tell it that’s what you wanted:

plan *;

Whereas we’d rather like it if you could just leave out the call to plan when you weren’t planning (hah!) to use a plan. We also decided to use the done_testing idea, so that you call this in planless mode after your tests are complete to mark explicitly that you’ve run as many as you wanted to.

I was able to put together a new version of Test.pm which contains all these enhancements for planless testing, while at the same time operating transparently in planned mode for all the test modules already out there. I for one didn’t want to go through the Perl 6 spectest suite and change all those…

The patch is submitted to RT, so hopefully it will land in Rakudo before too long (assuming nobody finds any screaming bugs in it). Then your test modules can look like this:

use Test;
use MyModule;

ok(MyModule::do-something(), "Doing something");

done_testing;

Imagine that Christmas has come: you’re writing a programme in Perl 6 (you can actually do this today, it’s just that there are large chunks of the language which nobody has implemented yet). In your wonderful Perl 6 programme, your world domination plan includes the need to parse some text, so you reach for a grammar, the overpowered, mutant grandson of Perl 5’s regular expressions.

Grammars are really a special kind of class, containing a special kind of subroutine (being regexes, rules or tokens rather than subs, methods or submethods). Say your grammar looks like this:

grammar Statements {
  regex TOP {
    ( <statement> ';' )*
    {*}
  }
  regex statement {
      'hello' <ws> <string> {*} #= hello
    | 'goodbye' <ws> <string> {*} #= goodbye
  }
  regex string {
    '"' (.*?) '"'
    {*}
  }
}

That is, you’re going to match against zero or more statements, which are separated by semicolons and consist of either ‘hello’ followed by some whitespace and a double-quoted string, or ‘goodbye’ followed by some whitespace and a double-quoted string. I know the string regex is bad in that it’s awful at delimiter matching, but it’ll do for now in the interests of keeping down the complexity. Also, in real life you’d probably have rather more flexible rules for separating statements, but I’m showing off AST-building here, not fancy regexes.

So what does it all mean? The syntax isn’t too difficult to get your head around: an identifier inside angle brackets invokes another regex as a submatch. | delimits alternatives and * is the zero-or-more quantifier. {*} is where the clever bit comes in. When the matcher sees {*} in a rule, it knows that it should go and call a method with the same name as the current rule in an object which was passed to the matcher when it was invoked. These are called action methods, and offer a fantastic way to use the grammar system for building abstract syntax trees or other constructions which represent your parsed text. To make this even more useful, if the line with the {*} on it ends with a comment like those seen in the statement rule above, the text of the comment will be passed to the action method, so you can signal to it which alternative branch you want it to deal with.

Now in real life your action methods might produce one of several possible classes, maybe from a huge tree of classes which can represent your AST, or from a set of classes which use a common set of roles, or maybe you’re going down the route of some of the calculator examples which have been developed which do the calculations right there in the action methods, and represent nodes by their results which are just numbers. In this example, to avoid having to define a class for statements (which is really quite simple), I’m just going to produce a hash with two keys: verb and string.

So we write a class to hold the action methods. We can call it whatever we like, but the method names and the rule names they’re triggered by have to match.

class Actions {
  method TOP($/) {
    my @statements = gather for @( $/[0] ) -> $submatch {
      take $submatch<statement>.ast;
    }
    make @statements;
  }
  method statement($/, $key) {
    my %s;
    %s<verb> = $key;
    %s<string> = $/<string>;
    make %s;
  }
  method string($/) {
    make $/[0].ast;
  }
}

Now we see the real magic at work. First, each action method has a parameter $/, which is the default name for the match object. Calling it $/ lets you call make without any special considerations because the background match object is just there. A match object, by the way, is what a regex returns when it matches something, and it contains all the information about where and what it matched, including other match objects for subpattern matches.

Next, we see that the statement method has a second parameter, $key – this is where the comment-syntax key strings arrive. It’s entirely optional, so I only put it on the method that would actually get some use from it.

The make function is really key to this business – it alters the match object to provide a different result. The match object itself stays intact, but it has a slot (accessed through the .ast method) which can carry any object you like which represents the matched text. It’s called .ast because of its frequent use in building abstract syntax trees, and here it’s going to carry our hashes.

Let’s look at each method in turn. The action method for string is straightforward. If you look back at the regex for string, you’ll see that it has a captured subpattern between the double-quote delimiters. Subpatterns like this appear in the Match object as array elements, so you can use the array indexing operator to access them. We know here that there will only be one, and as it’s an array it’ll be in slot 0, so we can just say $/[0]. We call .ast on it, because the default result is a string representing what was parsed by that pattern. In this case that’s exactly what we want, so we set it to our result and we’re done.

Next, the statement method is a little more complicated. It creates a new hash, then sets the verb to the passed-in $key. Because string was invoked as a named subpattern, it can be found as a hash element of the match object, so we retreive $/<string>.ast, which will be the string that was produced by the string action method, and store that in the hash before using make to set it in our ast slot.

Finally, TOP has to deal with a list of subpattern captures. Because there’s a repetition quantifier on the capture, $/[0] will be an array. We can iterate over that with a for loop to get at the match object for each repetition, and then use the hash access to retrieve the match object for the statement subrule – which is of course the hash made by the statement action method.

In this case, we use gather for to construct the list lazily (or at least, it will be lazy once Rakudo has lazy lists). gather needs a loop construct, and within that construct you can call take as many times as you like – each time a new element will be added to the end of the list which is being constructed by the entire gather expression. Ultimately that list is assigned to @statements, and we make that as the final result from the match. In this case we take the ast slot from the submatch, but you may notice that there’s a backslash before its $. Why is this? Perl 5 programmers are probably thinking ‘ah, he’s taking a reference so that he can make an array of hashes’, and they’d be sort of right. Without that, the hash wants to flatten into the array, which is not what we want, so we use @\@ which in Perl 6 creates a capture (an astonishing Perl 6 construct which is used for everything from stopping things flattening into lists to passing subroutine parameters around), which in this case will just prevent the hash from flattening into the array. There may be a more elegant way to do this by ensuring that the hash isn’t taken in a context that’s going to cause flattening, but I haven’t figured that part out yet. In any case, the end result is an array of hashes, which is exactly what we want. The captures are entirely transparent to everything that tries to use it.

How do you run this? Very easily:

my $actions = Actions.new;
my $result = Statements.parsefile('somefilename', :action($actions));
for $result.ast -> $statement {
  given $statement<verb> {
    when 'hello' { say "Hello, $statement<string>"; }
    when 'goodbye' { say "Goodbye, $statement<string>"; }
  }
}

And that, as they say, is that, except that you can do all this in Rakudo Perl 6 today.

I’m not going to say much more, because I can’t make the point any better than he did, but the point is a good one. At work I designed a simple prefix-based ASCII protocol for two servers to use to talk to each other. Much more lightweight than XML, much easier to encode and decode, and you only need the first few bytes of a message to figure out if the message is incomplete or not. And you can transmit pretty much anything with it. It’s even human-readable, to a certain extent – you need the protocol spec in hand (or you have to know it) but you can figure out what a message is trying to say.

Estimated bandwidth saving over using XML? I reckon it’s about 30% lighter than a reasonable XML application to do the same job.

XML does of course have its place, and having tools like XSLT and XPath available to play with structured data is great, but sometimes you don’t need it, and sometimes you just can’t take the bandwidth hit, because like it or not XML gets verbose very quickly.

This post brought to you by the thinking about work on a Sunday society.

The problem now is that libglade isn’t finding half the widgets in the glade file, and returns null pointers for them instead of the widget pointers. Every single one of the ones it’s failing to find is definitely in the XML, and the file’s definitely in the right place. I’ve checked – lots of times.

This might just be what makes me start to go bald.

Okay so it took a manual edit to the Makefiles. I think I’m going to have to consult somebody who knows this whole autoconf/automake business a bit better to see how I can get the Glade libraries referenced properly.

Then I need to get it to actually start up as it currently tries to dereference a null pointer and segfaults. This is probably due to my fairly cavalier #ifdefing out to get it to compile. Now I need to go back and figure out which bits are actually needed and how to get them to work.

Ultimately of course the UI under Maemo is going to be a bit different, especially once I get the thing working as an input method, but for now I just want it to work at all!

In other news, I think I finally figured out how to do the positioning system for Growl. It’s not the total solution I originally intended, but it’s probably more practical to do it this way than the way I wanted to do it — which was far beyond my small mind’s ability to cope with.

But before I can engage on that again, time for work.

For the curious, GuidePost is a simple text-based editor for GuideML, the XML application used for h2g2 articles, and other DNA-powered sites on bbc.co.uk. I wrote it many years ago and haven’t touched it since, largely because it’s a Windows app and I don’t do those anymore. I do keep meaning to do a Mac version, but haven’t found the time yet.

With that out of the way, what else was I going to write about? Ah yes. Towel proceeds. It needs a good deal more polishing, but it’s gradually getting there for another release, this one with significant enhancements. The things I’m aiming for are:

• metadata caching in saved playlists, leading to improved load times
• don’t have GDK markup errors on track titles with double quotes in
• use GStreamer 0.8 (done)
• use GStreamer for metadata reading, instead of id3lib, libvorbis and libflac
• metadata editing using GStreamer
• miscellaneous UI improvements
• anything else that seems like it’ll fit

After that, the next release will likely dual-build with gtkmm 2.5, which is important because there are some handy new widgets I would find useful, lots of nice bug fixes and stock icons for play, pause etc.

Another project I’m starting to help with related to Towel is gstmm, the C++ bindings for GStreamer. They use the same system as gtkmm does, so they should be eminently compatible with a gtkmm application such as Towel. Development has unfortunately been stalled for quite some time, so hopefully Bryan Forbes and I will be able to get something passably wrapping GStreamer 0.8. And hopefully it won’t take us all year to do it.

Last night was the penultimate Tai Chi before Christmas. Not many people came so we didn’t do a new move in the longsword form, but instead spent quite a while doing Chi Kung and pushing hands. Learned some interesting things, improved my technique in one kind of movement particularly. Then we revised the longsword form as far as we have reached, which was useful to pick up on the details that tend to get lost in those first few weeks after doing a move where you’re more worried about which arm to stick out than the finer points of the move. Curiously, everyone at Tai Chi thinks the way Christmas is handled by the shops is very irritating. Where are the people that the shops are doing what they’re doing all this for?

Not sure what I’m doing for Yule yet this year. We shall see. The full moon group are doing a Yule ritual, so it’s likely I’ll go to that. It is highly likely to be a significant improvement on doing one solo.

Recently I’ve been working on getting it to compile on my native 64bit Linux system, which wasn’t too hard because it was fairly intelligently coded the first time around (actually the most serious problem was with printf format strings generating warnings about variable sizes). The nastiest work was porting to GStreamer 0.8, which should have been done a long time ago but is done now and works quite nicely.

Currently I’m implementing properties dialogs for groups and tracks, which will help people figure out what the checkboxes in the groups and tracks views are for as they can be set from the properties dialogs as well — this is until GtkTreeView gains proper tooltip support, but I suspect that might take a while and may even be a GTK+ 3.0 feature. Also on the list are metadata caching, so Towel doesn’t have to probe every file in the library for its title/artist/length data every time it starts up, better handling of file read errors (there are going to be some holes exposed by the metadata caching system, as previously we relied heavily on all the files being read at startup), and general UI love for the library window.

Moving onward, there’ll be a release, then I’ll port to gtkmm 2.5 to take advantage of some useful new features, such as ellipsizing labels and stock icons for play, pause, stop and so forth. For now I intend to keep the project at SourceForge, but it’s going to have to move into a Subversion repository somewhere at some point; I just can’t be doing with CVS anymore.

And many thanks to Bryan Forbes, who says he’ll do some testing and offer assistance with the UI design.

Now I just need to get hold of Jonathan again…