Here are Rob's Blog entries for December 2004.
Blog entries for other months can be found in the main blog index.
We launched FHM mobile at work this week.
Currently it's only available on O2 active, but as the screenshots show, it's bloody brilliant!
If you have access to an O2 phone give it a try. We've tried to make it best mobile site currently out there.
Nokia have finally released Python for Series 60 phones.
This is great getting an interactive language onto the phone, if only it were Perl though.
The release notes say it is best suited prototyping and proof of concept applications, and I'd have to agree. J2ME and C++ are too time consuming to just play about with.
I've downloaded the SDK, so I just need to learn Python now. :-)
It all started in the FHM newsletter when they announced they would be chucking goodies out of their window at 1pm today.
This is was the scene on Winsley Street in London just after 1. Crowds in the road, and cds, dvds, books, games, etc being thrown down.
However, as crowds were blocking the road, desperate to get their hands on all the free gear, it didn't take long before community support officers appeared to try to get everyone to move on.
When I came back from getting my lunch, there was an queue outside the office of people waiting for freebies, with police and cso's making sure things were kept in hand.
Those of you with access to a 3GP player can see a video clip of the window drop.
I received a spam message today on my phone that was really clever and bloody annoying.
It came from "Apanatchi" offering me a free ringtone.
The details revealed it to be a WAP Push message.
WAP Pushes are messages sent to a mobile phone that takes the user to an external WAP site where they can download content easily without having to enter a long URL into their phone. They are really useful, but it's a shame to see them now being used for spam.
Opening the message takes you a WAP site hosted on the IP addres 195.74.152.83. A quick WHOIS for 195.74.152.83 shows it to owned by a company called Wireless Information Network Ltd and hosted by Rednet.
When you select your free ringtone, the terms and conditions show that this is really a spam. You are signing up for "promotional" messages, and also a subscription ringtone service that costs you 1.50UKP every 5 days.
The WAP site you are taken to seems very generic. It has a parameter at the end of the URL that probably relates to the recipients phone number. I've masked mine out in the images on this page so I am not signed up for too much junk by unwary readers.
I was particularly annoyed with this spam. Unsolicited junk to my phone is a massive invasion of privacy. I wonder how many people have been caught out with this?
I'll be contacting Ofcom, Wireless Information Network Ltd and my phone operator Orange about this. Let's see what they have to say...
UPDATE: 23/12/2004
I've heard back from Lianne at Wireless Information Network Ltd who says the promotion is run by a company called Venista. They can be contacted directly on 0870 609 1720. Very sneaky, if you call them, then they make a profit as 0870 numbers generate 2p of revenue a minute for their owners.
If you are being charged for unwanted downloads by Venista as a result of their spam, I'd suggest giving ICSTIS a call, as they regulate premium numbers (not 0870 though) and mobile short codes.
I was at Oxford Circus tube station earlier, and I was please to see that the Slashing Santa poster has returned this year.
It appears to show Santa taking a slash (urinating) against the tube map at Manor Park station.
What do you think?
I've updated my train arrival and departure RSS feeds as livedepartureboards.co.uk have changed their page to XHTML strict.
The script relies on screen-scraping, so the change broke the original script. It's welcome to see they are using strict XHTML. I should really rewrite the entire script to use XML parsing now, but I'm lazy, and all that needs changing are a few regular expressions in the original script for it to work.
An example of the feed is shown below in my FeedReader.
Here is the full updated Perl code.
#!/usr/bin/perl -w
## Script to convert train departures from livedepartureboards.co.uk
## into RSS feeds.
## Takes the parameter "stationcode", as defined on the page nationalrail.co.uk
## page - http://www.nationalrail.co.uk/frameset.asp?location=ldb
##
## Changed from the original to understand their lovely new XHTML site.
##
## Robert Price - www.robertprice.co.uk - 7 December 2004
use strict;
use CGI;
use HTTP::Request;
use LWP::UserAgent;
use XML::RSS;
## Take the stationcode parameter, else default to COL (Colchester)
my $CGI = new CGI;
my $stationcode = $CGI->param('stationcode') || 'COL';
## The URL we have to screen scrape for the information.
my $url = 'http://www.livedepartureboards.co.uk/ldb/summary.aspx?T=' . $stationcode;
## Create our new browser, and pretend to be IE.
my $ua = LWP::UserAgent->new;
$ua->agent("Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)");
## Get the data from the livedeartureboards.co.uk website.
my $req = HTTP::Request->new(GET => $url);
$req->header('Accept' => 'text/html');
my $res = $ua->request($req);
if ($res->is_success) {
my $page = $res->content;
## Get the station name and when the board was last updated.
my ($updated) = ($page =~ m[Last updated: (.*?)</td>]sg);
my ($station_name) = ($page =~ m[<h1>Train Times for (.*?)</h1>]sg);
## Create our RSS data, update every 10 minutes.
my $rss = new XML::RSS (version => '1.0');
$rss->channel(
title => $station_name . ' Train Departures',
link => 'http://www.nationalrail.co.uk/',
description => 'Train times from ' . $station_name,
syn => {
updatePeriod => 'hourly',
updateFrequency => '6'
},
);
## Monster regex to extract the data.
while ($page =~ m[<tr.*?>\r\n<td><a href="(.*?)">(.*?)</a></td>\r\n<td.*?>(.*?)</td>\r\n<td.*?>(.*?)</td>\r\n<td.*?><a href=".*?">(.*?)</a></td>\r\n<td.*?>(.*?)</td>\r\n<td.*?>(.*?)</td>\r\n<td.*?><a href=".*?">(.*?)</a></td>\r\n</tr>]sg) {
## Build a hash to store the information in.
my $train = {
'link' => 'http://www.livedepartureboards.co.uk/ldb/' . $1,
'from' => $2,
'timetabled_arrival' => $3,
'expected_arrival' => $4,
'to' => $5,
'timetabled_departure' => $6,
'expected_departure' => $7,
'operator' => $8
};
## Add the train departure to the RSS feed.
$rss->add_item(
title => ($train->{'timetabled_departure'} ? $train->{'timetabled_departure'} . ' to ' : '') . $train->{'to'} . ($train->{'expected_departure'} ? ( ' (' . $train->{'expected_departure'} . ')' ) : ''),
link => $train->{'link'},
description =>
"From: " . $train->{'from'} . ",\n" .
"Timetabled Arrival: " . $train->{'timetabled_arrival'} . ",\n" .
"Expected Arrival: " . $train->{'expected_arrival'} . ",\n" .
"To: " . $train->{'to'} . ",\n" .
"Timetabled Departure: " . $train->{'timetabled_departure'} . ",\n" .
"Expected Departure: " . $train->{'expected_departure'} . ",\n" .
"Operator: " . $train->{'operator'} . "\n",
);
}
## Return the RSS feed, ensuring we have the correct MIME type.
print "Content-type: application/rdf+xml\n\n";
print $rss->as_string;
}
Just install it as a CGI script on your webserver, and point your RSS reader at it. You'll need to add your station code as the stationcode parameter, codes can be found on the National Rail website's station code page.
In this article I'll describe how to parse and extract data from an RDF file using Jo Walsh's RDF::Simple::Parser module in Perl.
RDF::Simple::Parser does what it says on the tin, it provides a simple way to parse RDF. Unfortunately, that can make it hard to extract data. All it returns from a successful parse of the RDF file, is what Jo calls a "bucket-o-triples". This is just an array of arrays. The first array contains an list of all the triples. The second array contains the actual triples broken down so Subject is in position 0, Predicate is in position 1 and Object in position 2.
Let's define these as constants in Perl as they're not going to be changing.
use constant SUBJECT => 0;
use constant PREDICATE => 1;
use constant OBJECT => 2;
I'm going to use my usual example of my parsing my FOAF file, and I'll be extracting the addresses of my friend's FOAF files from it. See the example in What Is An RDF Triple, for a full breakdown of this.
We'll define the two predicates we need to look for as constants.
use constant KNOWS_PREDICATE => 'http://xmlns.com/foaf/0.1/knows';
use constant SEEALSO_PREDICATE => 'http://www.w3.org/2000/01/rdf-schema#seeAlso';
We need to load in the FOAF file, so we'll take advantage of File::Slurp's read_file method to do this and put it in a variable called $file.
my $file = read_file('./foaf.rdf');
Before we can use RDF::Simple::Parser, we need to create an instance of it. I'll set the base address to www.robertprice.co.uk in this case.
my $parser = RDF::Simple::Parser->new(base => 'http://www.robertprice.co.uk/');
Now we have the instance, we can pass in our FOAF file for parsing and get back our triples.
my @triples = $parser->parse_rdf($file);
Let's take a quick look at my FOAF file to get an example triple.
I know Cal Henderson, and this is represented in my FOAF file as...
<foaf:knows>
<foaf:Person>
<foaf:nick>Cal</foaf:nick>
<foaf:name>Cal Henderson</foaf:name>
<foaf:mbox_sha1sum>2971b1c2fd1d4f0e8f99c167cd85d522a614b07b</foaf:mbox_sha1sum>
<rdfs:seeAlso rdf:resource="http://www.iamcal.com/foaf.xml"/>
</foaf:Person>
</foaf:knows>
Using the RDF validator we can get a the list of triples represented in this piece of RDF.
| Triple | Subject | Predicate | Object |
|---|---|---|---|
| 1 | genid:ARP40722 | http://www.w3.org/1999/02/22-rdf-syntax-ns#type | http://xmlns.com/foaf/0.1/Person |
| 2 | genid:ARP40722 | http://xmlns.com/foaf/0.1/nick | "Cal" |
| 3 | genid:ARP40722 | http://xmlns.com/foaf/0.1/name | "Cal Henderson" |
| 4 | genid:ARP40722 | http://xmlns.com/foaf/0.1/mbox_sha1sum | "2971b1c2fd1d4f0e8f99c167cd85d522a614b07b" |
| 5 | genid:ARP40722 | http://www.w3.org/2000/01/rdf-schema#seeAlso | http://www.iamcal.com/foaf.xml |
| 6 | genid:me | http://xmlns.com/foaf/0.1/knows | genid:ARP40722 |
The part we are interested are triples 5 and 6. We can see that triple 6 has Predicate value the same as our KNOWS_PREDICATE constant, and triple 5 has the Predicate value of our SEEALSO_PREDICATE constant. The part this links the two is that triple 6 has the Object value of triple 5's Subject.
We know if we search for triples with the same predicate as our KNOWS_PREDICATE we'll get triples that are to do with people I know. We can use Perl's grep function to get these triples, then we can interate over them in a foreach loop.
foreach my $known (grep { $_->[PREDICATE] eq KNOWS_PREDICATE } @triples) {
We are only interest in the triples that have the same Subject as matching triple's Object. Again, we can use grep to get these out so we can interate over them.
foreach my $triple (grep { $_->[SUBJECT] eq $known->[OBJECT] } @triples) {
Now we just need to make sure that the triple's Predicate matches our SEEALSO_PREDICATE constant, and if it does, we can print out the value of it from it's Object.
if ($triple->[PREDICATE] eq SEEALSO_PREDICATE) {
print $triple->[OBJECT], "\n"
}
Let's put this all together into a working example...
#!/usr/bin/perl -w
use strict;
use File::Slurp;
use RDF::Simple::Parser;
## constants defining position of triple components in
## RDF::Simple triple lists.
use constant SUBJECT => 0;
use constant PREDICATE => 1;
use constant OBJECT => 2;
## some known predicates.
use constant KNOWS_PREDICATE => 'http://xmlns.com/foaf/0.1/knows';
use constant SEEALSO_PREDICATE => 'http://www.w3.org/2000/01/rdf-schema#seeAlso';
## read in my foaf file and put it in $file.
my $file = read_file('./foaf.rdf');
## create a new parser, using my domain as a base.
my $parser = RDF::Simple::Parser->new(base => 'http://www.robertprice.co.uk/');
## parse my foaf file, and return a list of triples.
my @triples = $parser->parse_rdf($file);
## iterate over a list of triples matching the KNOWN_PREDICATE value.
foreach my $known (grep { $_->[PREDICATE] eq KNOWS_PREDICATE } @triples) {
## iteratve over a list of triples that have the same subject
## as one of our KNOWN_PREDICATE triples object.
foreach my $triple (grep { $_->[SUBJECT] eq $known->[OBJECT] } @triples) {
## find triples that match the SEEALSO_PREDICATE
if ($triple->[PREDICATE] eq SEEALSO_PREDICATE) {
## print out the object, should be the address
## of my friends foaf file.
print $triple->[OBJECT], "\n"
}
}
}
The example will load in the FOAF file, parse it and print out any friends of mine that have FOAF files defined by the seeAlso predicate.
These two snaps were taken from the window of an Electrostar train from Eastbourne to London late this morning on my Nokia 7610 phone.
The South Downs looked very beautiful and ethereal shrouded in mist, with the winter sun low in the sky.
