RDF::Simple And Proxies

I was trying to work with Jo Walsh‘s RDF::Simple 0.12 Perl module, but it kept failing as it wasn’t able to connect past the office proxy server.

It turned out the solution wasn’t too hard. I tweaked the RDF::Simple::Parser to accept an http_proxy parameter in its new method. The ua method then checks this when it creates a LWP::UserAgent object, and either sets the proxy method directly, or uses the settings from environment variables.

The following patch has already been emailed to Jo for her to consider inclusion in the next module release, but in the mean time it is available here.

--- Parser.pm Wed Apr 28 11:20:00 2004
+++ Parser.pm.robsnew Wed Apr 28 11:19:52 2004
@@ -4,9 +4,9 @@
use XML::SAX qw(Namespaces Validation);
use LWP::UserAgent;
-our $VERSION = '0.1';
+our $VERSION = '0.2';
-use Class::MethodMaker new_hash_init => 'new', get_set => [ qw(base)];
+use Class::MethodMaker new_hash_init => 'new', get_set => [ qw(base http_proxy)];
sub parse_rdf {
my ($self,$rdf) = @_;
@@ -36,7 +36,15 @@
sub ua {
my $self = shift;
- $self->{_ua} ||= LWP::UserAgent->new(timeout => 30);
+ unless ($self->{_ua}) {
+ $self->{_ua} = LWP::UserAgent->new(timeout => 30);
+ if ($self->http_proxy) {
+ $self->{_ua}->proxy('http',$self->http_proxy);
+ } else {
+ $self->{_ua}->env_proxy;
+ }
+ }
+ return $self->{_ua};
package RDF::Simple::Parser::Handler;
@@ -481,6 +489,11 @@
'base' supplies a base URI
for relative URIs found in the document
+ 'http_proxy' optionally supplies
+ the address of an http proxy server.
+ If this is not given it will try to use
+ the default environment settings.
=head2 parse_rdf($rdf)

LWP::Simple And Proxies

Some Perl scripts I’ve been writing lately have to connect to remote pages to collect data. Normally I use LWP::Simple, but this hasn’t been working too well since we started having to use a proxy server in the office.

One solution is to either set the environment variable http_proxy with the address of the proxy server.

Another is to export the useragent variable $ua when you use the module, and set the proxy details directly in the resulting LWP::UserAgent object.

As an example, we’ll download the contents of my homepage in the variable $page.

use LWP::Simple qw($ua get);
my $page = get 'http://www.robertprice.co.uk/';

I also came across a useful piece on Perl Monks called Getting more out of LWP::Simple, that has lots more useful tips on it.

Aloud.com’s Revolution Award

Aloud.com won the Revolution Award for Best online property from a media owner on Friday night. We were up against Runners World, The Sun Online and ThisIsMoney.co.uk.

Here’s what the judges had to say about us…

To capitalise on the growth of the music event market, Emap Performance relaunched Aloud.com in March 2003. It aimed to attract customers with a unique and innovative ticketing service, based on a number of core principles.

It wanted event search and ticket purchase to be succinct while offering users all the necessary information options. Aloud.com aimed to draw on the heritage of other Emap Performance brands to provide a depth of editorial content that would interest users.

Aloud.com looked to exploit the UK’s largest music-media machine to push brand-specific inventory across multi-platform campaigns, maximise the potential of each event as it became available, and drive sales by event-specific marketing.

The site was given a clean, functional design, showing users how to find their choice from more than 5,000 events in the UK, with instant access to tickets and only four clicks to purchase. Users can search by a combination of artist, date, town or venue and then the results by date, price or alphabetically.

Aloud.com came through one of its biggests tests with flying colours by selling 90,000 Glastonbury tickets in 11 hours, which it claims is a world-record festival sell out. Overall, ticket sales grew by 400 per cent, averaging 20,000 a month with unique users rising by 60 per cent a month. Registered users went up by 120 per cent.

Aloud.com is now the seventh most-visited UK music website, according to WebTrends.

I built and maintain the site technically.

RDF And Barcodes

Chris Heathcote has the really interesting idea of using RDF as barcodes.

Barcodes in Japan use something called a QR Code, and the format is in the public domain.

Camera phones are able to decode these images so they are a great way of loading short data into a mobile device.

Time for me to read up on this and maybe have a play with the technology at the weekend.

XML::Parser Problem Fixed

I was having serious problems using the Perl module XML::Parser to extract data from an XML file.

The file was several megabytes in size, but every so often the content of one element would not extract correctly. I was convinced the problem must of been with expat, the C parser behind XML::Parser.

It turns out I should of read the manual a little better.

I was resetting a variable in my Char handler each time the handler was called. I was assuming this would only be called once per element. However, it can be called multple times per element and this is what was happening.

For my small test file, the original code was fine. However, resetting the variable in the Start handler and making sure that the Char handler now appends data each call has fixed the problem for all files, including very large ones.

New Blog

Well I’ve decided to write my own little mini blog for the site. The software is custom written in Perl for fun, so things may change for a bit while I bed down the code and decide exactly what I want.