Tag: Web Development

Framing Pages With JavaScript

Normally, when building a website, we’d not want our content to be in frames.

However, a project at work involved hosting some content externally from the parent site that was framed in the parent site.

That’s fine, nothing to difficult, but this content also provided an RSS feed and those links had to feed back into the parent page, so needed to launch the site framework around itself if it didn’t already exist.

The solution to this was to be have two JavaScript’s, one in the piece of content, and the other in the parent framework. When the content is loaded it needs to check if it’s framed, and if it’s not to load the parent framework which then needs to reload the content.

Here’s the code that goes into the content…

<script language="JavaScript" type="text/javascript">
if (self == top) {
var newurl = "http://www.myparentsite.com/?url=" + encodeURI(window.location.href);
if (document.images) {
top.location.replace(newurl);
} else {
top.location.href = newurl;
}
}
</script>

So what’s going on here? Well firstly we need to check if the content page has been launched in a frameset or not. To do this we check if self is the same as top. self is a JavaScript object that refers to the current page, and top is a JavaScript object that refers to the top of the current page, so this could be the parent page of multiple framesets. If it’s the same we know we’re at the top of the page and not in a frameset, so we need to launch the parent page architecture. To do this we work out what the URL of a parent will be and get JavaScript to tell the browser to goto to that page. If you look at the code here, you’ll see the parent site I’ve called www.myparentsite.com, but this could be anything. I append a parameter called url that is an encoded version of the current page’s URL.

When our parent page loads, it needs to see if it has a URL parameter to load in the content frame correctly. We use the following code on the parent page to achieve this.

<script language="JavaScript" type="text/javascript">
var newurl = decodeURI(location.search.slice(5));
if (newurl) {
// nothing actually :-)
} else {
newurl = "http://www.framedcontent.com/defaultcontent.html";
}
document.write('<iframe src="' + newurl + '" scrolling="no" width="492" height="680" frameborder="0" ></iframe>')
</script>

All we’re doing here is getting a whatever parameter string is passed in and treating it as a URL. If we don’t have a parameter we use a default. We then create a new iframe referring to this URL.

As you can see we make no effort to check the parameter passed in is a valid URL or even if it’s in the url parameter. This is very bad code, but serves to demonstrate the technique. In real life we’d check for the url parameter and that it is valid.

Using SOAP::Lite With Perl

I’ve been trying to access a SOAP service using Perl for a project, so I took a quick look at CPAN to see what was available. The answer seemed to be to use SOAP::Lite.

This module turned out to be a real devil to use, and my problems with it were probably compounded by my lack of SOAP experience.

SOAP (incase you didn’t know) is a method of accessing remote services using XML. In my case, I was trying to access content supplied by a third party ringtone provider using SOAP over HTTP.

Using SOAP::Lite you have to specify a uri and a proxy. What stumped me for a while is that the proxy is the URI of service you wish to access, and uri is the namespace of the service.

So if the provider I was using had their SOAP service available at ws.robstones-services.co.uk/external.asmx, I would use this as the proxy, and in this case I would use ws.robstones-services.co.uk/External as the urinamespace.

Great, I knew the service I needed was called getCallList, and required a Username and Password to be passed to it. In return it would give me a list of valid content types. The SOAPAction header to be added to the call to proxy, letting the remote SOAP service I wanted to use the service http://ws.robstones-services.co.uk/External/getCallList.

My first attempt was the following code…

#!/usr/bin/perl -w
use strict;
use SOAP::Lite 'trace', 'debug';
my $server = SOAP::Lite
->uri('http://ws.robstones-services.co.uk/External')
->proxy('http://ws.robstones-services.co.uk/external.asmx');
my $returned = $server
->getCallList({
'Username' => 'RobsUser',
'Password' => 'RobsPassword'
});
foreach my $type ($returned->valueof('//getCallListResult/string')) {
next unless ($type); ## ignore any undefs
print "$typen";
}

You’ll notice I’m using trace and debug to see the SOAP messages being sent and received. The dialogue from this script was…

> perl -w test.pl
SOAP::Transport::HTTP::Client::send_receive: POST http://ws.robstones-services.co.uk/external.asmx HTTP/1.1
Accept: text/xml
Accept: multipart/*
Content-Length: 615
Content-Type: text/xml; charset=utf-8
SOAPAction: "http://ws.robstones-services.co.uk/External#getCallList"
<?xml version="1.0" encoding="UTF-8"?><SOAP-ENV:Envelope xmlns:xsi="http://www.w3.org/1999/XMLSchema-instance" xmlns:SOAP-ENC="http://schemas.xmlsoap.org/soap/encoding/" xmlns:xsd="http://www.w3.org/1999/XMLSchema" SOAP-ENV:encodingStyle="http://schemas.xmlsoap.org/soap/encoding/" xmlns:SOAP-ENV="http://schemas.xmlsoap.org/soap/envelope/"><SOAP-ENV:Body><namesp1:getCallList xmlns:namesp1="http://ws.robstones-services.co.uk/External"><c-gensym3><Username xsi:type="xsd:string">RobsUser</Username><Password xsi:type="xsd:string">RobsPassword</Password></c-gensym3></namesp1:getCallList></SOAP-ENV:Body></SOAP-ENV:Envelope>
SOAP::Transport::HTTP::Client::send_receive: HTTP/1.1 500 (Internal Server Error) Internal Server Error.
Cache-Control: private
Connection: close
Date: Tue, 29 Nov 2005 15:23:38 GMT
Server: Microsoft-IIS/6.0
Content-Length: 508
Content-Type: text/xml; charset=utf-8
Client-Date: Tue, 29 Nov 2005 15:23:09 GMT
Client-Response-Num: 1
X-AspNet-Version: 1.1.4322
X-Powered-By: ASP.NET
<?xml version="1.0" encoding="utf-8"?>
<soap:Envelope xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema">
<soap:Body>
<soap:Fault>
<faultcode>soap:Client</faultcode>
<faultstring>Server did not recognize the value of HTTP Header SOAPAction: http://ws.robstones-services.co.uk/External#getCallList.</faultstring>
<detail />
</soap:Fault>
</soap:Body>
</soap:Envelope>

The call didn’t work, you’ll see the SOAPAction variable is wrong, it’s http://ws.robstones-services.co.uk/External#getCallList instead of http://ws.robstones-services.co.uk/External/getCallList.

I needed to get SOAP::Lite to use the correct URL. The secret turned out to be to change the use SOAP::Lite line to the following.

use SOAP::Lite on_action => sub {sprintf '%s/%s', @_},
'trace', 'debug';

on_action is a parameter SOAP::Lite uses to separate the URI from the action, here we’re telling it to use /.

This gave me the following SOAP dialogue…

> perl -w test.pl
SOAP::Transport::HTTP::Client::send_receive: POST http://ws.robstones-services.co.uk/external.asmx HTTP/1.1
Accept: text/xml
Accept: multipart/*
Content-Length: 615
Content-Type: text/xml; charset=utf-8
SOAPAction: http://ws.robstones-services.co.uk/External/getCallList
<?xml version="1.0" encoding="UTF-8"?><SOAP-ENV:Envelope xmlns:xsi="http://www.w3.org/1999/XMLSchema-instance" xmlns:SOAP-ENC="http://schemas.xmlsoap.org/soap/encoding/" xmlns:xsd="http://www.w3.org/1999/XMLSchema" SOAP-ENV:encodingStyle="http://schemas.xmlsoap.org/soap/encoding/" xmlns:SOAP-ENV="http://schemas.xmlsoap.org/soap/envelope/"><SOAP-ENV:Body><namesp1:getCallList xmlns:namesp1="http://ws.robstones-services.co.uk/External"><c-gensym3><Username xsi:type="xsd:string">RobsUser</Username><Password xsi:type="xsd:string">RobsPassword</Password></c-gensym3></namesp1:getCallList></SOAP-ENV:Body></SOAP-ENV:Envelope>
SOAP::Transport::HTTP::Client::send_receive: HTTP/1.1 200 OK
Cache-Control: private, max-age=0
Connection: close
Date: Tue, 29 Nov 2005 15:22:53 GMT
Server: Microsoft-IIS/6.0
Content-Length: 405
Content-Type: text/xml; charset=utf-8
Client-Date: Tue, 29 Nov 2005 15:22:25 GMT
Client-Response-Num: 1
X-AspNet-Version: 1.1.4322
X-Powered-By: ASP.NET
<?xml version="1.0" encoding="utf-8"?><soap:Envelope xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema"><soap:Body><getCallListResponse xmlns="http://ws.robstones-services.co.uk/External"><getCallListResult><string xsi:nil="true" /></getCallListResult></getCallListResponse></soap:Body></soap:Envelope>

Bugger, it still didn’t work. I looked again at the SOAP message being sent. It was sending a lot of namespace information and a wrapper around the Username and Password. The wrapper was definitely wrong, and the remote .NET service didn’t seem to like the namespaces.

I needed to sort that out, so I had to tell SOAP::Lite to not to include namespaces or wrap up the Username and Password. The way to do this is to use SOAP::Data to hardcode the data being sent.

This meant changing the call to getCallList to the following…

my $returned = $server
->getCallList(
SOAP::Data->name('Username')->value('RobsUser')->type(''),
SOAP::Data->name('Password')->value('RobsPassword')->type('')
);

The SOAP dialogue after these changes looked like this..

SOAP::Transport::HTTP::Client::send_receive: POST http://ws.robstones-services.co.uk/external.asmx HTTP/1.1
Accept: text/xml
Accept: multipart/*
Content-Length: 548
Content-Type: text/xml; charset=utf-8
SOAPAction: http://ws.robstones-services.co.uk/External/getCallList
<?xml version="1.0" encoding="UTF-8"?><SOAP-ENV:Envelope xmlns:xsi="http://www.w3.org/1999/XMLSchema-instance" xmlns:SOAP-ENC="http://schemas.xmlsoap.org/soap/encoding/" xmlns:xsd="http://www.w3.org/1999/XMLSchema" SOAP-ENV:encodingStyle="http://schemas.xmlsoap.org/soap/encoding/" xmlns:SOAP-ENV="http://schemas.xmlsoap.org/soap/envelope/"><SOAP-ENV:Body><namesp1:getCallList xmlns:namesp1="http://ws.robstones-services.co.uk/External"><Username>RobsUser</Username><Password>RobsPassword</Password></namesp1:getCallList></SOAP-ENV:Body></SOAP-ENV:Envelope>
SOAP::Transport::HTTP::Client::send_receive: HTTP/1.1 200 OK
Cache-Control: private, max-age=0
Connection: close
Date: Tue, 29 Nov 2005 15:19:49 GMT
Server: Microsoft-IIS/6.0
Content-Length: 405
Content-Type: text/xml; charset=utf-8
Client-Date: Tue, 29 Nov 2005 15:19:21 GMT
Client-Response-Num: 1
X-AspNet-Version: 1.1.4322
X-Powered-By: ASP.NET
<?xml version="1.0" encoding="utf-8"?><soap:Envelope xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema"><soap:Body><getCallListResponse xmlns="http://ws.robstones-services.co.uk/External"><getCallListResult><string xsi:nil="true" /></getCallListResult></getCallListResponse></soap:Body></soap:Envelope>

Close, but still not right. The problem was with the getCallList itself, it was still trying to use a namespace.

The final solution was to this problem was to not use getCallList, but instead to use SOAP::Lite’s call method to implicitly set how the function was being called.

The code after making this change was the following…

my $returned = $server
->call(SOAP::Data->name('getCallList')->attr({xmlns => 'http://ws.robstones-services.co.uk/External'}) =>
SOAP::Data->name('Username')->value('RobsUser')->type(''),
SOAP::Data->name('Password')->value('RobsPassword')->type('')
);

This gave me the SOAP dialogue…

> perl -w test_soap.pl
SOAP::Transport::HTTP::Client::send_receive: POST http://ws.robstones-services.co.uk/external.asmx HTTP/1.1
Accept: text/xml
Accept: multipart/*
Content-Length: 524
Content-Type: text/xml; charset=utf-8
SOAPAction: http://ws.robstones-services.co.uk/External/getCallList
<?xml version="1.0" encoding="UTF-8"?><SOAP-ENV:Envelope xmlns:xsi="http://www.w3.org/1999/XMLSchema-instance" xmlns:SOAP-ENC="http://schemas.xmlsoap.org/soap/encoding/" xmlns:xsd="http://www.w3.org/1999/XMLSchema" SOAP-ENV:encodingStyle="http://schemas.xmlsoap.org/soap/encoding/" xmlns:SOAP-ENV="http://schemas.xmlsoap.org/soap/envelope/"><SOAP-ENV:Body><getCallList xmlns="http://ws.robstones-services.co.uk/External"><Username>RobsUser</Username><Password>RobsPassword</Password></getCallList></SOAP-ENV:Body></SOAP-ENV:Envelope>
SOAP::Transport::HTTP::Client::send_receive: HTTP/1.1 200 OK
Cache-Control: private, max-age=0
Connection: close
Date: Tue, 29 Nov 2005 15:16:07 GMT
Server: Microsoft-IIS/6.0
Content-Length: 540
Content-Type: text/xml; charset=utf-8
Client-Date: Tue, 29 Nov 2005 15:15:39 GMT
Client-Response-Num: 1
X-AspNet-Version: 1.1.4322
X-Powered-By: ASP.NET
<?xml version="1.0" encoding="utf-8"?><soap:Envelope xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema"><soap:Body><getCallListResponse xmlns="http://ws.robstones-services.co.uk/External"><getCallListResult><string>ROBS_JAVA</string><string>ROBS_WALL</string><string>ROBS_POLY</string><string>ROBS_MONO</string><string>ROBS_REAL</string><string xsi:nil="true" /></getCallListResult></getCallListResponse></soap:Body></soap:Envelope>
ROBS_JAVA
ROBS_WALL
ROBS_POLY
ROBS_MONO
ROBS_REAL

Hurrah, after all that effort, I have a list of 5 types of content offered by my third party supplier. I can now go ahead and build my service.

SOAP::Lite is a powerful module, but the lack of simple, easy to follow documentation and examples holds it back. I hope this small article helps out other programmers just starting on the SOAP path.

Using del.icio.us Bookmarks With Perl

Earlier this week I put live some Perl code that took bookmarks I was posting to del.icio.us and added them automatically to my blog.

I’ve had a few people ask how I was able to do this, and it’s no big secret.

del.icio.us expose an API that anyone can use to interact with their service. I’m using this with some Perl glue to aggregate the previous days posts and put them up on my own site.

The function I’m using is GET, which can take a date as an optional parameter. I use this parameter to get all postings from yesterday. So if the date today is 25th November 2005, to get yesterdays posts from del.icio.us I call the URL http://del.icio.us/api/posts/get?dt=2005-11-24.

All API calls to del.icio.us use HTTP-Auth, these are your site login and password.

To access this data using perl, we can use the LWP modules from CPAN.

my $ua = LWP::UserAgent->new;
$ua->agent('robslinkbot/0.1 (+http://www.robertprice.co.uk)');
my $get_url = 'http://del.icio.us/api/posts/get?dt=' . $yesterday;
my $req = HTTP::Request->new(GET => $get_url);
$req->authorization_basic(USERNAME, PASSWORD);
my $res = $ua->request($req);
if ($res->is_success) {
my $xml = $res->content;
} else {
warn "unable to get content from del.icio.usn";
}

In this case USERNAME and PASSWORD are both two constant values with my username and password defined in. You’ll need to put your own details in here.

You’ll also notice that I’m setting the agent to be robslinkbot/0.1 (+http://www.robertprice.co.uk). del.icio.us specicially request that a user-agent is set as they tend to ban generic user-agents from time to time. If I didn’t set this, it would be set to something like lwp-perl.

So now we have this code, and if it’s worked correctly, I should have my posts from the previous day in the variable $xml, and if it’s not worked, I should have seen a warning informing me that the script was unable to get content from del.icio.us.

We can parse the XML provided very easily using one of Perl’s many XML modules. In this case, I’m going to use XML::XPath.

As we’ll have multiple bookmarks (hopefully), we’ll create a list of hashrefs. The hashrefs will contain the information relating to the post.

Ok, so firstly we create a variable called @posts to store the hashrefs in.

my @posts;

Now we can create our XML::XPath object, and get it to parse the xml we’ve already downloaded from del.icio.us and have in the $xml variable.

my $xp = XML::XPath->new(xml => $xml);

We need to see if the XPath /posts/post exists, if it does it means we have posts to parse.

if ($xp->exists('/posts/post')) {

Now we have to find al the posts and iterate over them

foreach my $posts ($xp->find('//post')->get_nodelist) {

Now all we do is to extract the information we need from each post, store it in our hashref and finally store it in the @posts list.

my $post_hashref;
## we use .= to stringify the find value, must be a better way to do that.
$post_hashref->{'href'} .= $posts->find('@href');
$post_hashref->{'description'} .= $posts->find('@description');
$post_hashref->{'time'} .= $posts->find('@time');
$post_hashref->{'hash'} .= $posts->find('@hash');
$post_hashref->{'extended'} .= $posts->find('@extended');
my @tags = split(/ /, $posts->find('@tag'));
$post_hashref->{'tags'} = @tags;
push @posts, $post_hashref;

You may have noticed that we have split the tags and stored them as a list. I just find this easier to work with.

And that is about it. You should have list you can iterate over, or pass to something like Template Toolkit for displaying. This is a process I use on robertprice.co.uk.

Whatson.com – The Best In UK Commercial Radio

Whatson.com - Kerrang

I’m really pleased as I made the new Whatson.com site live today. It’s the result of several months hard work by a small team, with me as the lead developer.

Currently it’s the home for 8 radio players at present, with more to follow soon!

Kerrang Radio, Kerrang 105.2, Kiss 100 and Magic 105.4 contain a mix of live radio streams, the best of the weeks previous shows and some other special content.

The BBC has been doing something similar for a while with the BBC Radio Player, and ourselves with the Kiss player, but this a first for UK commercial radio.

All the players cross link to each other and we try to offer highlights from other stations to give visitors a real choice of listening.

Anyway, there’s more to come, so keep tuned in.

Whatson.com - Kiss

Whatson.com - Heat

Whatson.com - Smash Hits

Datasherpa And Automatic Page Tagging

A new product called Datasherpa has just been launched by Clickstream Technoligies with the aim of ensuring all pages served by a webserver are automatically loaded with web analytics tags.

They claim their new product eliminates the burden of creating, inserting and testing page tags, and ensures all pages are tracked accurately.

It’s a really simple idea, and bloody good one. We’ve been caught out before at work when a page hasn’t been correctly tagged and we’ve lost valuable traffic information.

It sounds like it would be really simple to build as a mod_perl handler for Apache. The handler would scan each page served, probably using the HTML::Parser module or even just a simple regular expression, detect the closing page </body> tag, and just before that insert the tracking tag corresponding to the virtual host being served.

We use Webtrends at work, and this approach sounds like it should work really well with their system. It may even be worth mocking up a proof of concept quickly.

The Current UK Population In JavaScript

One interesting webpage going round the office yesterday were the UK Population Statistics.

Looking at the figures for 2002-2003 we saw the total UK population grow from 59,321,700 people to 59,553,800 people.

That means the population grows by approximately 1 person every 2 and a half minutes.

I’ve knocked together a quick JavaScript that can calculate the approximate population of the UK assuming this increase is constant.

We work out the per minute increase in population. We also know the starting population of the UK on the 1st January 2003, all we have to do is work out how minutes have elapsed between then and now, and multiply that by the population growth per minute.

var startDate = new Date("January 1, 2003 00:00:00");
var perMinuteIncrease = 232100 / (365 * 24 * 60);
var startPop = 59553800;
// return the estimated current population of the UK.
function currentPopulation() {
var currentDate = new Date();
var diffMinutes = Math.floor((currentDate.getTime() - startDate.getTime()) / 60000);
return Math.round(startPop + (diffMinutes * perMinuteIncrease));
}

To get the current population you just call the currentPopulation() function.

Let’s see what the approximate current UK population is now according to this script…



Just reload/refresh the page to get an updated population count.

Serving MS Excel Documents From A Perl CGI Script

How can you serve a Microsoft Excel spreadsheet from a Perl CGI script?

It’s actually quite easy, and just a case of sending the right headers.

Assuming we have our raw binary Excel sheet in a variable called $excel, we can use this simple block of code to allow it to be downloaded from a web script with the filename of text.xls.

print "content-type: application/vnd.ms-exceln";
print "content-disposition: attachment; filename=text.xlsnn";
print $excel;

This will prompt the user with a file download box asking where to store text.xls.

The key here is the content-disposition tag. We could change the content-disposition to inline to try to force the browser to open the Excel document in browser itself, but that’s not really very friendly.