My First Python Script

It’s a full day since I started to try to teach myself Python for my Nokia 3230 phone.

My first script is running, and though it isn’t anything impressive, I’m still very pleased with the results.

So what is this amazing script? Well it just gets the current location of the phone by CellID, using the location module supplied with Python, and displays it on the console.

Here’s the code…

# we need access to the location module so have to import it.
import location
# get the mmc, mnc, lac and cellid by calling the gsm_location
# method from the location module.
(mmc, mnc, lac, cellid) = location.gsm_location()
# print out the retrieved details to the console.
print "mmc %sn" % mmc
print "mnc %sn" % mnc
print "lac %sn" % lac
print "cellid %sn" % cellid

screenshot of the python location script results
As you can see it’s not going to win any prizes at PyCon, but it does provide some very practical information for me, namely my location.

But what are those 4 variables returned?

  • MCC = Mobile Country Code
  • MNC = Mobile Network Code
  • LAC = Location Area Code
  • Cell ID = The Cell’s ID 🙂

Stay tuned for more Python adventures over the coming weeks.

It’s Time To Learn Python

Learning Python
No one would have believed, in the first years of the twenty first centry, that mobile affairs were being watched across many timeless RSS feeds. No one could have dreamed that they were being scrutinized as someone studies creatures that swarm and multiply in a drop of water. Few Perl coders even considered the possibility of life with other programming languages. And yet, across the gulf of the blogosphere, a mind immeasurably superior to theirs regarded a certain language with envious eyes, and slowly and surely, he drew his plans to use it.

Yes, I’m fed up waiting for J2ME‘s long development times and restrictive programming model. I’ve been watching Python enviously for a while, and I’ve decided it’s time I added another language to my developers toolbelt.

I’ve gone out and bought O’Reilly’s Learning Python 2nd Edition, and installed Series 60 Python on my Nokia smartphone.

I’m about a quarter of the way through the book, and it all seems fairly simple so far.

Once I’ve got myself up to speed with the standard language, I’m going to move to the Symbian specific stuff. I may then treat myself to Programming Python and the Python Cookbook to get my skills up to a decent level.

Series 60 Python looks to be really full featured offering…

  • 2D Graphics, Images and Full-screen applications
  • Camera and Screenshot API
  • Contacts and Calendar API
  • Sound recording and playback
  • Access to system info such as IMEI number, disk space, free memory etc
  • Rich text display (fonts, colours, styles)
  • Support for Scabale UI
  • Expanded key events
  • Telephone dialing
  • Zip compression
  • Networking support for GPRS and Bluetooth
  • Native GUI widget
  • SMS

In other words, the ability to do nearly anything you want quickly and easily on the phone once you’ve installed the Python sis file.

Another benefit of learning Python, will be the ability to script in Civilization IV. 😉

Framing Pages With JavaScript

Normally, when building a website, we’d not want our content to be in frames.

However, a project at work involved hosting some content externally from the parent site that was framed in the parent site.

That’s fine, nothing to difficult, but this content also provided an RSS feed and those links had to feed back into the parent page, so needed to launch the site framework around itself if it didn’t already exist.

The solution to this was to be have two JavaScript’s, one in the piece of content, and the other in the parent framework. When the content is loaded it needs to check if it’s framed, and if it’s not to load the parent framework which then needs to reload the content.

Here’s the code that goes into the content…

<script language="JavaScript" type="text/javascript">
if (self == top) {
var newurl = "http://www.myparentsite.com/?url=" + encodeURI(window.location.href);
if (document.images) {
top.location.replace(newurl);
} else {
top.location.href = newurl;
}
}
</script>

So what’s going on here? Well firstly we need to check if the content page has been launched in a frameset or not. To do this we check if self is the same as top. self is a JavaScript object that refers to the current page, and top is a JavaScript object that refers to the top of the current page, so this could be the parent page of multiple framesets. If it’s the same we know we’re at the top of the page and not in a frameset, so we need to launch the parent page architecture. To do this we work out what the URL of a parent will be and get JavaScript to tell the browser to goto to that page. If you look at the code here, you’ll see the parent site I’ve called www.myparentsite.com, but this could be anything. I append a parameter called url that is an encoded version of the current page’s URL.

When our parent page loads, it needs to see if it has a URL parameter to load in the content frame correctly. We use the following code on the parent page to achieve this.

<script language="JavaScript" type="text/javascript">
var newurl = decodeURI(location.search.slice(5));
if (newurl) {
// nothing actually :-)
} else {
newurl = "http://www.framedcontent.com/defaultcontent.html";
}
document.write('<iframe src="' + newurl + '" scrolling="no" width="492" height="680" frameborder="0" ></iframe>')
</script>

All we’re doing here is getting a whatever parameter string is passed in and treating it as a URL. If we don’t have a parameter we use a default. We then create a new iframe referring to this URL.

As you can see we make no effort to check the parameter passed in is a valid URL or even if it’s in the url parameter. This is very bad code, but serves to demonstrate the technique. In real life we’d check for the url parameter and that it is valid.

Adding Class Variables To Object Orientated Inline::C

I’ve been working using the Perl module Inline::C lately.

I’ve been using it for some Object Orientated code, and I’ve had the need to have a shared class variable between various C classes. I thought I’d share how I’d accomplished this.

The code I’m using here is based on the object example in Inline::C-Cookbook.

I’m going to add a class variable called count to the example Soldier class.

Firstly we need to add a new line of code to declare this variable

## class variable
static int counter = 0;

Noticed I’ve made the variable static to ensure only one copy of it exists. I’ve also initialised it to 0.

One thing I’ve done, that isn’t really necessary is to add it to the Soldier object. The new Soldier object looks like this…

typedef struct {
char* name;
char* rank;
long serial;
int* count;
} Soldier;

The new line here is int* count, a pointer to an integer called count. We’ll point this at our static counter variable.

To point count to counter we modify the new function to add the following…

solder->serial = serial; ## existing code
soldier->count = &counter;

Here we’re specifically saying that our pointer count should point to the address of counter.

Great, but we need a way of accessing this data. We’ll create a new method called get_count to return the current value of count.

int get_count(SV* obj) {
return *((Soldier*)SvIV(SvRV(obj)))->count;
}

Great, now we can just call get_count from our Soldier object.

By itself this isn’t really much use as we never manipulate the value of counter. To prove this works, just modify the new method to add a line that increments counter. For example,

## increment counter
counter++;

Voila! We we have a static class variable that can now be used with Inline::C objects!

Using SOAP::Lite With Perl

I’ve been trying to access a SOAP service using Perl for a project, so I took a quick look at CPAN to see what was available. The answer seemed to be to use SOAP::Lite.

This module turned out to be a real devil to use, and my problems with it were probably compounded by my lack of SOAP experience.

SOAP (incase you didn’t know) is a method of accessing remote services using XML. In my case, I was trying to access content supplied by a third party ringtone provider using SOAP over HTTP.

Using SOAP::Lite you have to specify a uri and a proxy. What stumped me for a while is that the proxy is the URI of service you wish to access, and uri is the namespace of the service.

So if the provider I was using had their SOAP service available at ws.robstones-services.co.uk/external.asmx, I would use this as the proxy, and in this case I would use ws.robstones-services.co.uk/External as the urinamespace.

Great, I knew the service I needed was called getCallList, and required a Username and Password to be passed to it. In return it would give me a list of valid content types. The SOAPAction header to be added to the call to proxy, letting the remote SOAP service I wanted to use the service http://ws.robstones-services.co.uk/External/getCallList.

My first attempt was the following code…

#!/usr/bin/perl -w
use strict;
use SOAP::Lite 'trace', 'debug';
my $server = SOAP::Lite
->uri('http://ws.robstones-services.co.uk/External')
->proxy('http://ws.robstones-services.co.uk/external.asmx');
my $returned = $server
->getCallList({
'Username' => 'RobsUser',
'Password' => 'RobsPassword'
});
foreach my $type ($returned->valueof('//getCallListResult/string')) {
next unless ($type); ## ignore any undefs
print "$typen";
}

You’ll notice I’m using trace and debug to see the SOAP messages being sent and received. The dialogue from this script was…

> perl -w test.pl
SOAP::Transport::HTTP::Client::send_receive: POST http://ws.robstones-services.co.uk/external.asmx HTTP/1.1
Accept: text/xml
Accept: multipart/*
Content-Length: 615
Content-Type: text/xml; charset=utf-8
SOAPAction: "http://ws.robstones-services.co.uk/External#getCallList"
<?xml version="1.0" encoding="UTF-8"?><SOAP-ENV:Envelope xmlns:xsi="http://www.w3.org/1999/XMLSchema-instance" xmlns:SOAP-ENC="http://schemas.xmlsoap.org/soap/encoding/" xmlns:xsd="http://www.w3.org/1999/XMLSchema" SOAP-ENV:encodingStyle="http://schemas.xmlsoap.org/soap/encoding/" xmlns:SOAP-ENV="http://schemas.xmlsoap.org/soap/envelope/"><SOAP-ENV:Body><namesp1:getCallList xmlns:namesp1="http://ws.robstones-services.co.uk/External"><c-gensym3><Username xsi:type="xsd:string">RobsUser</Username><Password xsi:type="xsd:string">RobsPassword</Password></c-gensym3></namesp1:getCallList></SOAP-ENV:Body></SOAP-ENV:Envelope>
SOAP::Transport::HTTP::Client::send_receive: HTTP/1.1 500 (Internal Server Error) Internal Server Error.
Cache-Control: private
Connection: close
Date: Tue, 29 Nov 2005 15:23:38 GMT
Server: Microsoft-IIS/6.0
Content-Length: 508
Content-Type: text/xml; charset=utf-8
Client-Date: Tue, 29 Nov 2005 15:23:09 GMT
Client-Response-Num: 1
X-AspNet-Version: 1.1.4322
X-Powered-By: ASP.NET
<?xml version="1.0" encoding="utf-8"?>
<soap:Envelope xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema">
<soap:Body>
<soap:Fault>
<faultcode>soap:Client</faultcode>
<faultstring>Server did not recognize the value of HTTP Header SOAPAction: http://ws.robstones-services.co.uk/External#getCallList.</faultstring>
<detail />
</soap:Fault>
</soap:Body>
</soap:Envelope>

The call didn’t work, you’ll see the SOAPAction variable is wrong, it’s http://ws.robstones-services.co.uk/External#getCallList instead of http://ws.robstones-services.co.uk/External/getCallList.

I needed to get SOAP::Lite to use the correct URL. The secret turned out to be to change the use SOAP::Lite line to the following.

use SOAP::Lite on_action => sub {sprintf '%s/%s', @_},
'trace', 'debug';

on_action is a parameter SOAP::Lite uses to separate the URI from the action, here we’re telling it to use /.

This gave me the following SOAP dialogue…

> perl -w test.pl
SOAP::Transport::HTTP::Client::send_receive: POST http://ws.robstones-services.co.uk/external.asmx HTTP/1.1
Accept: text/xml
Accept: multipart/*
Content-Length: 615
Content-Type: text/xml; charset=utf-8
SOAPAction: http://ws.robstones-services.co.uk/External/getCallList
<?xml version="1.0" encoding="UTF-8"?><SOAP-ENV:Envelope xmlns:xsi="http://www.w3.org/1999/XMLSchema-instance" xmlns:SOAP-ENC="http://schemas.xmlsoap.org/soap/encoding/" xmlns:xsd="http://www.w3.org/1999/XMLSchema" SOAP-ENV:encodingStyle="http://schemas.xmlsoap.org/soap/encoding/" xmlns:SOAP-ENV="http://schemas.xmlsoap.org/soap/envelope/"><SOAP-ENV:Body><namesp1:getCallList xmlns:namesp1="http://ws.robstones-services.co.uk/External"><c-gensym3><Username xsi:type="xsd:string">RobsUser</Username><Password xsi:type="xsd:string">RobsPassword</Password></c-gensym3></namesp1:getCallList></SOAP-ENV:Body></SOAP-ENV:Envelope>
SOAP::Transport::HTTP::Client::send_receive: HTTP/1.1 200 OK
Cache-Control: private, max-age=0
Connection: close
Date: Tue, 29 Nov 2005 15:22:53 GMT
Server: Microsoft-IIS/6.0
Content-Length: 405
Content-Type: text/xml; charset=utf-8
Client-Date: Tue, 29 Nov 2005 15:22:25 GMT
Client-Response-Num: 1
X-AspNet-Version: 1.1.4322
X-Powered-By: ASP.NET
<?xml version="1.0" encoding="utf-8"?><soap:Envelope xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema"><soap:Body><getCallListResponse xmlns="http://ws.robstones-services.co.uk/External"><getCallListResult><string xsi:nil="true" /></getCallListResult></getCallListResponse></soap:Body></soap:Envelope>

Bugger, it still didn’t work. I looked again at the SOAP message being sent. It was sending a lot of namespace information and a wrapper around the Username and Password. The wrapper was definitely wrong, and the remote .NET service didn’t seem to like the namespaces.

I needed to sort that out, so I had to tell SOAP::Lite to not to include namespaces or wrap up the Username and Password. The way to do this is to use SOAP::Data to hardcode the data being sent.

This meant changing the call to getCallList to the following…

my $returned = $server
->getCallList(
SOAP::Data->name('Username')->value('RobsUser')->type(''),
SOAP::Data->name('Password')->value('RobsPassword')->type('')
);

The SOAP dialogue after these changes looked like this..

SOAP::Transport::HTTP::Client::send_receive: POST http://ws.robstones-services.co.uk/external.asmx HTTP/1.1
Accept: text/xml
Accept: multipart/*
Content-Length: 548
Content-Type: text/xml; charset=utf-8
SOAPAction: http://ws.robstones-services.co.uk/External/getCallList
<?xml version="1.0" encoding="UTF-8"?><SOAP-ENV:Envelope xmlns:xsi="http://www.w3.org/1999/XMLSchema-instance" xmlns:SOAP-ENC="http://schemas.xmlsoap.org/soap/encoding/" xmlns:xsd="http://www.w3.org/1999/XMLSchema" SOAP-ENV:encodingStyle="http://schemas.xmlsoap.org/soap/encoding/" xmlns:SOAP-ENV="http://schemas.xmlsoap.org/soap/envelope/"><SOAP-ENV:Body><namesp1:getCallList xmlns:namesp1="http://ws.robstones-services.co.uk/External"><Username>RobsUser</Username><Password>RobsPassword</Password></namesp1:getCallList></SOAP-ENV:Body></SOAP-ENV:Envelope>
SOAP::Transport::HTTP::Client::send_receive: HTTP/1.1 200 OK
Cache-Control: private, max-age=0
Connection: close
Date: Tue, 29 Nov 2005 15:19:49 GMT
Server: Microsoft-IIS/6.0
Content-Length: 405
Content-Type: text/xml; charset=utf-8
Client-Date: Tue, 29 Nov 2005 15:19:21 GMT
Client-Response-Num: 1
X-AspNet-Version: 1.1.4322
X-Powered-By: ASP.NET
<?xml version="1.0" encoding="utf-8"?><soap:Envelope xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema"><soap:Body><getCallListResponse xmlns="http://ws.robstones-services.co.uk/External"><getCallListResult><string xsi:nil="true" /></getCallListResult></getCallListResponse></soap:Body></soap:Envelope>

Close, but still not right. The problem was with the getCallList itself, it was still trying to use a namespace.

The final solution was to this problem was to not use getCallList, but instead to use SOAP::Lite’s call method to implicitly set how the function was being called.

The code after making this change was the following…

my $returned = $server
->call(SOAP::Data->name('getCallList')->attr({xmlns => 'http://ws.robstones-services.co.uk/External'}) =>
SOAP::Data->name('Username')->value('RobsUser')->type(''),
SOAP::Data->name('Password')->value('RobsPassword')->type('')
);

This gave me the SOAP dialogue…

> perl -w test_soap.pl
SOAP::Transport::HTTP::Client::send_receive: POST http://ws.robstones-services.co.uk/external.asmx HTTP/1.1
Accept: text/xml
Accept: multipart/*
Content-Length: 524
Content-Type: text/xml; charset=utf-8
SOAPAction: http://ws.robstones-services.co.uk/External/getCallList
<?xml version="1.0" encoding="UTF-8"?><SOAP-ENV:Envelope xmlns:xsi="http://www.w3.org/1999/XMLSchema-instance" xmlns:SOAP-ENC="http://schemas.xmlsoap.org/soap/encoding/" xmlns:xsd="http://www.w3.org/1999/XMLSchema" SOAP-ENV:encodingStyle="http://schemas.xmlsoap.org/soap/encoding/" xmlns:SOAP-ENV="http://schemas.xmlsoap.org/soap/envelope/"><SOAP-ENV:Body><getCallList xmlns="http://ws.robstones-services.co.uk/External"><Username>RobsUser</Username><Password>RobsPassword</Password></getCallList></SOAP-ENV:Body></SOAP-ENV:Envelope>
SOAP::Transport::HTTP::Client::send_receive: HTTP/1.1 200 OK
Cache-Control: private, max-age=0
Connection: close
Date: Tue, 29 Nov 2005 15:16:07 GMT
Server: Microsoft-IIS/6.0
Content-Length: 540
Content-Type: text/xml; charset=utf-8
Client-Date: Tue, 29 Nov 2005 15:15:39 GMT
Client-Response-Num: 1
X-AspNet-Version: 1.1.4322
X-Powered-By: ASP.NET
<?xml version="1.0" encoding="utf-8"?><soap:Envelope xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema"><soap:Body><getCallListResponse xmlns="http://ws.robstones-services.co.uk/External"><getCallListResult><string>ROBS_JAVA</string><string>ROBS_WALL</string><string>ROBS_POLY</string><string>ROBS_MONO</string><string>ROBS_REAL</string><string xsi:nil="true" /></getCallListResult></getCallListResponse></soap:Body></soap:Envelope>
ROBS_JAVA
ROBS_WALL
ROBS_POLY
ROBS_MONO
ROBS_REAL

Hurrah, after all that effort, I have a list of 5 types of content offered by my third party supplier. I can now go ahead and build my service.

SOAP::Lite is a powerful module, but the lack of simple, easy to follow documentation and examples holds it back. I hope this small article helps out other programmers just starting on the SOAP path.

Using del.icio.us Bookmarks With Perl

Earlier this week I put live some Perl code that took bookmarks I was posting to del.icio.us and added them automatically to my blog.

I’ve had a few people ask how I was able to do this, and it’s no big secret.

del.icio.us expose an API that anyone can use to interact with their service. I’m using this with some Perl glue to aggregate the previous days posts and put them up on my own site.

The function I’m using is GET, which can take a date as an optional parameter. I use this parameter to get all postings from yesterday. So if the date today is 25th November 2005, to get yesterdays posts from del.icio.us I call the URL http://del.icio.us/api/posts/get?dt=2005-11-24.

All API calls to del.icio.us use HTTP-Auth, these are your site login and password.

To access this data using perl, we can use the LWP modules from CPAN.

my $ua = LWP::UserAgent->new;
$ua->agent('robslinkbot/0.1 (+http://www.robertprice.co.uk)');
my $get_url = 'http://del.icio.us/api/posts/get?dt=' . $yesterday;
my $req = HTTP::Request->new(GET => $get_url);
$req->authorization_basic(USERNAME, PASSWORD);
my $res = $ua->request($req);
if ($res->is_success) {
my $xml = $res->content;
} else {
warn "unable to get content from del.icio.usn";
}

In this case USERNAME and PASSWORD are both two constant values with my username and password defined in. You’ll need to put your own details in here.

You’ll also notice that I’m setting the agent to be robslinkbot/0.1 (+http://www.robertprice.co.uk). del.icio.us specicially request that a user-agent is set as they tend to ban generic user-agents from time to time. If I didn’t set this, it would be set to something like lwp-perl.

So now we have this code, and if it’s worked correctly, I should have my posts from the previous day in the variable $xml, and if it’s not worked, I should have seen a warning informing me that the script was unable to get content from del.icio.us.

We can parse the XML provided very easily using one of Perl’s many XML modules. In this case, I’m going to use XML::XPath.

As we’ll have multiple bookmarks (hopefully), we’ll create a list of hashrefs. The hashrefs will contain the information relating to the post.

Ok, so firstly we create a variable called @posts to store the hashrefs in.

my @posts;

Now we can create our XML::XPath object, and get it to parse the xml we’ve already downloaded from del.icio.us and have in the $xml variable.

my $xp = XML::XPath->new(xml => $xml);

We need to see if the XPath /posts/post exists, if it does it means we have posts to parse.

if ($xp->exists('/posts/post')) {

Now we have to find al the posts and iterate over them

foreach my $posts ($xp->find('//post')->get_nodelist) {

Now all we do is to extract the information we need from each post, store it in our hashref and finally store it in the @posts list.

my $post_hashref;
## we use .= to stringify the find value, must be a better way to do that.
$post_hashref->{'href'} .= $posts->find('@href');
$post_hashref->{'description'} .= $posts->find('@description');
$post_hashref->{'time'} .= $posts->find('@time');
$post_hashref->{'hash'} .= $posts->find('@hash');
$post_hashref->{'extended'} .= $posts->find('@extended');
my @tags = split(/ /, $posts->find('@tag'));
$post_hashref->{'tags'} = @tags;
push @posts, $post_hashref;

You may have noticed that we have split the tags and stored them as a list. I just find this easier to work with.

And that is about it. You should have list you can iterate over, or pass to something like Template Toolkit for displaying. This is a process I use on robertprice.co.uk.

Using RDF::Redland On Debian

This had me fooled for a few hours, so I thought i’d just record it here incase anyone else is having the same problem.

When installing the RDF::Redland Perl modules to use Dave Beckett’s Redland on Debian with a custom compiled version of Perl, the libraries on CPAN will not work.

Instead, the version of Redland on the current Debian (11th September 2005) is version 1.0.0.2, so you need to download the redland-bindings for 1.0.0.2 on librdf.org, and compile them yourself by running perl Makefile.PL and make, make test and finally make install.

You should now be able to use the RDF::Redland modules successfully.

In London, But OK

It looks like there have been more attacks in London on a bus, and at 3 tube stations.

Although I’m working close to Warren Street, I’m fine.

We’ve been told to stay in the office and not leave.