HTTP Archives - Robert Price

Using Disqus On WordPress Behind A Proxy

I had to implement the Disqus Comment System WordPress plugin on a website that will be located behind an outgoing proxy server. By default the Disqus WordPress plugin does not support proxies, so it is unable to run if a proxy is blocking it’s access to the internet.

Since WordPress 2.8, WordPress supports proxy servers using a few defined values, WP_PROXY_HOST and WP_PROXY_PORT. I have now forked the Disqus WordPress plugin on GitHub, and added support that will look to see if these exist, and use them if they do.

To use it, add the following to your wp-config.php file…

define('WP_PROXY_HOST', 'proxy.yourdomain.com');
define('WP_PROXY_PORT', '3128');

Changing the values to match those of your proxy server of course.

Now replace the url.php file in wp-content/plugins/disqus-comment-system/lib/api/disqus/url.php with the url.php found in my github repository.

Visit your WordPress admin panel and you should now be able to activate and configure the Disqus plugin successfully.

I have issued a pull request for my changes to be pulled back into the main plugin, but it’s up to Disqus is they want to implement this or not.

POSTing JSON To A Web Service With PHP

I needed to POST JSON formatted data to a RESTful web service using PHP earlier, so I thought I’d share my solution.

There are a couple of approaches that could be taken, using the CURL extension, or file_get_contents and a http context. I took the later way.

When POSTing JSON to a RESTful web service we don’t send post fields or pretend to be a form, instead we have to send the JSON data in body of the request, and let the web service know it’s JSON data by setting the Content-type header to application/json.

$article = new stdClass();
$article->title = "An example article";
$article->summary = "An example of posting JSON encoded data to a web service";

$json_data = json_encode($article);

$post = file_get_contents('http://localhost/rest/articles',null,stream_context_create(array(
    'http' => array(
        'protocol_version' => 1.1,
        'user_agent'       => 'PHPExample',
        'method'           => 'POST',
        'header'           => "Content-type: application/json\r\n".
                              "Connection: close\r\n" .
                              "Content-length: " . strlen($json_data) . "\r\n",
        'content'          => $json_data,
    ),
)));

if ($post) {
    echo $post;
} else {
    echo "POST failed";
}

Here I’m first creating an example PHP stdClass object, populating it with some data, then serialising it to a JSON string. The real magic is using file_get_contents to POST it over HTTP to my test web service. If the POST succeeds, then it’s displayed, else an error message is shown.

It’s important to note I send the header, Connection: close . Without this, your script will hang until the web server closes the connection. If we include it, then as soon as the data has POSTed the connection is closed and control returned to the script.

HTTP Methods For RESTful Web Services

The use of HTTP methods when designing and building a RESTful web service is very important.

Most developers only use the GET and POST methods offered by the HTTP specification, and for most web sites this is fine.

However, there are other useful methods in the specification that are often overlooked by web developers, but as a RESTful web service developer you will need to know. The most important of these are PUT and DELETE.

REST uses HTTP methods as a one to one mapping to the CRUD (Create, Read, Update and Delete) operations you may be familiar with as a developer. These are

Create a resource – POST
Read a resouce – GET
Update a resource – PUT
Delete a resource – DELETE

There are also two other terms associated with RESTful services that you should be aware of, “safe” and “idempotent”.

A safe request is a request to read some data, not to change any server state. GET requests are safe as you should be able to GET a resource any number of times without affecting it’s state.

An idempotent request is request that however many times it is invoked, the end result is the same. GET, PUT and DELETE are all idempotent. You should be able to DELETE a resource any number of times, once deleted it’s gone, trying to delete it again won’t change the fact it’s gone.

POST is the method that causes problems, it is neither safe or idempotent. Using our CRUD example from earlier, POSTing to a server, could create duplicate resources.

For more details on other HTTP methods, such as HEAD and OPTIONS, have a look at RFC2616 – The HTTP 1.1 specification.

Uploading Files To A LAMP Server Using .NET

We had to setup and run a webcam for the Q Awards red carpet earlier this week.

The original plan was to have an old Axis ethernet webcam, connected to a wifi network using a Dlink bridge. After a nightmare of not being able to get the Dlink to work, we gave up and went for a USB webcam connected to a laptop approach.

I had to write some software to handle the capture of the image and the upload to the webserver. Because we wanted to do a few custom things on the way we weren’t able to use out of the box software.

I’ll post on how to use a webcam with .NET another time. What I wanted to document was how to upload a file using .NET to a LAMP server.

It turns out to be easier than I thought for .NET, one line of code can achieve this using the My.Computer.Network.UploadFile Method.

For example, to upload test.txt to http://www.robertprice.co.uk/asp_upload.pl we can do the following (in Visual Basic)…

My.Computer.Network.UploadFile("C:test.txt","http://www.robertprice.co.uk/asp_upload.pl")

Now we need some code at the other end to store this upload. As I was building a webcam application, the uploaded file, an image in this case, was always overwritten.

The uploaded .NET file uses an bog standard HTTP upload as described in RFC 1867.

If we use Perl to read this file, we can use the standard CGI.pm module to do most of the hard work for us.

The uploaded file is passed a parameter called file, so we can create a CGI object and get a reference to this using the param method. Once we have this, we can treat it as a filehandle and read the date off it. Then all we have to do is to write it out to a file.

The following code achieves this.

#!/usr/bin/perl -w

use strict;

use CGI.pm;

my $CGI = new CGI;

my $file = $CGI->param('file');

open(FILE, ">/path/to/my/file/test.txt") or die $!;

while(<$file>) {

print FILE $_;

}

close(FILE);

print "Content-type: text/htmlnn";

print "OKn";

Using Twitter From Perl

The world and his dog is currently looking at Twitter and eyeing up the possibilities it offers.

I thought i’d jump on the bandwagon, and have a look at the Twitter API.

I wanted to post to a timeline, so the solution is to use one of the update methods. I chose to the XML one, though there is a JSON one also available.

To post to the timeline, Twitter expects an HTTP POST request with a status parameter containing the message you want to post. It associates this to your account by using HTTP’s basic authorization functionality.

It’s simple to throw together a spot of Perl code to post messages to Twitter knowing this. Have a look at this example…



my $message = "A test post from Perl";

my $req = HTTP::Request->new(POST =>'http://twitter.com/statuses/update.xml');

$req->content_type('application/x-www-form-urlencoded');

$req->content('status=' . $message);

$req->authorization_basic($username, $password);

my $resp = $ua->request($req);

my $content = $resp->content;

print $content;

You need to set $username and $password to your username and password, and $message to whatever message you want to appear on your timeline (in this case, “A test post from Perl”).

Lifeblog Proxy Idea

Sitting in a Lifeblog debrief earlier, one thing that struck me was that others had the same problem as me regarding wanting to post to multiple blogs.

It seems most would like to seperate a work blog from a personal blog, but unless it’s hosted on the same Typepad account for example, Lifeblog doesn’t let you do this. From a service point a of view it’s a one to one match.

Posting on Lifeblog

Sitting there, my mind was mulling the problem over, and it would appear that a simple Lifeblog proxy would solve the problem. If blogs are hosted on the same service and accessible by the same username and password, Lifeblog lets you post to different blogs. Why not just build a service that can proxy between various Lifeblog compatible blogs, so you wouldn’t have to host them all together.

Posting on Lifeblog via a proxy

So how may this work from a technical perspective.

Well Lifeblog posts using a flavour of the Atom protocol. For security it uses WSSE encryption on the posts. This means that the proxy would need to it’s own username and password to authenticate against when talking to Lifeblog. The various blogs it would be proxying onto would also need different username and passwords, and proxy would have to insert these as it passes the post onto the relevant blog. We could potentially store all the blogs we’re allowing posts to in an XML config file. For example…

<blogs>

<blog>

<name>My Blog</name>

<url>http://work.blog.com/post.pl</url>

<username>robertprice</username>

<password>secret</password>

</blog>

<blog>

<name>My Blog 2</name>

<url>http://my.website.com/post.pl</url>

<username>rob</username>

<password>lifeblog</password>

</blog>

</blogs>

Here all the blogs are listed, along with their name, posting url, username and password. The proxy would take this list and return a localised list of blogs that when posted to, would just pass the relevant data across. So this means there are two areas to break the proxy down into.

First, the list of blogs. This reads the XML and returns a list of localised blogs and posting URL’s that Lifeblog can use to upload content.

Secondly, the actual localised posting URL needs to remove the Lifeblog WSSE authentication, and replace it with the correct username and password for the real blog before passing it on to the real upload URL.

It could be as simple as that. Maybe I’ll mock something up in Perl to test the theory out.

Anyway, who’s to say this just has to proxy Lifeblog. It could alternatively be a gateway that could translate into one of the common blogging API’s, instantly opening up Lifeblog to millions more users. Now that would be cool!

UPDATE 23/04/05

Hugo emailed me to say Lifeblog 1.6 can handle some of what I have suggested…

Actually, Lifeblog 1.6 can have post to more than one account, and is
available for the Nokia 6630, 6680, 6681, 6682. Unfortunately Lifeblog
1.5 (for 7610, 6670, 6260, 3230) can only post to one blog. And the PC
can post to multiple accounts.

WSSE Authentication For Atom Using Perl

Atom uses the WSSE authentication for posting and editing weblogs.

Mark Pilgrim explains more about this in layman’s terms in an old XML.com article, Atom Authentication.

This information is passed in an HTTP header, for example…

HTTP_X_WSSE     UsernameToken Username="robertprice",  PasswordDigest="l7FbmWdq8gBwHgshgQ4NonjrXPA=", Nonce="4djRSlpeyWeGzcNgatneSA==", Created="2005-2-5T17:18:15Z"

We need 4 pieces of information to create this string.

Username
Password
Nonce
Timestamp

A nonce is a cryptographically random string in this case, not the word Clinton Baptiste gets in Phoenix Nights (thanks to Matt Facer for the link). In this case, it’s encoded in base64.

The timestamp is the current time in W3DTF format.

The for items are then encoded together to form a password digest that is used for the verification of the authenticity of the request on the remote atom system. As it already knows you username and password, it can decrypt the password the nonce and timestamp passed in the WSSE header. It uses the well known SHA1 algorithm to encrypt the pasword and encodes it in base64 for transportation across the web.

We can use Perl to create the password digest, as shown in this example code.

my $username = "robertprice";

my $password = "secret password";

my $nonce = "4djRSlpeyWeGzcNgatneSA==";

my $timestamp = "2005-2-5T17:18:15Z";

my $digest = MIME::Base64::encode_base64(Digest::SHA1::sha1($nonce . $timestamp . $password), '');

The password digest is now stored in the variable $digest.

We can also create the HTTP header from this if needed.

print qq{HTTP_X_WSSE     UsernameToken Username="$username", PasswordDigest="$digest", Nonce="$nonce", Created="$created"n};

Please note, to use this Perl code, you have to have the MIME::Base64 and Digest::SHA1 modules installed. Both are freely available on CPAN.

Update – 22nd November 2006

Some more recent versions of Atom expect the digest to be generated with a base64 decoded version of the nonce. Using the example above, some example code for this would be…



## generate alternative digest

my $alternative_digest = MIME::Base64::encode_base64(Digest::SHA1::sha1(MIME::Base64::decode_base64($nonce) . $timestamp . $password), '');

When using WSSE for password validation, I now always check the incoming digest with both versions of my generated digested to ensure it’s compatible with different versions of Atom enabled software. One of the best examples of this is the Nokia Lifeblog. Older versions expect the nonce to be left, newer versions expect the nonce to be decoded first.

Serving MS Excel Documents From A Perl CGI Script

How can you serve a Microsoft Excel spreadsheet from a Perl CGI script?

It’s actually quite easy, and just a case of sending the right headers.

Assuming we have our raw binary Excel sheet in a variable called $excel, we can use this simple block of code to allow it to be downloaded from a web script with the filename of text.xls.

print "content-type: application/vnd.ms-exceln";

print "content-disposition: attachment; filename=text.xlsnn";

print $excel;

This will prompt the user with a file download box asking where to store text.xls.

The key here is the content-disposition tag. We could change the content-disposition to inline to try to force the browser to open the Excel document in browser itself, but that’s not really very friendly.