LWP::Simple And Proxies

Some Perl scripts I’ve been writing lately have to connect to remote pages to collect data. Normally I use LWP::Simple, but this hasn’t been working too well since we started having to use a proxy server in the office.

One solution is to either set the environment variable http_proxy with the address of the proxy server.

Another is to export the useragent variable $ua when you use the module, and set the proxy details directly in the resulting LWP::UserAgent object.

As an example, we’ll download the contents of my homepage in the variable $page.

use LWP::Simple qw($ua get);
my $page = get 'http://www.robertprice.co.uk/';

I also came across a useful piece on Perl Monks called Getting more out of LWP::Simple, that has lots more useful tips on it.