Using Python and Urllib2 to Access Pages with Basic Authentication

Need to grab material from a site that uses basic authentication? We will show you a quick example of using URLLIB2 and Python to do such.



This is a quick bit of code that connects to protected pages on 'Example.com'. We are going to go with the assumption that the page we want to grab data from, 'orders_waiting.py', simply contains an string representative of the number of orders we have awaiting in the queue. Therefore, we will grab the html, a digit as a string, replace any newline characters, and handle the integer value, reporting if it is a non-zero return.

#!/usr/bin/python
import urllib2
SERVER = 'example.com/pass-protected/'
authinfo = urllib2.HTTPPasswordMgrWithDefaultRealm()
authinfo.add_password(None, SERVER, 'testuser', 'test-user-pass')
page = 'HTTP://'+SERVER+'/cgi-bin/tools/orders_waiting.py'
handler = urllib2.HTTPBasicAuthHandler(authinfo)
myopener = urllib2.build_opener(handler)
opened = urllib2.install_opener(myopener)
output = urllib2.urlopen(page)
out = int(output.read().rstrip('\n'))
if out > 0:
     print 'orders waiting'