Python - Web Services: The xmlrpclib Module

Web Services: The `xmlrpclib` Module
	Chapter 36. Programs: Clients, Servers, the Internet and the World Wide Web

Web Services: The `xmlrpclib` Module

When we looked at HTTP in the section called “Web Servers and the HTTP protocol”, we were interested in its original use case of serving web pages for people. We can build on HTTP, creating an interface between software components, something called a web service. A web service leverages the essential request-reply nature of HTTP, but takes the elaborate human-centric HTML web page out of the response. Instead of sending back something for people to read, web services use XML to send just the facts.

Web services allow us to have a multi-server architecture. A central web server provides interaction with people. When a person's browser makes a request, this central web server can make web service requests of other servers to gather the information. After gathering the information, the central server aggregates it and builds the final HTML-based presentation, which is the reply sent to the human user.

Web services are an adaptation of HTTP; see the section called “About HTTP” for a summary. Web services rely on a number of other technologies. There are several competing alternatives, so we'll look at web services in general before looking at the xmlrpclib module in particular.

Web Services Overview

There are a number of ways of approaching the problem of coordinating work between clients and servers. All of these alternatives have their advantages and disadvantages.

XML-RPC: The XML-RPC protcol uses XML notation to make a remote procedure call (RPC). It works by sending an HTTP request that contains the name of the procedure to call and the arguments to that procedure. This protocol uses HTTP "POST" requests to provide the XML document.
SOAP: There are two variations on the Simple Object Access Protocol (SOAP): remote procedure call variation and document. The RPC variant is basically the next generation of XML-RPC, where an XML document encodes the name of the procedure and the arguments. The document variant merely sends an XML document; the document provides all the information required by the server. This protocol is heavily supported by additional standards like Web Services Definition Language (WSDL).
REST: The Representational State Transfer (REST) protcol uses the HTTP operations (POST, GET, PUT, DELETE) and Uniform Resource Identifiers (URI) to manipulate remote objects. This protocol is perhaps the simplest of the web services protocols; for this reason it is very popular.

We'll focus on the XML-RPC since it is relatively simple and well-supported by Python. REST is also very popular because it can be done largely using urllib2 features (see the section called “Mid-Level Protocols: The urllib2 Module”).

The essence of RPC is that we are calling a procedure that resides on another, remote computer. In order to do this, the argument values on our local computer must be marshalled and sent through the internet to this remote service. The service must unmarshall the argument values, evaluate the procedure, marshall the results, and send them back to the original requester. Finally, the requester must unmarshall the response.

We have, therefore, three separate issues that we have to address.

Packaging the data. This means writing the argument values in XML notation. We'll see how the xmlrpclib module handles this transformation between XML and Python. The Simple Object Access Protocol (SOAP) is an alternative to the XMLRPC approach to sending objects from one computer to another; it is not widely used in the Python community.
Making the client request. This means marshalling the arguments, making the request, and unmarshalling the response. Since this is based on HTTP, this is a kind of HTTP client, akin to what a browser does when it makes a POST request.
Serving requests. This means unmarshalling arguments, doing something useful, and marshalling a response. Since this is based on HTTP, this is handling a POST request with an XML request, and providing an XML reply.

The essential ingredient in making RPC work is to have a local object which acts as a proxy for the remote service. This ServerProxy appears as if it is doing the work. In fact, it is merely marshalling arguments, transmitting the request via HTTP and unmarshalling the response.

Web Services Client

Let's imagine that a colleague has built a web service which provides us with an extremely good simulation of a roulette wheel. (We'll actually build this in the next section.) Our colleague has provided us with the following summary of this web service.

host: 10.0.1.5. While IP address numbers are the lowest-common denominator in naming, some people will create Domain Name Servers (DNS) which provide interesting names instead of numeric addresses.
port number: 8008. While the basic HTTP service is defined to run on port 80, you may have other web services which, for security reasons, aren't available on port 80. Port numbers from 1024 and up may be allocated for other purposes, so port numbers are often changed as part of the configuration of a program.
path: /. Your HTTP handler may have different families or collections of web services, each with a different path.
method name: spin
return: Python tuple with the outcome. The tuple includes number, color, even/odd, high/low for the result.

To create a web services client, we can use the xmlrpclib module to access an XML-RPC protocol web service. We'll need to define a proxy for this service.

Example 36.2. wheelclient.py

#!/usr/bin/env python
""" Quick Demo of the spin service.
"""
import xmlrpclib

server= xmlrpclib.ServerProxy( "https://10.0.1.5:8008/" )
for i in range(10):
    print server.spin()

	We import the `xmlrpclib` module.
	We synthesize the interface information (protocol, host, port and path) into a URI which identifies the web service. This statement creates a local object that appears to have all of the methods that are part of the remote service.
	We evalulate the spin method on the remote server, and we get back a result that we simply print. In this case, we expect to get a tuple with number, color, even/odd and high/low attributes of the number.

There are some limitations on what kind of structures can be marshalled by the XML-RPC protocol. For example, Python makes a distinction between tuple and list. The XML-RPC protocol, however, can only create lists.

Web Services Server

A web services is usually built into a more complete web application framework. Often the server will have a human interface as well as a web service interface. The human interface will use HTML and port 80. The web service interface will use XML and some other port number, usually a number above 8000. Since web services are built in HTTP, we can adapt our SimpleHTTPServer example toward providing web services.

A common technique is to have a single server process that includes a dispatcher. The dispatch method examines the path of the request to determine which group of web services are being invoked. In this example, we won't include any path dispatching.

The following example shows how to implement the wheel service.

Example 36.3. wheelservice.py

#!/usr/bin/env python
"""Wheel Server.
"""
import SimpleXMLRPCServer as xmlrpc
import random

class Wheel( object ):
    redSet= [1,3,5,7,9,12,14,16,18,19,21,23,25,27,30,32,34,36]
    def _color( self, number ):
        if number in ("0", "00"): return "GREEN"
        elif int(number) in self.redSet: return "RED"
        else: return "BLACK"
    def _even( self, number ):
        if number in ("0", "00"): return "ZERO"
        elif int(number) % 2 == 0: return "EVEN"
        else: return "ODD"
    def _high( self, number ):
        if number in ("0", "00"): return "ZERO"
        elif int(number) <= 18: return "LOW"
        else: return "HIGH"
    def __init__( self ):
        self.rng= random.Random()
        self.wheel= map( str, range(0,37) ) + ["00"]
    def spin( self ):
        number= self.rng.choice( self.wheel )
        return ( number, self._color(number), self._even(number), self._high(number) )
    def spinList( self, spins=1 ):
        return [ self.spin() for i in range(spins) ]

def server():
    theWheel= Wheel()
    service = xmlrpc.SimpleXMLRPCServer(("", 8008))
    service.register_instance( theWheel )
    service.serve_forever()

if __name__ == "__main__":
    server()

	We import the `SimpleXMLRPCServer` module. Since the name is so long, we provide an alias, `xmlrpc`, to make it easier to type. We also import the `random` module.
	This class contains the state and methods that we will expose as a web service. We included some private methods, with names prefaced by `_`. The `SimpleXMLRPCServer` class uses this "leading `_`" convention to identify a private method that isn't published on the web.
	We've defined three private methods, `_color`, `_even`, and `_high`. These methods will be used as part of responding to web service requests.
	We initialize an instance of Wheel by creating a random number generator. We also initialize the set of numbers on the wheel. In this case, we are folding in double zero to make an American-style wheel. A subclass could modify this initialization to create a European wheel with only a single zero.
	The `spin` and `spinList` methods are the public interface to this class. The `spin` method makes a random choice of the numbers on the wheel. It then creates a tuple with the number, the color, the even/odd and the high/low values. The `spinList` method returns a list of tuples by calling `spin` repeatedly.
	The `server` function creates and runs the web server. First, it creates the `Wheel` object, `theWheel`, that we'll use. Second, it creates the server, `service`, using the server's web address and port number. The empty string is a special short-hand meaning "this host's IP address". We register all public methods of `theWheel`. The `server_forever` method handles requests until we stop the Python interpreter by killing the process.
	We use the main switch so that we can easily reuse this `Wheel` class definition.


Web Servers and the HTTP protocol		Mid-Level Protocols: The `urllib2` Module