Distributed applications have faced significant challenges. In the past, when computing resources were scarce and systems were disconnected, distributed application development was difficult for a variety of reasons: network outages were common, architecutres were significantly different, and transport standards did not exist. Developers were responsible for building both business logic and the plumbing required to maintain a distributed computing environment.

Recent advances in computing power and gains in system connectivity have made distributed computing achievable. Distributed applications are not only feasable, but are desirable in many situations. Several computing problems become simpler through the decomposition offered by a distributed computing environment. As a result, several different distributed computing models have evolved, each with different tradeoffs.

The Distributed Computing Environment

The first attempt to create a standardized environment for distributed computing was proposed by the Open Software Foundation. The Distributed Computing Environment (DCE) specified a Remote Procedure Call (RPC) mechanism that allowed developers to make calls to functions residing on remote systems. This offered the first possibility of remote execution, but with the opportunity came challenges.

A significant challenge faced by DCE developers was the issue of “endian-ness”, the order of bits within a byte. Different architectures approach bit ordering using big-endian or little-endian notation. When systems with different architecutres were utilized, developers had to code logic to handle the difference in endian notation. Development of this logic was time consuming, and duplicated across multiple projects. Despite it's weaknesses, DCE was the first accomplishment in the field of distributed computing, and was a notable accomplishment.

Distributed Component Object Model (DCOM) and

Common Object Request Broker Architecture (CORBA)

DCOM and CORBA improved upon DCE by solving the endian problem. Unlike DCE, these protocols marshalled and unmarshalled user data and handled the “endian” problem without assistance from the calling application, leading to heterogenious platform adoption. DCOM and CORBA also made strides in language neutrality: objects managed by a COBOL application were accessible from a C++ application.

DCOM and CORBA also managed the difficulties associated with distributed concurrent access to resources. Multiple clients could access the same object on a server, with DCOM and CORBA managing the concurrent access. Moreover, DCOM and CORBA enabled object reference passing by clients, which allowed complex calls to be made as opposed to simple RPCs.

The benefits of language neutrality, marshalling of program data, management of concurrent access, and portable remote object references led to significant adoption of DCOM and CORBA. The two technologies made distributed computing a viable solution to a large community of developers. Unfortunately, the complexities of the wire protocol steepened the learning curve. Additionally, both protocols were binary, making debugging difficult.

Adding Transaction Management to Distributed Computing

DCOM and CORBA provided a component framework on which to build distributed applications. However, the core specifications lacked support for transaction management, something many applications required in order to be deployed in a distributed model.

The Java2 Enterprise Edition (J2EE) specification built upon DCOM and CORBA by building transaction management into the protocol. J2EE applications can access components managed by CORBA but could also define transaction boundaries at the deployment descriptor level, instead of within application code. The inclusion of transaction management in J2EE led to it's widespread adoption, and enabled a new set of applications to be deployed distributedly.

SOAP: A New Platform for Distributed Computing

DCE, DCOM and CORBA, and J2EE specify the rules for distributed computing in a binary format. Each innovation capitalized on the strengths of it's successor and attempted to improve previous shortcomings. DCE defined rules for remote procedure calls, but didn't solve the endian problem. DCOM and CORBA solved the endian problem, but lacked native transaction support. J2EE included native transaction support, but was language specific (Java). All specifications solved a wide variety of problems, however, each used a binary protocol, was difficult to deploy and debug, and had a very sharp learning curve.

Simple Object Access Protocol (SOAP) is a newcomer to distributed computing. SOAP doesn't solve all the distributed computing problems, but rather focuses on improving specific shortcomings of previous protocols. The SOAP specification came as a result of speedups in computing resources, network connectivity improvements, and overall software technology advancements.

SOAP is a lightweight protocol for information exchange over the Internet. It operates on top of existing technology, using a simple and very flexible text format (Extensible Markup Language (XML)), as well as any application level protocol (but primarily HyperText Transfer Protocol (HTTP)). SOAP supports a variety of invocation styles (request-response, notification, one-way, ...), allowing for a a wide range of applications to be deployed into a distributed environment.

SOAP defines an extensible distributed computing framework that is easy to learn and implement. It is a language on top of an existing text format (XML) that many developers are familiar with. Moreover, because XML is an abstract text format, numerous tools have been developed to assist developers in the formulation, translation, and comprehension of XML documents.

Similarly, instead of definining a new application layer protocol, SOAP uses existing protocols (usually HTTP) that many developers are familiar with. Like XML, many tools have been created to facilitate in the establishment and servicing of these protocols. The strategy of leveraging existing technologies has made SOAP easier to debug and integrate, which has led to wide adoption.

SOAP specifies the rules for exchanging messages and making remote procedure calls. However, exchanging operation information and endpoint locations is a manual process. The client developer must communicate with the server developer to comprehend the interface, as well as the complex type structures used by the service. The client developer must also know the names of the operations as well as the locations that service the operations. To make this process more suitable for machine based processing, the Web Services Descriptor Language (WSDL) was created.

Web Services Descriptor Language

WSDL formalizes web service comprehension by defining service meta information (in XML) for operations callable via SOAP. Because the meta information is defined in XML, machine based comprehension is possible. In WSDL, a developer defines the input and output parameters for an operation. Additionally, a developer defines the faults that may occur during the processing of each operation. Tools have been developed to generate software (in multiple languages) for input and output parameters, as well as faults.

Developers usually construct two documents: an interface definition document, and a binding document. The interface definition document defines the complex types each operation uses, as well as the names of the operations. The binding document defines the access point to use to invoke the operations in the inerface document. The separation of interface from implementation allows the server side developer to re-deploy a service implementation wherever and whenever necessary without impacting existing clients.

The interface document contains complex type structures (e.g. Person, Loan, Account). These types (along with basic primitive types) are defined using XML Schema and are the constructs used to assemble the input and output messages associated with an operation. A “port” is defined to contain operations along with their input and output messages. Fault information is included.

The binding document contains access point information for the port defined in the interface WSDL document. This SOAP endpoint information, as well as encoding rules and operation style (document vs RPC). Encoding rules are roughly analogous to the endian challenges faced by the DCE when multiple architectures were accessed. In the text based world of SOAP, a server must identify which character set (US-ASCII, UTF-8, UTF-16, etc) will be returned from a call. This information is intended for use by the client SOAP library and not the client application.

Universal Description, Discovery, and Integration

Combined, WSDL and SOAP provide a mechanism to describe and expose operations capable of machine basd processing. Universal Description, Discovery, and Integration (UDDI) is a specification that defines a registry for categorizing WSDLs into a hierarchical structure searchable by business, service, or interface. Through UDDI, a developer can expose their interface and binding definition to a large development community.

UDDI allows a client to locate services based upon data types. For example, a developer that needs to convert celsius to fahrenheight can query the UDDI registry for operations from the “Weather” service that accept a floating point number and return a floating point number. Moreover, the developer could search for the operation “float celsiusToFahrenheight(float)”, and results would be returned regardless of the business, service, or interface in which the implementation resides.

UDDI also supports dynamic binding, a process by which a client discovers the endpoint for transmission of SOAP calls. A developer can deploy multiple access points that implement the same interface WSDL, and register binding templates for each endpoint. A client then queries the UDDI registry for the access points to the interface, and binds to the first access point. If unsuccessful, the client attempts binding to the second access point. This loop continues until either the client exhausts the list of access points returned from the UDDI registry, or until binding succeeds.

Tying it all Together: Web Services

Together, UDDI, WSDL, and SOAP provide the plumbing required to build a service oriented architecture whereby a client developer can discover, comprehend, and bind to an operation defined at an endpoint. These three technologies form the basis of “web services”, the next generation of distributed computing. Although Web Services is currently a buzzword, the technologies behind the standards have strong industry support and have been accepted as standards by W3C and other standards organizations. Bindings for web service technologies exist in many languages, and web service connectivity is one of the cornerstones behind Microsoft's .NET initiative.

Web Services for Python

Web Services for Python is an open source project that brings many web services technologies to the Python community. Contained within the project are two implementations of a SOAP 1.1 compliant client and server, each containing WSDL comprehension capabilities. Using Web Services for Python, a Python developer can expose their functions via a SOAP interface. Similarly, a Python developer can bind to operations exposed via SOAP on a server, which may not necessarily be implemented in Python.

Web Services for Python contains two sub-projects, SOAPpy, and the Zolera SOAP Infrastructure (ZSI) – both of which are SOAP implementations with WSDL parsing capabilities. Originally written by Rich Salz, ZSI has experienced several iterations of refactoring, and is overall a stable platform on which to expose Python modules as web services.

Exposing Operations via ZSI: Methods of Dispatch

Exposing Python functions as web services is achieved through the dispatch module provided by ZSI. Multiple dispatching routines are available, each using HTTP as the application layer protocol. The most basic method is dispatching via ZSI's internal web server. The following code demonstrates:

def hello():

return “Hello, World”

from ZSI import dispatch

dispatch.AsServer(port=80)

The above code instructs ZSI to bind to port 80, and expose the methods defined in the current module as web services. The call to dispatch.AsServer() can be appended to any Python module, resulting in all functions exposed as SOAP operations. The AsServer() dispatching method is intended to be used for development purposes - other dispatch methods are encouraged for production environments.

Dispatching from within a CGI environment is achieved using the AsCGI() function. The ability to dispatch requests from within a CGI environment allows ZSI to be deployed into an existing CGI-enabled environment without making very many changes. The following code demonstrates how to dispatch requests from within a CGI environment:

def hello():

return “Hello, World”

from ZSI import dispatch

dispatch.AsCGI()

Dispatching from within mod_python is possible via the AsHandler() method. The mod_python dispatching approach is slightly more complex than the standalone and CGI dispatching methods, but offers significant performance benefits. To dispatch out of mod_python, a developer must define their operations within a separate module, and then reference that module in the handler() method called by mod_python. The only trick is to use __import__ to load the XML encodings necessary to accomplish SOAP marshalling (this is a workaround to avoid the pitfalls of restricted execution mode in mod_python):

MyHandler.py:

def hello():

return “Hello, World”

ModPython.py:

from ZSI import dispatch

from mod_python import apache

mod = __import__('encodings.utf_8', globals(), locals(), '*')

mod = __import__('encodings.utf_16_be',

globals(), locals(), '*')

import MyHandler

def handler(req):

dispatch.AsHandler(modules=(MyHandler,),

request=req)

return apache.OK

The modules argument to the AsHandler() function instructs ZSI which module SOAP requests should be dispatched to. The modules argument is a tuple, allowing for functions in multiple modules to be exposed via SOAP. If the same function exists in multiple modules, ZSI dispatches to the function encountered first in the module tuple.

Complex Types: Server Side Specifics

The examples above demonstrate functions that operate on primitive types. ZSI supports complex type parameters using the TypeCode mechanism. To use the TypeCode mechanism, define a class in a separate module that represents the complex structure to be passed to or returned from an operation. Then augment the class definition with a field (named 'typecode') that contains marshalling and unmarshalling information for the class. The following example demonstrates:

CommonTypes.py:

from ZSI import TC

class Account:

def __init__(self, number=0,

balance=0.0,

ssn=0):

self.number = number

self.balance = balance

self.ssn = ssn

Account.typecode = TC.Struct(Account,

[TC.Ilong('number'),

TC.FPfloat('balance'),

TC.Ilong('ssn')],

'Account')

The 'typecode' field instructs ZSI how to marshall your Account type to and from XML format. In the example above, the Struct TypeCode is used to define a complex structure named 'Account', consisting of three fields: number, balance, and ssn (each having a specific type). Lastly, a name parameter is provided to define what text is to be used when referencing the complex type within XML.

Nesting TypeCodes

The TypeCode mechanism supports nested structure definition. Each element in the list in typecode definition is an instance of a TypeCode. To nest classes within other classes, include their typecode definition thus:

from ZSI import TC

class Person:

def __init__(self, name=''):

self.name = name

class Account:

def __init__(self, number=0, owner=None)

self.number = number

self.owner = owner

Person.typecode = TC.Struct(Person,

[TC.String('name')],

'Person')

Account.typecode = TC.Struct(Account,

[TC.String('number'),

Person.typecode],

'Account')

In the above example, a Person class is defined along with a TypeCode definition. The Account class contains a field named owner of type Person. The TypeCode definition for the Account class references the typecode definition for the Person class in the list parameter to the Struct constructor. Multiple nestings are possible, supporting numerous object models.

Arrays

ZSI also supports the marshalling and unmarshalling of arrays to and from XML via the Array TypeCode. In the situation where a parameter is an array, the Array TypeCode should be used:

from ZSI import TC

class Person:

def __init__(self, favorite_colors=[]):

self.favorite_colors = favorite_colors

Person.typecode = TC.Struct(Person,

[TC.Array('favorite_colors')],

'Person')

ZSI marshalls and unmarshalls other constructs, including dates, times, boolean values, and choices via the TypeCode mechanism. Information regarding these constructs is available on the ZSI project page, and is outside the scope of this document.

Using Complex Types on the Server Side

Complex type definitions are exposed to ZSI by passing the typesmodule parameter to the dispatching methods (AsServer(), AsCGI(), and AsHandler()). The module referenced in the typesmodule parameter is consulted by ZSI anytime a non-primitive type is marshalled on and unmarshalled off of the wire. An example illustrates how to accept a complex type as an argument to a function:

from ZSI import dispatch

import CommonTypes as MyComplexTypes

def createAccount(account):

print account.number

dispatch.AsServer(port=80, typesmodule=MyComplexTypes)

It is not necessary to pass the typesmodule parameter to the dispatch method in the situation where a function returns a complex type. In the situation where the complex type is returned from a function call, ZSI extracts the TypeCode information from the typecode attribute entombed within the instance.

Binding to Remote Operations: The Binding and ServiceProxy classes

Binding to an operation is accomplished using two classes: Binding and ServiceProxy. The Binding class is useful in situations where the SOAP operation is not defined in WSDL, and the parameters to the operation are primitive types. The ServiceProxy class is useful in situations where a WSDL containing binding information is available.

Both Binding and ServiceProxy accept a tracefile parameter as an optional argument in the constructor. The tracefile argument is a file pointer (previously opened for writing) that is used to print the request and response message(s). When provided, ZSI will log all request and response messages to the file pointer provided. This practice is recommended for debugging purposes, as it aids the developer in diagnosing problems. Contained below is an example of how to bind to a SOAP operation, printing the request and response messages to the file 'client.out':

from ZSI.client import Binding

fp = open('client.out', 'a')

server = url='http://localhost/cgi-bin/simple-test',

tracefile=fp)

print server.hello()

fp.close()

In the above example, the client invokes the hello() operation without any parameters. One string value is returned from the invocation. After the above code executes, the file 'client.out' will contain the request and response SOAP messages.

If a binding WSDL document existed, the above code could be re-written using the ServiceProxy class:

from ZSI.client import ServiceProxy

server = ServiceProxy('http://localhost/hello_binding.wsdl')

print server.hello()

Contained within the binding WSDL is the URL to the SOAP endpoint (in this case 'http://localhost/cgi-bin/simple-test'), as well as the interface definition. The interface definition determines which operations exist, and the SOAP endpoint URL determines where requests are to be sent. Both pieces of information are required for ZSI to create a SOAP call.

Complex Types: Client Side Specifics

Passing complex types to SOAP operations is similar to returning complex types from server side operations. Because the instance has been created (and the typecode field defined), ZSI can automatically marshall the complex type onto the wire for transport to the server. There is no need to use the typesmodule attribute when passing a complex type to a SOAP operation:

from ZSI import ServiceProxy

from ComplexTypes import Account

account = Account(1234, 1000000.00, 123456789)

bank = ServiceProxy('http://localhost/bank_binding.wsdl')

bank.createAccount(account)

The typesmodule parameter is required for SOAP calls that return a complex type. Without the typesmodule parameter, ZSI does not know how to unmarshall the complex type data returned from the SOAP operation into a Python object. Following the pattern of explicitness, ZSI requires the typesmodule parameter to tell it what module contains the classes to be instantiated from complex types returned from SOAP operations:

from ZSI import ServiceProxy

import ComplexTypes as MyComplexTypes

bank = ServiceProxy('http://localhost/bank_binding.wsdl',

typesmodule=MyComplexTypes)

account = bank.findAccount(1234)

Enabling Location Based Definitions: The nsdict Parameter

Universal Resource Names (URNs) enable location independent identification of a resource, and can be used to identify complex type structures as well as SOAP operations. In many situations, complex types of the same name are defined in different locations In order to uniquely identify complex types of the same name defined in different locations, a namespace is created using a URN. The URN typically references the Universal Resource Locator (URL) associated with the location.

For example, an “Account” complex type may be defined at two separate locations within a banking organization: accounting and fraud investigation. The definition of an “Account” according to the accounting location consists of an account number and a balance. The definition of an “Account” according to the fraud investigation location contains the same information as the accounting definition, but also includes transaction history. To distinguish the two definitions of an “Account”, namespaces are used: accounting:Account and fraud:Account. The “accounting” namespace is associated with a URL for the accounting department (http://foo.com/accounting) and the “fraud” namespace is associated with a URL for the fraud investigation department (http://foo.com/fraud). Both URLs are unique according to a global naming scheme.

Operations can also be defined in separate locations, requiring URNs for distinction. Using the above example, the accounting location may define the “PresentBankAccount” operation to return the account number and the balance, the fraud investigation location may define the “PresentBankAccount” operation to return the account number, balance, and transaction history. The same namespace mechanism used to distinguish complex types is also used to distinguish operation names.

In some situations, a web service is deployed containing operations defined in multiple locations. Namespaces are required to uniquely identify which operation a client wishes to invoke. ZSI currently supports namespaces used in complex type structures and in the future will support namespaces used in operations.

The last parameter to the TypeCode constructor is the name to use to identity the complex type, which can optionally include a namespace. For example, the following module includes the accounting and fraud definitions of an Account:

from ZSI import TC

class Accounting_Account:

def __init__(self, number=0, balance=0.0):

self.number = number

self.balance = balance

class Fraud_Account:

def __init__(self, number=0,

balance=0.0,

history=''):

self.number = number

self.balance = balance

self.history = history

Accounting_Account.typecode = \

TC.Struct(Accounting_Account,

[TC.Ilong('number'),

TC.FPfloat('balance')],

'accounting:Account')

Fraud_Account.typecode = \

TC.Struct(Fraud_Account,

[TC.Ilong('number'),

TC.FPfloat('balance'),

TC.String('history')],

'fraud:Account')

In the above example, two separate classes are defined to support the accounting and fraud investigation “Account” construct. The typecode definitions for the two classes differ in the name they are associated with: the accounting Account is defined using the name “accounting:Account”, and the fraud investigation Account is defined using the “fraud:Account” name. Both the accounting and fraud namespaces are associated with a URL at runtime when a client binds to a service.

The “nsdict” parameter in the ServiceProxy and Binding constructor encapsulates the URLs associated with namespaces used in complex types. The following example instantiates a ServiceProxy using the complex types defined above:

import ComplexTypes as MyComplexTypes

WSDL = 'http://foo.com/services.wsdl'

nsdict = { 'accounting' : 'http://foo.com/accounting',

'fraud' : 'http://foo.com/fraud' }

bank = ServiceProxy(WSDL, nsdict=nsdict,

typesmodule=MyComplexTypes)

Extending ZSI

ZSI has been developed as a layered architecture. The code that generates SOAP requests and parses SOAP responses is encapsulated within the SoapWriter and Binding classes respectively. Our encapsulations enables additions to made to the project.

Likewise, the WSDL parsing engine is encapsulated within the WSDL class. The WSDL class defines accesor methods for retrieving the operations, types, ports, and services defined within a WSDL. Using the WSDL class, it's possible to extend ZSI to perform code generation based upon the contents of a WSDL.

Because our platform is open, and our architecture is layered, we hope developers will improve ZSI and contribute their additions back to the open source community. In fact, one organization under the leadership of Keith Jackson, developed a WSDL to Python converter, which consumes a WSDL and outputs a Python module containing all of the complex types (and their TypeCode definitions), and a Python module that facilitates binding to the service implementation. We hope that future developers will use ZSI in ways we haven't forseen and will further the web service momentum within the Python community.

Conclusion

Web Services is a newcomer to the field of disributed computing and offers a solution to some of the challenges faced by previous distributed computing platforms. It's is not a silver bullet, and is not appropriate for use in every situation, but web services is well suited to address a specific domain of problems. Web services offers many of the benefits of previous platforms (most importantly language neutrality), and improves upon previous platforms as well.

Web Services for Python is an open source project with the objective to bring web service technologies to Python developers. Through ZSI and SOAPpy, Web Services for Python has achieved significant success, making the Python language a possibility for any developer wishing to deploy or bind to web services.

2003-11-11T22:03:25