I’ve been spending a bit of time lately playing around with Ruby on Rails the last couple days, giving it a bit of a test to see what everyone’s raving about and understand how it could be used. Overall it’s a pretty impressive framework, and worth a little time investment to give it a whirl. For those without the patience, I highly recommend viewing the impressive presentation that demonstrates development of a blogging tool in 15 minutes.

That said, all is not sunshine and chocolate in the world of Ruby. I spent the better part of today trying to get some basic SOAP functionality operating to allow me to interact with Amazon Web Services’ E-Commerce Service. I was using Hiroshi Nakamura’s soap4r library to auto-generate Ruby class definitions from a WSDL file, but I was running into a bit of pain. There seems to be next to no information out there about how to use soap4r and the associated wsdl2ruby class generation utility, and even less about the current shortcomings of the current stable release of the library. In the interest of saving someone a day of time, I thought I’d put together some details about the wsdl2ruby tool, the files it generates, and what does and doesn’t work.

To complete this exercise, I assume you have already:

1. Downloaded and installed Ruby (I used 1.8.2)
2. Downloaded and installed soap4r (I used 1.5.5)
3. Downloaded and installed http-access2 (I used 2.0.6)
4. Signed up for an Amazon Web Services developer token (it’s free)

Just a disclaimer: I’m no whiz in the whole SOAP/WSDL arena, but I think the information I’m about to provide you with will be enough to help you figure out what’s going on when using soap4r. Your mileage may vary.

Generating Classes with wsdl2ruby

To create applications capable of accessing web services via SOAP, you could compose raw SOAP requests yourself (see the “Behind the Screens” article for more detail), but that would be a bit painful and require a fair amount of manual labor. I’m a lazy, lazy programmer, and I’m betting you’re the same.

A better approach is to use a framework that can automatically generate class definitions for a framework that can be used to create objects, map those objects to SOAP, and vice-versa. This is exactly what soap4r and the wsdl2ruby provides. Using soap4r, a developer can easily generate both client and server classes to handle consuming and providing SOAP-accessible services. For my purposes, I’m only interested in generating client code to allow me to develop an application that can consume services.

The wsdl2ruby application does exactly what it name implies: it takes a Web Services Description Language definition of a web service, and transforms it into Ruby code. For this exercise, I’m going to use the 2006-03-08 WSDL definition of the Amazon Web Services’ E-Commerce Service web service available here.

To generate client code for the Amazon.com web service from the WSDL description, run wsdl2ruby like this (your platform may require the path to be set appropriately):
wsdl2ruby.rb --wsdl http://webservices.amazon.com/AWSECommerceService/AWSECommerceService.wsdl --type client --force

This command will generate three files:

• AWSECommerceServiceClient.rb: An example client that provides skeleton code for exercising the web service. This code can’t really be run “out-of-the-box”, something I’ll talk about in a moment.
• default.rb: The set of class definitions for all elements defined by the WSDL file. Using these class definitions, a developer will be able to produce and consume the various building blocks required to interact with the web service without needing to search an XML tree, or perform any other similar ugliness.
• defaultDriver.rb: This file contains a single class, AWSECommerceServicePortType, which is used to conduct all requests of the web service.

Although I won’t be doing it for this exercise, you could easily rename default.rb and defaultDriver.rb as you see fit; however, you’ll have to make sure to update any require statements to reflect your new naming.

Using the Generated Sample Client

AWSECommerceServiceClient.rb provides a skeleton application that initializes a AWSECommerceServicePortType object:

#!/usr/bin/env ruby
require 'defaultDriver.rb'

# run ruby with -d to see SOAP wiredumps.
obj.wiredump_dev = STDERR if $DEBUG


and then uses it to call each of the operations made available by the web service. The wsdl2ruby application auto-generates a skeleton for each operation that looks something like this:

# SYNOPSIS
#   ItemLookup(body)
#
# ARGS
#   body            ItemLookup - {http://webservices.amazon.com/AWSECommerceService/2006-03-08}ItemLookup
#
# RETURNS
#   body            ItemLookupResponse - {http://webservices.amazon.com/AWSECommerceService/2006-03-08}ItemLookupResponse
#
body = nil
puts obj.itemLookup(body)

Notice that body is set to nil, whereas the web service requires an ItemLookup object to work. To make this code work, you’d need to create an ItemLookup object – as it turns out, ItemLookup relies on ItemLookupRequest, so you’ll have to create one of those as well. The parameters required to create these objects are determined by the WSDL definition, and the order of parameters to pass to new are documented in part in default.rb; the meaning of those parameters are given in the Amazon Web Services’ E-Commerce Service API documentation.

As an example, let’s say I want to perform a simple lookup for an item with an ASIN of B00005JLXH – which just happens to be the unique Amazon identifier for Star Wars, Episode III (it was the first thing I saw on the Amazon home page, I swear). First I create the specific ItemLookupRequest object for that item:

itemLookupRequest = ItemLookupRequest.new("", "", "", "", "", "", "", ["B00005JLXH"], [], "", "", "", "")

Note that the class constructor generated by wsdl2ruby requires all parameters to be specified (their default value is nil), so you need to provide all the parameters. Use empty strings for the ones you don’t need or want to provide.

Next, I create an ItemLookup object, adding both my developer token and the ItemLookupRequest object I created above.

body = ItemLookup.new("", "Your Amazon Web Services developer token goes here", "", "", "", "", "", [itemLookupRequest])

Finally, I call the itemLookup method on my AWSECommerceServicePortType instance to execute the call to the web service, and print some output using the resulting ItemLookupResponse object as the source of the response data:

itemLookupResponse = obj.itemLookup(body)
 
itemLookupResponse.Items.each do |item|
  item.Item.each do |innerItem|
    puts "ASIN: #{innerItem.ASIN}"
    puts "Detail Page URL: #{innerItem.DetailPageURL}"
    puts "Title: #{innerItem.ItemAttributes.Title}"
  end
end

If you run the AWSECommerceServiceClient application with the –d option and you’ll see should see something like this:
Wire dump:

0


Once the request completes, the itemLookup method will return an ItemLookupResponse object, which will allow you to programmatically access all of the returned data in a simple programmatic fashion via the accessor methds generated by wsdl2ruby. In theory.

In theory, Communism works. In theory.

Unfortunately, wsdl2ruby is still seems to be a work in progress, and therefore doesn’t work as cleanly “out of the box” as one might like. For one thing, it seems to have some difficult with complex types. The example above will undoubtedly choke with something like:


C:/Program Files/ruby/lib/ruby/1.8/soap/mapping/mapping.rb:202:in `const_from_name': private method `sub' called for nil:NilClass (NoMethodError)
from C:/Program Files/ruby/lib/ruby/1.8/soap/mapping/mapping.rb:221:in `class_from_name'
from C:/Program Files/ruby/lib/ruby/1.8/soap/mapping/wsdlliteralregistry.rb:302:in `add_elements2stubobj'
from C:/Program Files/ruby/lib/ruby/1.8/soap/mapping/wsdlliteralregistry.rb:298:in `each'
from C:/Program Files/ruby/lib/ruby/1.8/soap/mapping/wsdlliteralregistry.rb:298:in `add_elements2stubobj'
from C:/Program Files/ruby/lib/ruby/1.8/soap/mapping/wsdlliteralregistry.rb:283:in `soapele2stubobj'
from C:/Program Files/ruby/lib/ruby/1.8/soap/mapping/wsdlliteralregistry.rb:265:in `any2obj'
from C:/Program Files/ruby/lib/ruby/1.8/soap/mapping/wsdlliteralregistry.rb:59:in `soap2obj'
from C:/Program Files/ruby/lib/ruby/1.8/soap/mapping/mapping.rb:146:in `_soap2obj'
... 11 levels...
from C:/Program Files/ruby/lib/ruby/1.8/soap/rpc/driver.rb:178:in `call'
from C:/Program Files/ruby/lib/ruby/1.8/soap/rpc/driver.rb:232:in `help'
from C:/Program Files/ruby/lib/ruby/1.8/soap/rpc/driver.rb:227:in `help'
from D:/Permanent Backup/Development/soap4r-1_5_5/sample/wsdl/test/AWSECommerceServiceClient.rb:20


This problem arise from the lack of a class name being generated for the OperationRequest associated with an ItemLookupResponse. As defined by the WSDL, an ItemLookupResponse is defined as:

<xs:element name="ItemLookupResponse">
<xs:complexType>
<xs:sequence>
<xs:element ref="tns:OperationRequest" minOccurs="0"/>
<xs:element ref="tns:Items" minOccurs="0" maxOccurs="unbounded"/>
</xs:sequence>
</xs:complexType>
</xs:element>

Which results in the following code:

class ItemLookupResponse
  @@schema_type = "ItemLookupResponse"
  @@schema_ns = "http://webservices.amazon.com/AWSECommerceService/2006-03-08"
  @@schema_qualified = "true"
  @@schema_element = [["operationRequest", [<strong>nil</strong>, XSD::QName.new("http://webservices.amazon.com/AWSECommerceService/2006-03-08", "OperationRequest")]], ["items", [<strong>nil</strong>, XSD::QName.new("http://webservices.amazon.com/AWSECommerceService/2006-03-08", "Items")]]]

The problem here is the two highlighted nil class names generated by wsdl2ruby. The ClassDefCreator class in soap4r is responsible for generating this class definition – I took a lookup at the definition and found the following issue in dump_classdef (follow along in your own install of Ruby, in {ruby install path}lib/ruby/1.8/wsdl/soap/classDefCreator.rb):

if element.type == XSD::AnyTypeName
  type = nil
elsif klass = element_basetype(element)
  type = klass.name
elsif element.type
  type = create_class_name(element.type)
else
  type = nil      # means anyType.
  # do we define a class for local complexType from it's name?
  #type = create_class_name(element.name)
  # <element>
  #   <complextype>
  #     <seq ...>
  #   </seq></complextype>
  # </element>
end

The problem is that last type = nil statement. Basically, if ClassDefCreator encounters a complex type in the WSDL, it assigns a nil type. What’s odd is that there appears to be a perfectly good solution currently commented out of the code. If we change:

type = nil

to

type = create_class_name(element.name)

and regenerate the only the class definitions using wsdl2ruby:


wsdl2ruby.rb --wsdl http://webservices.amazon.com/AWSECommerceService/AWSECommerceService.wsdl --classdef –force


then the definition of schema_element in the ItemLookupResponse class changes to:

  @@schema_element = [["operationRequest", ["OperationRequest", XSD::QName.new("http://webservices.amazon.com/AWSECommerceService/2006-03-08", "OperationRequest")]], ["items", ["Items[]", XSD::QName.new("http://webservices.amazon.com/AWSECommerceService/2006-03-08", "Items")]]]

Now, the soap4r framework will be able to find the appropriate class to use to represent the OperationRequest when transforming the response SOAP XML into a object. I’m a little puzzled why this fix is currently commented out in ClassDefCreator – I assume there’s probably a good reason. In all likelihood, this solution is probably commented out because the class name alone isn’t enough to avoid namespace clashes. I’m sure for a more complicated application consuming several web services this would undoubtedly be an issue, but for my purposes, this is not an issue.

Everything works flawlessly. Kinda.

Running the AWSECommerceServiceClient with these changes in place, the application throws another error:

C:/Program Files/ruby/lib/ruby/1.8/soap/mapping/wsdlliteralregistry.rb:71: warning: Object#type is deprecated; use Object#class
C:/Program Files/ruby/lib/ruby/1.8/soap/mapping/wsdlliteralregistry.rb:71:in `soap2obj': cannot map SOAP::SOAPElement to Ruby object (SOAP::Mapping::MappingError)
from C:/Program Files/ruby/lib/ruby/1.8/soap/mapping/mapping.rb:146:in `_soap2obj'
from C:/Program Files/ruby/lib/ruby/1.8/soap/mapping/mapping.rb:59:in `soap2obj'
from C:/Program Files/ruby/lib/ruby/1.8/soap/mapping/mapping.rb:55:in `protect_threadvars'
from C:/Program Files/ruby/lib/ruby/1.8/soap/mapping/mapping.rb:55:in `soap2obj'
from C:/Program Files/ruby/lib/ruby/1.8/soap/rpc/proxy.rb:479:in `response_doc_lit'
from C:/Program Files/ruby/lib/ruby/1.8/soap/rpc/proxy.rb:478:in `collect'
from C:/Program Files/ruby/lib/ruby/1.8/soap/rpc/proxy.rb:478:in `each'
from C:/Program Files/ruby/lib/ruby/1.8/soap/rpc/proxy.rb:478:in `collect'
from C:/Program Files/ruby/lib/ruby/1.8/soap/rpc/proxy.rb:478:in `response_doc_lit'
from C:/Program Files/ruby/lib/ruby/1.8/soap/rpc/proxy.rb:444:in `response_doc'
from C:/Program Files/ruby/lib/ruby/1.8/soap/rpc/proxy.rb:348:in `response_obj'
from C:/Program Files/ruby/lib/ruby/1.8/soap/rpc/proxy.rb:149:in `call'
from C:/Program Files/ruby/lib/ruby/1.8/soap/rpc/driver.rb:178:in `call'
from C:/Program Files/ruby/lib/ruby/1.8/soap/rpc/driver.rb:232:in `help'
from C:/Program Files/ruby/lib/ruby/1.8/soap/rpc/driver.rb:227:in `help'
from D:/Permanent Backup/Development/soap4r-1_5_5/sample/wsdl/test/AWSECommerceServiceClient.rb:20


Further investigation reveals that wsdl2ruby did not generate a Header class, required as part of the HTTPHeaders returned as part of the ItemLookupResponse:

<xs:element name="HTTPHeaders">
<xs:complexType>
<xs:sequence>
<xs:element name="Header" minOccurs="0" maxOccurs="unbounded">
<xs:complexType>
<xs:attribute name="Name" type="xs:string" use="required"/>
<xs:attribute name="Value" type="xs:string" use="required"/>
</xs:complexType>
</xs:element>
</xs:sequence>
</xs:complexType>
</xs:element>

Hence, soap4r is unable to map the returned Headers XML to a Headers class. For some reason, wsdl2ruby seems to choke on nested complex type definitions. Saving the WSDL as a local file and changing the code above to:

<xs:element name="HTTPHeaders">
<xs:complexType>
<xs:sequence>
<xs:element ref="tns:Header" minOccurs="0" maxOccurs="unbounded"/>
</xs:sequence>
</xs:complexType>
</xs:element>
<xs:element name="Header" minOccurs="0" maxOccurs="unbounded">
<xs:complexType>
<xs:attribute name="Name" type="xs:string" use="required"/>
<xs:attribute name="Value" type="xs:string" use="required"/>
</xs:complexType>
</xs:element>


And regenerating the class definitions using

wsdl2ruby.rb --wsdl AWSECommerceService.wsdl --classdef –force


tricks wsdl2ruby into generating the correct Header class definition. As this similar construct exists throughout the WSDL, I performed similar changes throughout – primarily I changed the Arguments WSDL definition to make sure an Argument class is properly generated.

With those changes in place, regenerate the class definitions as before, and run the AWSECommerceServiceClient. This time it should provide the desired output:

ASIN: B00005JLXH
Detail Page URL: http://www.amazon.com/exec/obidos/redirect?tag=ws%26link_code=sp1%26camp=2025%26creative=165953%26path=http://www.amazon.com/gp/redirect.html%253fASIN=B00005JLXH%2526tag=ws%2526lcode=sp1%2526cID=2025%2526ccmID=165953%2526location=/o/ASIN/B00005JLXH%25253FSubscriptionId=0VS96BNQBVY904T3XZ02
Title: Star Wars, Episode III - Revenge of the Sith (Widescreen Edition)


Ah, Closure

Lesson of the day: my pain is your gain. While the wsdl2ruby utility is not fully baked to handle the full flexibility provided by WSDL, it can be cajoled into doing the right thing to get the results you desire. Although I was able to get my simple example working, I’m sure there’s any number of esoteric cases that the soap4r libraries don’t currently handle. Until they do, you’ll have to tinker a bit to get them working. Good luck!