Signing Amazon Web Service Requests in ActionScript

Amazon.com announced a change to its Product Advertising API requiring all requests after August 15, 2009 to be signed. I’d been meaning to update one of the Scannerfly example applications and the Shelfari Scanner to sign Amazon requests, but was hoping someone else would figure it out before me. As near as I can tell, no one has provided an implementation in ActionScript, so I cobbled one together.

Generally, the process is fairly straighforward: developers need to transform their REST request’s parameters into a canonical form, add a timestamp, and then append a signed version of the resulting string. However, in reality there a lots of finicky details that make the process frustrating…luckily for you, I’ve created an ActionScript implementation that you should be able to modify to suit your purposes.

The signature process relies on the as3crypto library to provide HMAC and SHA-256 implementations. In addition, you’ll need an instance of mx.rpc.http.HttpService that you’ve likely instantiated in your MXML, along with your Amazon Web Services developer ID and secret key:

// A HTTPService configured to perform the required Amazon Web Services request.
<mx:HTTPService id="AmazonSearch" url="http://webservices.amazon.com/onca/xml" showBusyCursor="true">
	<mx:request>
		<awsaccesskeyid>{amazonDeveloperId}</awsaccesskeyid>
		<idtype>EAN</idtype>
		<itemid>9781592400874</itemid>
		<operation>ItemLookup</operation>
		<responsegroup>ItemAttributes,Images,Tracks,EditorialReview</responsegroup>
		<searchindex>Books</searchindex>
		<service>AWSECommerceService</service>
		<signature>{signature}</signature>
		<timestamp>{timestamp}</timestamp>
	&lt;/mx:request&gt;
&lt;mx:HTTPService&gt;
 
// The Amazon host providing the Product API web service.
private const AWS_HOST:String = "webservices.amazon.com";
 
// The HTTP method used to send the request.
private const AWS_METHOD:String = "GET";
 
// The path to the Product API web service on the Amazon host.
private const AWS_PATH:String = "/onca/xml";
 
// The AWS Access Key ID to use when querying Amazon.com.
[Bindable]
private var amazonDeveloperId:String = "####################";
 
// The AWS Secret Key to use when querying Amazon.com.
[Bindable]
private var amazonSecretAccessKey:String = "####################";
 
// The request signature string.
[Bindable]
private var signature:String;
 
// The request timestamp string, in UTC format (YYYY-MM-DDThh:mm:ssZ).
[Bindable]
private var timestamp:String;
 
private function generateSignature():void
{
	var parameterArray:Array = new Array();
	var parameterCollection:ArrayCollection = new ArrayCollection();
	var parameterString:String = "";
	var sort:Sort = new Sort();
	var hmac:HMAC = new HMAC(new SHA256());
	var requestBytes:ByteArray = new ByteArray();
	var keyBytes:ByteArray = new ByteArray();
	var hmacBytes:ByteArray;
	var encoder:Base64Encoder = new Base64Encoder();
	var formatter:DateFormatter = new DateFormatter();
	var now:Date = new Date();
 
 
	// Set the request timestamp using the format: YYYY-MM-DDThh:mm:ss.000Z
	// Note that we must convert to GMT.
	formatter.formatString = "YYYY-MM-DDTHH:NN:SS.000Z";
	now.setTime(now.getTime() + (now.getTimezoneOffset() * 60 * 1000));
	timestamp = formatter.format(now);
 
	// Process the parameters.
	for (var key:String in AmazonSearch.request )
	{
		// Ignore the "Signature" request parameter.
		if (key != "Signature")
		{
			var urlEncodedKey:String = encodeURIComponent(decodeURIComponent(key));
			var parameterBytes:ByteArray = new ByteArray();
			var valueBytes:ByteArray = new ByteArray();
			var value:String = AmazonSearch.request[key];
			var urlEncodedValue:String = encodeURIComponent(decodeURIComponent(value.replace(/\+/g, "%20")));
 
 
			// Use the byte values, not the string values.
			parameterBytes.writeUTFBytes(urlEncodedKey);
			valueBytes.writeUTFBytes(urlEncodedValue);
			parameterCollection.addItem( { parameter : parameterBytes , value : valueBytes } );
		}
	}
 
	// Sort the parameters and formulate the parameter string to be signed.
	parameterCollection.sort = sort;
	sort.fields = [ new SortField("parameter", true), new SortField("value", true) ];
	parameterCollection.refresh();
	parameterString = AWS_METHOD + "\n" + AWS_HOST + "\n" + AWS_PATH + "\n";
	for (var i:Number = 0; i < parameterCollection.length; i++)
	{
		var pair:Object = parameterCollection.getItemAt(i);
 
 
		parameterString += pair.parameter + "=" + pair.value;
 
		if (i < parameterCollection.length - 1)
			parameterString += "&";
	}
 
	// Sign the parameter string to generate the request signature.
	requestBytes.writeUTFBytes(parameterString);
	keyBytes.writeUTFBytes(amazonSecretAccessKey);
	hmacBytes = hmac.compute(keyBytes, requestBytes);
	encoder.encodeBytes(hmacBytes);
	signature = encodeURIComponent(encoder.toString());
}
 
...
 
// Somewhere in your code you'll call the following to generate request signature and perform the search.
generateSignature();
AmazonSearch.send();

And for those who need a complete working example, you can download the MXML for an example application from here. The example simply performs an ItemLookup; however, you will still need to add your Amazon developer ID and secret key for the example to work.

A note to other developers struggling with implementing the proper request signature, see the Signed Request Helper. This Javascript application breaks down each step in the formulating the normalized parameter string and signature.

Bad User Experience: Country-Specific Media Embeds

More and more, there appear to be cracks appearing in the unified facade of media on the Internet – and this is not a good thing. The example which irks me the most is when traditional media hamfistedly attempts to put content online, and then erects artificial barriers to viewing that content based on the user’s geographic location. A perfect example: the behaviour of Comedy Central’s embedded videos when viewed from Canada.

If you’re cruising through the Internet from Canada and come across an embedded Comedy Central video, this is what you see:

Available on the Internet! *except in Canada

Available on the Internet! *except in Canada

I imagine the conversation at Comedy Central went something like this:

  1. “Hey! Let’s put our content online to drive awareness of our programs!”
  2. “And you know what would be cool? What if we let people embed our content on other sites, essentially advertising our programs to new audiences for free!”
  3. “Hmm, wait a minute. We’d better make sure we don’t piss off partners in other countries that license our content – let’s prevent people in other countries from viewing that embedded content.”

I understand the logic of #3 – Comedy Central licenses its content to other media players in other countries (in Canada, it’s the Comedy Network – a near-verbatim copy of Comedy Central). Of course, those licensees want people to understand that in certain geographies, the content is available from them, not Comedy Central. The problem is that #3 completely undermines #1 and #2; as implemented, the licensee completely fails to capitalize on the distributed, border-less nature of the Internet.

Instead of gaining a follower, this behavior simple annoys the user – after all, who really says to themselves “Oh, I get it – I guess I’ll just navigate to the Comedy Network and hunt down this clip to see it“? Nobody, that’s who. Instead,  the user says to themselves: “Wow, these guys are idiots because they don’t let me see the video. Oh well, luckily there’s a whole bunch of other stuff on the Internet. I’ll just go and watch those things instead.”

All it does is annoy the very audience the company is trying to reach:

Dear Comedy Central: Stop Being Stupid

The funny part here is that there is a simple solution: simply brand the player interface depending on the user’s location. If the user is from the US, brand it as “Comedy Central”; if it’s in Canada, brand it as “The Comedy Network”. If the user clicks on the embed, take them to the right site for their country. Easy.

My advice for media companies: as you venture online, ask yourself how you can interest as many people as possible in your content, and make the country-specific licensee issues as transparent to the user as possible.

Does It Matter If The Future Isn’t Available in Canada?

Vito Pilieci has written an interesting rebuttal to Macleans’ “You can’t buy that here” article – an article which mirrors many of the concerns I raised in my own “Borders Keep Out Innovation, Too” post. I can’t fault Vito’s logic – yes, Canadian developers could create iPhone applications using the SDK even before the device arrived in Canada; yes, Canadians can technically read e-books on iPhones, netbooks, laptops, or other devices without waiting for Amazon’s Kindle device; and yes, Canada is no more disadvantaged with respect to services like Hulu than countries like Japan, Korea, or even the entirety of Europe.

And yet, despite such a resounding thumping, there is still a ring of truth to the original article. A niggling, unsettling je ne sais quoi that speaks to a core injustice that unsettles Canadian technophiles. It’s not that Vito’s arguments aren’t sound – it’s just that they’re so unsatisfying.

Sure, you could develop iPhone applications in the simulator – but you would miss the experience of using the device in your daily life, of truly understanding the implications, applications, and untapped potential of the device. And yes, you could curl up with an ebook on your laptop or smartphone – despite the fact that the form factor is completely uncomfortable, and the screen technology strains your eyes. And yes, you could take comfort in the fact that you’re no worse off than consumers in any other country outside the US.

But every day, you’d still be getting pummeled with advertising on TV, in magazines, and on the net about The Next Big Thing, How Great It Is, only to discover it’s Coming Real Soon to Canada. And it would eat at you.

It’s neither a new problem, nor is it limited to technology. If you’re Canadian, you probably have noticed there’s an awful lot of extremely successful Canadians and, oddly enough, they all seem to have become extremely successful by leaving Canada, working outside Canada, and living the rest of their lives outside Canada. And you’ve probably suffered the simultaneous sensations of both giddy pleasure and guilty shame when any of these representatives of the Great White North receive any recognition in international press. The un-articulated outrage expressed in the Macleans article is an extension of this festering inferiority complex that riddles the Canadian national psyche.

It reminds me of an American Express ad that ran in Canada a couple years ago: the advertisement espoused that you could live the life you wanted (presumably using an American Express), but carried the laughable legal rider “Offer not available in Quebec” – apparently if you lived in Quebec, you were out of luck. And that’s how technophiles feel when the invisible, intangible web of national borders and legal frameworks or the lack of an insufficiently large population base denies them access to new technologies. These technologies promise freedom, capabilities, possibilities – they promise the future, and yet in a cruel twist of fate, these promises are held just out of reach by rules that many technophiles regard as either antiquated or out of touch with the borderless landscape of the Internet.

So while I can’t say that Vito is wrong, I can’t bring myself to say that he’s right either. Yes, there are other, more complicated issues that retard the Canadian technology industry (Vito names telecommunications regulation, investment regulations). But getting people to stand up and force politicians to take action on these issues requires them to understand what’s at stake, and what’s in it for them. Macleans’ attempt to point out how Canada is missing out on the future, however small a piece of it, seems like a valid tactic despite the weakness of its execution.

The Comedy Network Inadvertantly Advertises Porn Site

The other night while The Comedy Network (the Canadian equivalent of Comedy Central), I spotted an odd “Emergency Alert System” warning (see the video). I couldn’t quite put my finger on it why it struck me as odd until I realized I hadn’t seen an emergency warning since I left the US. In fact, I couldn’t recall seeing any type of emergency system warning in Canada. Ever.

And then my wife noted that the URL specified at the end of the alert led to a porn site. Whoops!

It turns out, some clever/enterprising/unscrupulous operator has hit upon an unusual opportunity. It turns out that the emergency alert was actually a commercial for a new show called Hotbox , and the associated URL for the show (www.hotbox/thecomedynetwork.ca) is incorrect. If the user enters this URL, then the user’s browser will attempt to correct the malformed URL into “www.hotbox.com/thecomedynetwork.ca” – which is a porn site. New versions of the commercial attempted to correct the URL to hotbox.thecomedynetwork.ca; however, this is still prime for a unique typing error – if the user simply mistypes the URL as “hotbox/thecomedynetwork.ca” they’ll end up at the same porn site.

Now, this could all be a coincidence/an error. But it’s especially funny given the faux Emergency Alert/ad states a “hotbox warning” is in effect for several towns with somewhat rude names: Shag Harbour, Crotch Lake, Little Bullhead, Beaverlodge.

Borders Keep Out Innovation Too

Since returning from living in Silicon Valley, I’ve been in the grips of something my father calls “expatriate blues” – it involves looking around at my new environment and critiquing what’s different, and not always in a positive manner. This is a natural response whenever you move countries – I’ve lived in five countries over the past dozen years, so this par for the course. So, without further ado, allow me to expound on my latest “expatriate blues”-derived pet peeve: the border.

It’s not that I’m against the border per se. Borders exist for a couple good reasons – ensuring the government’s ability to enforce laws, collect taxes, and keep out undesirables (although that seems to be increasingly interpreted as “primarily poor, unskilled immigrants”). That stuff is all good. Unfortunately, as technology becomes increasingly content-rich and network-dependent, we’re seeing a new generation of technologies that get tripped up at the border in the intricacies of international copyright law and licensing schemes.

I’m not the first one to lament the technologies not available in Canada from our neighbor to the southHulu, Mint, Netflix, an Amazon.com that doesn’t suck, hellooo? But this is nothing new for Canada – remember the iTunes store? Introduced in the US in April of 2003 and shortly thereafter in Canada in…December 2004. The streak continued with the iPhone (introduced in June 2007 in the US, July 2008 in Canada), and continues to this day with the continued unavailability of Amazon.com’s Kindle device. And, in case you didn’t realize, that last one also includes the Kindle iPhone App – it’s not available in Canada, which seems silly for a software application that can download books over wireless Internet.

In a world of technology that’s supposed enhance our capabilities and remove barriers to markets, it appears the last true barrier to market is still firmly in place. But why should we care? After all, all the innovations I’ve listed could be easily dismissed as trivial entertainment products. It’s not like we’re not getting access to lifesaving medical devices, right?

Wrong – that’s an incomplete and flawed argument. While it’s true these devices may appear to be frivolous, the future often arrives in unexpected forms. The real reason to be concerned about the unavailability of these services in Canada is that they impede Canada’s ability to innovate and to exploit new markets. In some cases, it is possible for engineers and others to work around the border – a US credit card here, a Washington state drop box there, a US IP address proxy service somewhere else – but these are all hacks. These hacks introduce friction, friction which slow down the system and places the country at a disadvantage to gain momentum.

If you’re a fan of Malcolm Gladwell, some of this argument might sound familiar. In his latest book, Outliers, Gladwell notes that many of the hockey players in the NHL are born between January 2nd, and the end of April. The reason? The cutoff dates for age brackets in the minor leagues is January 1st – meaning that anyone born on January 2nd ends up in a lower age bracket. These players’ age advantage translates into superior motor control and size – characteristics that predispose the players to appear more capable than others in their bracket. This means these players get picked for special coaching, which kicks off a virtuous feedback cycle. The players get better, and because they appear to be better players, they get more coaching, which makes them even better players. And so on.

The same is true for innovation. Without access to these products, Canada is placed at a disadvantage, just like a player born on January 1st. Smallest of the litter, slowest of the pack. Not only do our entrepreneurs not get the opportunity to build on top of these platforms, but our own companies are not forced to compete in the global market. This not only means that Canadians don’t get access to better products, but also that Canadians don’t learn how to build better products themselves for export to the world.

Canada needs to move quickly to remove this barrier, or we may end up finding ourselves out of much more than the latest shiny toy.

CBC Cuts: Inevitable?

cbcA friend lamented the impending CBC cuts via Twitter earlier today, retweeting Kevin Smith‘s exclamation:

someone explain the logic behind cutting the CBC, maker of Hockey Night in Canada, and bailing out GM, maker of the Hummer?

Now, I’m hardly one to defend the bailout of the auto industry, but given that cars are amongst Canada’s biggest imports and exports it seems a prudent decision. After all, if you’re going to violate the laws of the free market, you might as well go big.

But that alone doesn’t seem like a justifiable reason to leave the CBC out in the cold, now does it? Yet I still have no sympathy for the CBC – why? Here’s why:

Unlike the BBC, which is typically led by a board-appointed media veteran, the CBC is often led by a political appointee. Like most of his predecessors, Mr. Lacroix has never had a top job in a media company. At the same time, he is accountable only to the Prime Minister.

So…the CBC is run by a guy who doesn’t have the requisite skills to do the job. And is a political appointee.

Relatively speaking, not bailing out the CBC is actually a sensible decision in comparison.

Shelfari Scanner First Beta Released

Shelfari ScannerAnyone who has used one of the many online book review/inventory sites (such as Shelfari) knows how painful it can be to enter their inventory of books into these sites. Not only do you have to enter the data manually, but you’re likely to have to search through a page of results to try and find the right item/edition/version of your book. Which brings us to the purpose of this post…(subtle eh?)

Shelfari Scanner is an Adobe AIR-based application that turns any high-quality webcam into a barcode scanner you can use to quickly generate an inventory of your books suitable for use with Shelfari. The application uses my Scannerfly SDK to provide the barcode scanning capability, which provides barcode scanning that has been tested with the MacBook Pro’s internal iSight camera, but should also work with decent external webcams, or webcams with manual focus capabilities.

To install the application:

  1. Download and install Adobe AIR 1.5 or later (supported on Windows, Mac OS X, Linux)
  2. Download and run the Shelfari Scanner installer.

Once the application is installed, run the application, and grab a stack of books. To scan a book:

  • Make sure you have adequate lighting
  • Position the book so the three red lines in the Shelfari Scanner video window intersect with the book’s barcode
  • Each red scan line will show you where the application believes a barcode might exist with a blue line
  • When a barcode is detected, you’ll hear a beep, and the application will confirm the scanned barcode with a lookup on Amazon.com

Once you’ve scanned all your books, hit the save button to have Shelfari Scanner export an inventory file suitable for importing into Shelfari. Voila! You’re done!

This is, of course, a first pass at the application to get something out the door – if the response is positive, I’ll roll in feature requests posted in the comments as my time permits.

I, for One, Welcome Our Dystopian Yet Amazingly Convenient Future

Schneier’s “Privacy in the Age of Persistance” is a good summation of all the privacy-wrecking developments that have resulted from omnipresent CCTV cameras, limitless computing power, cheap storage, and loose data privacy regulation. This is hardly surprising to anyone with even a passing interest in privacy and security technologies.

Now, I’m not one to applaud privacy-invading technologies. I dislike the trend towards border laptop searches, and recent developments requiring defendants to disclose encryption passwords scare the hell out of me (what happens if you forgot your password?). But it’s not all bad news – having “big brother” track my every move could actually lead to some interesting benefits.

For one thing, maybe marketers could start to figure out that I’m in not now, nor likely ever to be in the future, interested in their products. Think of how much simpler life could be – no more commercials on TV for stuff you don’t want, no more magazine ads for products that you not only wouldn’t buy, but strongly believe may be one of the seven signs of the apocalypse. Less paper, less interruption, less “static”.

Maybe street lights would actually display some evidence of sentience, and change to let me pass when there’s no traffic in the other direction – instead of making me sit for five minutes, uselessly whittling away what little fossil fuel remains on the planet.

Perhaps my shopping experience would even get easier. I’ve long hoped that Safeway would finally do something useful with the bajillions of transactions I’ve performed using my rewards card. They know what I buy, they know how often I buy, and they know where I buy. Is it too much to expect to find a fully-stocked cart with my groceries ready to go when I show up to my usual Safeway on my regular bi-weekly trip?

One can only hope it gets this easy. But I’m willing to bet that it’s a long time before we get to that point. Most software companies I’ve worked for have had little capability to dissect their customer base. That’s right – the companies who make technology for a living are usually clueless about who buys their products. It’s not that they don’t have the data – it’s just that the data is complete crap. Sales guys don’t enter all the details, and the systems are incapable of drawing the most basic useful conclusions. Hence, a lot of effort goes into gathering data that is noisy, incomplete, or just plain wrong.

Of course, that last point is what really scares Schneier. What’s worse than all-seeing, all-knowing surveillance, than technology that people believe to be all-seeing, and all-knowing if you can’t trust its output? And until we have that capability, I’ll just have to be content to shop at the Safeway the old-fashioned way.

A Tale of Two Emergencies

An interesting tidbit I picked up at a review session at a UBC’s UILO (University Industrial Liaison Office) a couple weeks ago: apparently the FAA prevented use of unmanned drones during the post-Hurricane Katrina to perform early reconnaissance of the damage. But the story doesn’t stop there…

Apparently the FAA’s been a bit sticky about use of unmanned drones in civilian airspace, citing safety concerns. When military commanders wanted to use drones to survey the damage, the FAA stopped them on the basis that the drones weren’t certified for civilian airspace. The military responded by proposing to bolt the unmanned drone’s sensor package to a helicopter. Again, the FAA denied the request, noting that bolting the package to the helicopter resulted in a new configuration that would require re-certification of the helicopter.

Sigh.

In the end, the military overcame the objections of the FAA by duct taping the sensor package to the helicopter. Apparently, this configuration would not require re-certification, as the alteration did not result in a permanent change to the aircraft.

Keep in mind that, at this point, New Orleans was a disaster zone – even if the unmanned drone fell out of the sky, it’s doubtful that it would have any worse effect on the population than what had already occurred. A perfect example of bureaucracy run amok and working against those it was designed to help and protect.

Contrast that incident with this video of the response executed by New York’s ferry operators when US 1549 dropped into the Hudson. In less than five minutes, there were not one but three New York Water Taxi ferries picking up passengers from the disabled plane’s wings. One can’t help but marvel at their responsive to the emergency (see the 5:45pm entry):

“Someone came into my office and said a plane crashed,” said Tom Fox, general manager of New York Water Taxi, “and we ran out the door.”

Fox rode out to within several hundred yards of the plane on one of three Waterways boats that responded, but authorities indicated that additional help was not needed, apparently because most of the people had already been rescued.

It was probably not the smartest thing to do. That response probably violated a lot of the company’s rules. And yet, it was the right response.

Now, it’s probably not fair to compare the two incidents. Katrina was a slow evolving disaster with many facets hidden from plain view, whereas the US Airways crash was a clear and easily understood event. The water taxi manager’s response to the aircraft crash was likely further provoked by more recent experience with disasterous events (I’m thinking 9/11 specifically).

Yet, as I look at the multitude of global challenges we currently face, I hope that we will respond more like the guys driving the water taxis and less like the FAA.

Newspapers’ Decline Will Impact BC Severely

deloitteI attended Duncan Stewart’s roadshow presentation of Deloitte’s Technology, Media, and Telecommunications Predictions 2009. While there wasn’t much that was especially surprising for anyone that closely follows these sectors, the decline of the newspaper industry is worth noting:

The need for new business approaches has become increasingly apparent. While publishers have reacted, this has not always been at a sufficient pace. Given this context, up to one out of every ten print publications could be obliged either to reduce print frequency, cease physical printing or, in some cases, shut down entirely in 2009.

Duncan made an interesting point by connecting the decline of newspapers to BC’s pulp and paper industry. In short: failing newspapers is bad for BC. When you look BC’s reliance on the forestry industry, it’s pretty clear why:

  • The forest industry accounts for at least 15 percent of the province’s economy.
  • Direct 2006 forest industry activity totaled $10 billion, representing 29 percent of good producing industry GDP and 7.4 percent of total provincial GDP.
  • Forestry activity contributes approximately $17 billion to the province’s gross domestic product (GDP).
  • In 2006, forest products made up 41 percent of all B.C. exports, with a value of roughly $13.6 billion a year.
  • 90 percent of BC lumber exports and 71 percent of pulp and paper products are exported to the United States.

John Diack, former CEO of Circon Systems, put the magnitude of the problem in perspective for me – the Los Angeles Times newspaper uses approximately one-half of the pulp and paper output of the entire town of Port Alberni. Ouch.