Barcode Recognition in Flash
As part of my new year, I’ve decided to start putting more code out there earlier. I have a tendency to build projects in private, trying to get things perfect before I reveal them publicly. The end result is that I have a lot of code gathering dust in the corners of my hard drive. No more. That’s why I’m going to post one of the mini-projects I’ve been working on for the last little while: a Flash-based barcode recognition library.
I know what you’re thinking: Why the heck would anyone want to build a Flash-based barcode recognition library? Why not try the demo yourself and see! 😉
The idea is simple: allow use of a commodity webcam as a barcode reader, thus allowing users to efficiently input physical items into web-based applications. Using digital cameras as a cheap alternative to traditional barcode scanners isn’t an original idea – Delicious Library uses iSight to allow its users to input their CDs, DVDs, and other media into a personal media library. Unfortunately, that particular application is a heavy application that’s only available on the Mac – what I want to do is build a simple way to provide that same kind of functionality in platform independent web-based applications.
Think of the web applications that could benefit from quick and efficient input from physical items that carry a barcode – cataloging items you own or consume to drive recommendations on sites like AllConsuming or Shelfari (or any number of other similar sites), tracking your personal property for insurance purposes, or easily selling or trading large numbers of goods on eBay, Lala, or Swaptree. And of course, I’m sure there are some other applications that I haven’t even considered.
I built the original library in Java as a proof of concept, more to figure out how best to reliably extract a barcode from an image. I knew it had to be fairly straightforward as there are a number of SDKs available on the web to perform image-based barcode recognition, but I was uncertain if a commodity webcam was up to the job. I used the cheapest webcam I could find ($20 at Best-Buy) to make sure I had something that would work with even the worst hardware.
Once I had the basic algorithm for scanning an image and extracting an EAN-13 barcode, I ported the code to Flash 9 in an evening (ActionScript approximates Java). Flash was an obvious choice for a couple reasons:
- Cross-platform browser support: Flash has strong support across all platforms, operates in all web browsers, and is readily integrated with web-based applications. While the same is also generally true of Java, Flash is more widely deployed and is lighter weight.
- Native camera access: Flash provides a native API for accessing video cameras, as well as native support for manipulating bitmap and video data.
- Mobile support: While not strictly required, the ability to deploy this technology on mobile devices in the future would be a nice bonus. Again, not an especially original idea – Japanese cell phones have native QR barcode recognition capabilities – but a nice bonus nonetheless.
The core library is in a functioning state, with a couple of caveats:
- EAN-13 only: The current code only decodes EAN-13 barcodes, a superset of UPC typically used to represent barcodes on books, CDs, DVDs, and other boxed goods sold by retailers. While the framework is in place to support other symbologies, I haven’t yet implemented any code to decode other symbologies as EAN-13 covers the majority of barcodes that I believe to be of interest for a consumer-oriented web applications. I also have a funny feeling that there are still some nuances of EAN-13 that I haven’t implemented correctly, so those issues will need to be addressed.
- No fixed focus cameras currently work: Decoding a barcode with the current code requires a fairly high quality image that clearly resolves the separation between the bars and spaces in a barcode. A webcam shooting in 640×480 resolution is sufficient, provided the barcode is put close to the camera. The challenge here is that in order to get the clarity required, you need a webcam whose focal length is adjustable. Currently, webcams with a fixed focal length, such as the Logitech QuickCam Communicate STX or those built into the latest MacBook Pro and Sony VAIO laptops, can’t be adjusted to render a crisp image when the object is placed close to the camera. This is somewhat ironic given that webcams with manual focus, such as the Logitech QuickCam Chat, are generally very cheap.
The second shortcoming is the most problematic – we can only expect embedded webcams to become the norm. I’m not clear if recognizing a barcode from these types of cameras is possible – Delicious Library claims that it can work with the built-in cameras on MacBooks, but I’ve not had an opportunity to confirm it myself.
This brings me to my questions for the technologists out there:
- Does Delicious Library work with MacBooks? If so, I’d love to see some video of it in action. It would also be great if someone can point me in the direction of possible approaches to solve the fixed focus problem. This will also be useful if I’m going to be able to get this working on mobile devices in the future, as they also use fixed focus lenses (although at the rate their resolution is increasing, that problem may go away pretty soon).
- Does this work for you? I’ve put together a simple proof of concept Flash application that you can use to try the library out for yourself. Grab a book and hold its barcode close to your webcam with a steady hand; the application will pop up an alert showing the decoded barcode. Give it a try and comment here to let me know if you’re able to get it to work for you, and what webcam you’re using.
- Wanna build something with this? Come on, you know you do. I’m looking to see if people are interested in building an application to use this – are you interested? Drop me a line here.
Niall Kennedy reports that Delicious Library does indeed work with MacBooks, which is great, because it means that it’s technically possible. All I have to do is figure out how.
For those of you on a MacBook, you’ll probably just get a black screen – you will need to change your camera device to “USB camera” instead of “DV camera” which is the default in Flash. Control-Click the app, and choose “settings” to change the camera setting.
Oh wow – how about this for a “you are not a unique snowflake” moment: posting this on Digg, Digg informed me of a similar post from 285 days ago.
Apparently there’s a community-based barcode database project that has a Flash-based barcode recognition application. However, as with mine, it doesn’t appear to work with fixed focus cameras.
Hmph, well, that’s the challenge for me to address I guess.
There is a mobile platform that will read bar code symbologies as well as others with your camera phone or mobile device to go to the web with one click.
http://www.qode.com
Gavitec has a platform to read the symbologies from your cell phone or mobile device.
Oh yeah, Active shopper has a mobile application to scan or enter a product code and go to the web to find the cheapest price with your mobile device.
Nice article. Very telling of things to come.
Thanks for your talkback on my O’Reilly blog. Congratulations on getting this working!
I can’t speak to whether Delicious Library works on the MacBook’s and iMac’s built-in iSight (which are USB devices), but on my old FireWire iSight, it’s very obvious they programatically reset the focus to just a few inches away, as you can hear the servo spin inside the camera and see yourself unfocused in the preview window. I suppose this is accomplished by means of low-level camera-access calls like VDIIDCGetFeatures. Maybe there’s an equivalent QuickTime call for USB at this point, or maybe it’s done directly through USB device calls. At any rate, it seems like Flash would need to expose some sort of a camera settings API to let you programatically set focus, right?
–Chris
Chris – interesting. When you go to capture barcodes using your iSight using with Delicious Library, does this “refocus” happen automatically, or does it happen when you put a barcode/object in front of it? I thought the iSight had auto-focus capabilities.
I’m not aware of any way that Flash exposes to manipulate focus (see the Camera class). Even worse, the new breed of built-in webcams that are appearing on new laptops have their focal point fixed to the position where your face is usually located when it’s in front of the laptop (for conferencing), so even if there were a Flash-accessible API, it wouldn’t help in this case.
Swampthing – the challenge with the Qode and Gavitec approaches is that they appear to rely on new barcode formats, or barcode formats that aren’t in wide use on consumer goods. That’s fine if the application allows you to embed these new barcodes to drive your application. However, there’s already a wide array of 1-dimensional barcodes out there, ripe for developers to build upon.
Brendon- the setting of the focus is immediate when you bring up Delicious Library’s camera. As soon as you click the camera button, you can hear the iSight servo and when the window comes up, you’ll see yourself out of focus.
I noticed, BTW, that the default QuickTime video capture settings dialog for the external iSight allows for setting the focus manually, so focus setting could also be done programatically with the SGSetChannelSettings API, which is interesting because a) it’s not FireWire-dependent like the VDIIDC stuff, and b) it could be called from QuickTime for Java, which in turn could live in a signed applet. I should see if I can get to a MacBook and bring up its video capture settings dialog (or is the MacBook fixed focus too?). Of course, I guess none of this is currently accessible from Flash, is it?
Chris – nope, none of this is accessible from Flash (and I’m pretty sure it’s not available in Java either).
I found an interesting note from Wil Shipley (Delicious Monster) on the approach to handle fixed-focus lenses: “It’s a totally different approach, and it can actually de-blur a blurry barcode using some techniques from astronomy.”
And off I go to do some research!
Sorry about inaccessibility from Flash, but having done more capture in the last year than I’d care to go over, I can say that anything in the default video capture config dialog is also accessible from QuickTime for Java. Not that you’ll have a fun time munging atoms by hand (or that you won’t settle for just persisting all the settings in one UserData and restoring them in the future, like I did), but I’m confident it’s possible.
Delicious Library 2 is a wonderfully done app and this barcode recognition idea works great on iPhone using the app “Barcode”.
@Max: Agreed. Unfortunately, I think Delicious Library limits itself by not being a web-based application – hence this component will enable similar applications to live on the web.
As for the Barcode iPhone App, it’s interesting, but unfortunately I believe it will be a while before we see useful barcode apps on the iPhone. Unfortunately, the cameras on most phones are fairly low quality, which makes barcode detection difficult. Two dimensional barcodes are easier to decode under these conditions; however, there aren’t a lot of two dimensional barcodes in the real world (at least not in North America and Europe).
The real applications most people want (product price lookup on products, for example) will require one dimensional barcode support. Without that, these developers will have to bootstrap a whole new barcode ecosystem to make their applications worthwhile. Hopefully, I’ll get this to a point where it’s suitable for a phone applications in addition to a web application.
so, i just got a macbook pro, and was super excited to categorize my book collection; however, the fixed focus isight will not focus on any barcodes. total bust, kinda disappointed.
Any updates about this project ? 🙂