« March 2010 | Main | January 2010 »

Thursday, February 04, 2010

Flash 10 versus JS/HTML5: The Tour Guide

I have seen an enormous amount of uninformed speculation regarding what the Flash and JS/HTML5 platforms are each capable of as of right now(February 2010). This post is an attempt to comprehensively round up the differences by going through the Flash API and filtering through it to create a "FAQ" of features worthy of direct comparison, with the focus being "what does Flash offer that is not covered in existing or proposed spec?" I used a variety of sources to research this info and linked to them as appropriate; I offer this with the disclaimer that my expertise mostly lies on the Flash side of things, and even there, I have probably made mistakes and omissions in attempting this ambitious piece of documentation. Please help me improve this overview by telling me what's wrong, what's inaccurate, or if there's a side of the story I'm missing.

Features

Name of Flash API class, namespace or feature Notes Can JS/HTML5 Do It?
Accessibility, AccessibilityProperties Display objects on the Flash stage can provide descriptions, keyboard shortcuts, and different behavior for screen readers. This is a wide-ranging issue and deserves to be addressed with its own post. HTML5 introduces several new accessability concerns, most notably Canvas and its ability to replace traditional layout entirely. Not all issues are resolved. The best answer I've found on the subject is in this Stack Overflow thread.
BitmapData An API for low-level manipulation of bitmaps. Yes, Canvas contains similar bitmap-manipulation methods, albeit performance may differ.
Graphics A set of methods you can use to create a vector shape. Yes, with SVG or Canvas.
Loader A class used to load SWF and image files. Yes, with URI resources and DOM manipulation.
flash.filters A set of classes for bitmap filter effects(glow, blur, convolution, etc.) Emulatable, possibly with lower performance. SVG contains a filter effects spec which could potentially be applied to all document elements. WebGL also allows programmable shaders.
Camera, Microphone Captures video/audio from a local webcam or microphone. Proposed post-HTML5: device tag
Sound Loads and plays back MP3 and WAV audio from files or streams. Besides built-in pan and volume controls, real-time inspection and manipulation via Actionscript are allowed. audio tag plays back sound and streams; however, as currently specified it is a black box and does not allow inspection/maniplation. An extension to <audio> in Firefox for these features has been prototyped.
Video Displays live or recorded video via embedding, streaming, or progressive download. Sorenson Spark, ON2 VP6, H.264 codecs are supported for video, and ADPCM, HE-AAC, and MP3 are supported for audio. Source
video tag plays back video, and allows for a scriptable user interface using HTML/JS equivalent to any Flash interface. However, codec support differs across existing implementations due to patent considerations; however, interest in an open video standard is extremely high now, so I expect this problem to resolve itself soon.
Continuous streaming(as opposed to progressive download of a file) is not discussed in detail in the current spec.
The FileReference class provides a means to upload and download files between a user's computer and a server. An operating-system dialog box prompts the user to select a file to upload or a location for download. FileReference may also be used to read and write local files. the "file" input type supports custom upload behavior, and the DOM now supports drag and drop elements. File download triggers from JS already exist.
The SWF format, LocalConnection, Security, ApplicationDomain SWF files are independent namespaces that can run simultaneously, sandboxed from each other, and may optionally open themselves up to intercommunicate. Each SWF's namespace may contain fields representing various resources. SWF interconnectivity is primarily used in applications to provide a lightweight front-end or preloader for other content.
While the overall nature of the SWF format is a foreign concept, iframes have a similar namespace access ability.
DisplayObject and Stage Flash builds its runtime display around the "Stage," which is a tree-structure of DisplayObjects and DisplayObjectContainers; all the kinds of things Flash can display(vectors, bitmaps, video...) are inherited from DisplayObject. Elements on the Stage can attach input event listeners and use the visual display to inform functionality(e.g. overlayed elements will take a mouse click topmost-first). DisplayObjects are able to access all of the available effects and filters, making it easy to apply effects to both single elements and deeply nested groups. The DOM bears broad similarity to the Stage, both being tree structures and using event listeners attached to nodes, but being focused on a document rather than a display makes the actual implementation far different. Nothing prevents the overall functionality from being emulated, however.
Socket, XMLSocket
The Socket class enables code to make socket connections and to read and write raw binary data. Socket is a "raw" TCP/IP connection with user-definable endianness. XMLSocket is the precursor to the general Socket and allows ASCII data to be sent, delimited by \00 markers.
Since Flash 7, server-side policy files have been used to control cross-domain interactions; one policy file is used for loading data, and a second is used for sockets. The overall system is called the meta-policy system.
WebSocket is most similar to XMLSocket in nature; i.e. it is text-oriented, not a low level TCP socket.
Cross-domain access control is being added to HTTP.
ByteArray The ByteArray class provides methods and properties to optimize reading, writing, and working with binary data: endianness control, string conversions, and assistance in reading and writing a variety of formats(8, 16, 32-bit ints, floats, UTF, etc.) The present spec does not include an explicit byte array; strings and numeric arrays can be used to get equivalent results, but with some difficulty, as shown for example here. The WebGL spec has proposed a typed array format that would encompass ByteArray's core functionality.
PrintJob The PrintJob class lets you create content and print it to one or more pages. Browsers have "print page" built in; Canvas elements can save to an image with toDataUrl(). Similar functionality to the Flash API should be possible with window.print().
Shader, ShaderFilter, ShaderJob A Shader instance represents a Pixel Bender shader kernel in ActionScript. Pixel Bender implements image processing algorithms(filters or effects) in a hardware-independent manner. Pixel Bender uses a parallelized, XML-based graph language. Documentation Filters and shaders can be emulated in JS; SVG contains filtering options. WebGL allows programmable shaders. WebWorkers provide a more general-purpose paralleization method.
setClipboard Copies string data to the operation system clipboard. "getClipboard" is not allowed for security reasons. Firefox and WebKit presently provide internal APIs for clipboard use. The draggable attribute includes some language dealing with drag+drop events and the clipboard. Access to the clipboard is a tricky problem because of security risks surrounding its contents and the potential of data theft from an unaware user. HTML5 also has a proposed Undo API.
TextField and Text Layout Framework
TextField provides an API to print and input text with options for formatting, selection, and HTML text.
Text Layout Framework is a "next-gen" text implementation in Flash, containing new features for layout, UI controls, and internationalization.
Two issues here: Input and output. Output will come first.
In the past web developers have resorted to using Flash's embedded font support to achieve a unique typeface. Going forwards, CSS @font-face will allow embedded fonts, covering this deficiency. Common layouts are doable within existing CSS; Canvas and SVG are available as last-resort options if your text needs demand elaborate transformations or filters.
TextField's input support is known to be buggy, especially for non-English input; browser forms are more reliable input editors, but less visually flexible. A "fancy-text" input is probably emulatable from within SVG or Canvas.
The new features of Text Layout Framework are impressive. It should fix some of the existing issues with TextField, and the implementation is open-source, which bodes well for future inclusion in a standard.
RTMP, RTMFP, and Protected Streaming
The Real Time Messaging Protocol is a proprietary protocol for streaming audio, video and data. It has three variations: plain TCP using port 1935, RTMPT, which encapsulates itself in an HTTP request to traverse restrictive firewalls, and RTMPS, which uses HTTPS. RTMP allows streaming of multiple data channels, and includes remote procedure call methods. Adobe has multiple server implementations available for the RTMP series, the latest of which is Stratus.
The Real Time Messaging Flow Protocol is a UDP-based version of RTMP with additional peer-to-peer functionality, allowing users to connect directly with microphone and webcam streaming.
Protected Streaming is Adobe's term for the DRM solution provided by the RTMP series; Flash Media Server is able to encrypt media on-the-fly to prevent a variety of data theft attacks.
There is no exact equivalent to any of this. Some of it can be emulated in some fashion, but not necessarily with the same performance characteristics owing to differences in protocol capabilities between a low-level TCP/UDP implementation and what can be done with HTTP and Web Sockets. The main power of RTMP is that it is designed specifically for the Flash platform and works directly on its API(see NetStream). The peer-to-peer features of RTMFP are wholly unique.
HTML5/JS is unlikely to ever gain a spec for DRM, both because of the political implications of the technology, and because past DRM solutions, including Adobe's, have a track record of being quickly broken. The "analog loophole" will remain a viable option for the foreseeable future, if nothing else.

Implementation Strength

There's a second set of considerations to address beyond the API, of course, and that's in the overall performance/reliability/security considerations that color the implementations of each platform. I'll admit to some guesswork, some bias, and handwavy estimation going on here, especially where I try to extrapolate.
  • Flash has taken the initative to speed itself up, starting with Flash 9 and the move to Actionscript 3. This language upgrade involved a new VM that could optimize against static type annotations and manipulate ByteArrays(raw byte data). Flash 10 has added Vector(typed arrays) and a "fast ByteArray", used by the Alchemy C->AVM2 compiler and haXe's Memory API. These speedups have enabled some dramatic new uses of Flash for the coders sophisticated enough to make use of them. Flash can also parallelize through Pixel Bender and the Shader class. This approach does have some demonstrated utility for general computation.
JS/HTML5 equivalents: JS is not as fast....yet. Web Workers look like an elegant near-term parallelization solution. JS5 has no real equivalent to AS3's annotated types or ByteArray, but the "ECMAScript Harmony" project aims to continue pursuing those language features and others that were left over from the ECMA4 process. Google's Native Client effort should also deserve a mention, since it also attacks low-level performance, but in a far more comprehensive way than just memory manipulation. And, of course, general-case JS performance continues to improve dramatically.
  • Flash also has a reasonably fast software renderer....at least, if you are running on Windows. (It is abysmal to useless everywhere else, in my experience.) The 3D support introduced in Flash 10 is not a real solution for 3D acceleration, but the 2D, at least, seems to be more mature than the existing implementations of Canvas, and it is much more comprehensive for immediate use-cases, especially in terms of effects and filters.
JS/HTML5 equivalents: Canvas has been proof-of-concepted multiple times over as an acceptable 2D graphics solution for typical existing browser apps, including games. (The JS emulations of the Game Boy and NES are proof of soft real-time bitmap manipulation ability, Another World JS shows off flat-poly rendering, and this particle demo makes hefty use of alpha-blended gradients.) Now that efforts are starting to consolidate on WebGL, which is a very promising standard, it is possible that JS will leapfrog past Flash for high-performance browser graphics. Adobe probably faces a major engineering challenge in reconciling 3D against their existing graphics use-cases, as this has been an oft-requested feature for years running. Regardless, the strong set of built-in graphics features, optimized filters and effects, and tool support remain compelling reasons to use Flash.
  • Flash being Windows-centric with respect to graphics brings up another point, which is that Flash has historically overpromised on what it can do across platforms; major differences in both performance and behavior across platforms have brought a lot of ire down upon the plugin over the years, and it is not especially known for spec-correctness or stability either(although, again, on Windows, at least, it is stable most of the time and backwards compatible with most content). The Flash security model is also rather byzantine, between the multiple-SWF-namespaces, the meta-policy system, and some odd restrictions on how different media types may be loaded or streamed(for example: Video can be loaded remotely, or embedded in the SWF, but cannot be loaded from bytes, while Sound can be loaded from all sources.), and several major exploits have been found in the plugin regardless.
JS/HTML5 equivalents: Browsers, as a group, have had a better track record of correct behavior, backwards compatibility, security, and stability. The addition of cross-domain access-control headers to HTTP through this draft spec should provide similar functionality to the Flash cross-domain security model, but with a less "hackish" approach to implementation.

Conclusion

Because Flash has been able to quickly implement some "low-hanging fruit" performance features, it will stay ahead of the game for the moment. But the indications are that Flash's codebase has grown quite creaky and inflexible, and unless Adobe can summon the resources to overhaul the code, support all the platforms properly, and continue pushing out new features, the browsers are likely to resolve their current issues and pull ahead for most use-cases in just another year or two; games and other media-intensive apps will lag behind, out of sheer ecosystem inertia, but the provisions to support them are mostly visible on the horizon. And it is worth remembering that there's a fundamental conceptual difference between the browser "document/resource" orientation and the Flash "display/SWF namespace" orientation; the Flash model simplifies distribution of "apps as content" like games because it can bundle the content into one file, and I don't see a spec out there that provides that equivalent ability to the browser. This isn't a deal-breaker, but it makes it harder to support a portal-style business model with interchangable programmatic content.

Personally, I am hedging my bet by using haXe. I am happily using Flash for current projects, but the bulk of my code will remain portable. As well, haXe is an excellent "language upgrade" to both JS and Actionscript.
Posted by James Hofmann at 12:55 PM Comments
Edited on: Friday, May 21, 2010 2:17 PM
Categories: Articles