The Next-Gen Web: Browser Storage Support

The next-gen web is starting to gather pace, as this week MySpace integrated Google Gears, Yahoo! announced their new BrowserPlus product and Google launched a browser-based edition of their 3D Earth product. Technologies and formats such as AIR, Silverlight, JavaFX, Gears, XUL, Web Applications 1.0 (DOM5, HTML5 etc.) allow developers to accelerate beyond AJAX and towards a new generation of web applications with better performance, more functionality and tighter desktop integration.

Developers and users are now presented with more web technology choice then ever before; “DLL hell” has been superseded by “plug-in hell”, as a variety of companies present their versions of what the next-gen web will look like. But on the web, such choice can come at a cost to both users and developers. More than a decade has passed since the first battle over web formats, back then it was Microsoft, Netscape, Apple, AOL and others laying different foundations in the form of browsers, scripting languages, web servers and more. The legacy of that battle is still being felt today, as Javascript developers rely on whole libraries to assist them in developing cross-browser code and CSS developers depend on a catalog of hacks so that their sites can look consistent across different browsers.

With the new rich web application technologies still in the development phase, there is an opportunity to not repeat the mistakes of the past and instead take a standards-based approach. Thankfully during the course of the previous decade companies such as Microsoft became more receptive to open standards, data portability and cross-platform support. Having broad support for open standards simplifies technology for both users and developers, but it is obvious that not all of the currently announced technologies, such as those listed above, will

Browser-based Local Storage

As web applications became more popular there was a general demand for an ability to run web-based applications offline. The first such solutions that could work without requiring a browser plugin or separate application were those that relied on the caching headers within HTTP to store objects within the browsers cache. Javascript libraries such as Dojo implemented support for offline web applications using the same principals, but applications were very limited in scope as there was no easy way to store structured data on the browser (Dojo now also abstracts a variety of other storage engines including Gears - tip: Dylan) .

In May of 2007, Google launched Google Gears, a browser plugin that allows web applications to synchronize data into a local data store and then allow web applications to function offline. At the launch of Gears, Google Reader was adapted to support it, and the emphasis of the pitch for Gears was about offline application access. What was less known is that Gears is a lot more than just offline access, as it provides three primary functions:

• Caching of resources (HTML pages, images etc.)
• Structured data storage in a database
• Asynchronous background worker threads

The part of this we will focus on here is the local object and structured data storage. Gears provides these functions via a Javascript API, which can be accessed by any web application. The structured storage is provided by Sqlite, a popular lightweight RDBMS. With the local database, the developer can not only perform queries and inserts to record new data, but also more complex SQL like joining between tables etc. Although you can have multiple applications using Gears, each app runs in a sanboxed environment with a domain-based security model (similar to cookies and Ajax requests). Sqlite has been built into Firefox since version 2.0, but its API is only accessible from an add-on or a core Firefox component. The Gears plugin bridges that gap and makes it available within the client scripting environment.

Before Gears was launched, the Web Hypertext Application Technology Workgroup (WHATWG) had begun work on its Web Applications 1.0 draft spec, which included structured data storage as part of HTML5. The current draft spec from the working group includes definitions for a Database object for accessing and querying a local data store. The details of the implementation are left up to the vendor, but the API is detailed in the spec. Firefox will be implementing parts of the same storage API from the WHATW spec in version 3.0 of the browser, which is currently available as a preview release. The key components of the WHATWG spec are:

• ApplicationCache - for storing objects in the local browser cache (and checking them)
• navigator.onLine - check if the browser is online or not (and use cache plus local data store if required)
• Storage interface and events - used for storing name and value pairs via the sessionStorage DOM attribute.
• Database interface - used for connecting to the local database. Supports SQL (or a subset thereof, depending on the server used), versioning, error events via callback
• Threading and Callbacks - so that multiple requests can be sent to the local data store asynchronously.

Implementing local storage, caching and offline access are relatively easy. The application can first check to see if these functions are supported, and then setup the local cache by synchronizing the users data in background processes. While a thread is running, either uploading or downloading, you can query it to check on its status to provide the user with feedback (eg. a progress bar). Once the data is local, by running database queries on the local machine developers are able to drastically improve performance. Currently many web applications use the browser as only a presentation layer, for eg. a spreadsheet application may do a round-trip back to the server to work out even elementary calculations such as =1+1. By utilizing the local data store and client-side code, the developer is able to offload processing and storage to the client and provide a much smoother, desktop-like experience at the same time.

Current And Future Support

The issue is that the majority of the WHATW specification was written after Gears was released, so the Database and LocalServer objects used in Gears are not compatible with WHATW - for now. The good news is that Google have come out and fully backed the storage portions of the WHATWG HTML5 spec, so developers with apps running on Firefox 3 with Gears installed will have a choice to use either the native implementation, or the Google implementation. Google go on to say that they will likely offer extra features as an incentive for developers to continue to target Gears over-and-above the HTML5 implementation (features such as desktop shortcuts, etc.).

Other alternatives for local data storage, such as Flash local storage, are completely incompatible with the WHATW specification. The developers at WebKit were very quick to announce that they have started implementing the storage portions of the HTML5 spec also. It is currently available in nightly builds, so in the near future we will see support in both Konquror and Safari. Opera have also announced similar plans and they are actually leading everybody when it comes to implementing HTML5 and Web Forms. Yahoo! BrowserPlus was only announced yesterday, and it is currently unclear wether their local storage support is compatible with the specification as laid out by the working group.

Local storage is a major new feature of the new web API, and developers will not only have consistent support across browsers but will also have the option of Google Gears (which is already available) as well as Yahoo! BrowserPlus (depending on how it works). There is just one browser maker missing in this discussion so far and that is Microsoft. Microsoft have released an early preview of IE8 and announced a raft of new features, a lot of which are based on open standards such as better CSS and Javascript support (with a more standardized object model). The big question is wether we will see consistent local storage support from IE8 following the same spec as the other browser vendors. The IE team have announced that IE8 will support DOM Storage, but that is only part of the overall local storage spec (ie the Storage object described above only).

Overview of Current And Future Support

	Gears	BrowserPlus	Firefox	IE8	Webkit
ApplicationCache	soon
detect onLine
LocalServer
Storage
Database
Threading
SQL