Directions in Client-side Web Development
This document considers trends emerging towards the increasing focus on the client-side in development, constrasts the merits of various approaches, and speculates on future directions and requirements. At the close there will be a consideration of how these goals particularly from the point of view of RSF development. Since I am the lead developer of RSF, please note that this discussion may be somewhat situated.
Forces towards client-side development
An increased focus on client-side development is being impelled by a number of forces. Firstly, the capabilities of browsers are becoming more stable and understood, with growing expertise becoming established in a number of increasingly mature client-side frameworks (Dojo, JQuery, ExtjS, YUI to name but a few). Secondly, the web is becoming an increasingly heterogeneous mixture of server-side frameworks, and client-side primitives are becoming correspondingly seen as an inviting "lingua franca" for interchange, whether at the level of markup (HTML or XML) or data (JSON or XML). Thirdly, client-side development simplifies some aspects of architecture. Server-side webapp design is complicated by the approach to state - how to store it when it is required as part of a UI design, and how to avoid it where it can be argued unnecessary. Fourthly, client-side development promises to allow access back to development to "casual" developers and end-users - client-side environments offer short cycle times and "immediate gratification" in seeing changes reflected, and are often accessible to styles armed with a simple text editor.
Forces retarding client-side development
However, the negative factors which have retarded client-side development are still very much in effect. Historically the browser environment has been the focus of bitter commercial disputes amongst vendors, leading to enormous fragmentation and technology instability. Largely through this cause, many promising client-side environments have effectively stalled or even imploded though effective "vetos" placed by significant browser implementors (historically Microsoft and Mozilla as core players, but today a webapp which is considered "portable" also requires to give consideration to Safari, Opera and Konqueror). Once Java applets were widely seen as the most promising vehicle on the clientside, but they have languished through the increasing heavyweight nature of the environment, requirements for lengthy downloads and installations, sometimes with administrator intervention. Flash is a considerably more lightweight and widely deployed vehicle, but even these demands on user's time and environment are considered excessive for many applications. For a truly idiomatic experience, users are increasingly demanding applications which are ready to execute virtually instantaneously, and are completely integrated with the browser's document model (plugin-based technologies still fail to be proper DOM citizens, with regards to issues such as resizing, layout and styling).
The role and nature of Javascript
These considerations have increasingly forced attention onto Javascript as the delivery engine of choice for the core use cases for client-side development. Javascript as a language has many powerful (though often underappreciated) features, However, Javascript as with all client technologies has suffered disproportionately in the wars for influence. Although subject to various standards, browser profiles are eccentric, even across closely related versions, especially as regards the crucially useful functions of document manipulation. This situation is being improved considerably through increasingly scientific investigation and the development of more stable abstractions from the client-side frameworks, but Javascript development tools and guarantees lag those of server-side development enormously, perhaps by upwards of 5 years. Debuggers are increasingly capable, but the quality and capabilities of development environments are still poor, hampered also by the extremely dynamic and non-standard behaviour of the language.
Client-side and server-side development costs
Javascript implementations easily surpass server-side development in terms of time to initial prototype and accessibility to casual development, but many of these costs will be paid back once the application enters a longer lifecycle. Client-side code requires perpetual testing for portability across a constantly changing profile of browsers, and whilst use of unit testing frameworks are on the increase, enjoys few schemes for testing that the behaviour of an application will be as specified. QA is expensive, and development can lead to unexpectedly obscure and lengthy headaches as further and further portability wrinkles are discovered. Also, none of the established tool chains and idioms for managing and packaging multi-module applications are suitable on the client-side - as a Javascript app increases in complexity and capability, it will naturally decohere in its architecture.
Paths and responsibilities
Dividing up responsibilities for application function between client and server is a crucial design decision, which is influenced in a complex way by a number of considerations, including application complexity, available development resources, desired user idiom and application profile. The goals of rapid development and cheap maintenance will always be at odds, and it is a constant and ongoing technological battle to improve the range of available tradeoff points. An all-too-common phenomenon is that of a successful "grass-roots" app, development using a "rapid development" technology, rapidly outgrowing the complexity and capabilities that can be safely managed in its idiom. If the cost of performing rewrites is too high, the community will be saddled with a costly albatross of development. Often for this reasons implementations can be simply abandoned rather than maintained, as the costs become prohibitive, and the community simply shifts focus to a new product or strategy.
To reduce these ongoing costs, and to give more options during development, it is valuable to explore technologies which allow for more graceful transitions during the development lifecycle, and also allow for changing design goals to be reflected in development with greater flexibility and fewer rewrites. It should always be relatively clear and easy - in fact, as "mechanical" a process as possible, to see how to reimplement a particular piece of functionality on a client or a server.
Protocols and rendering
A crucial design choice in a client-server design centres around the semantics and payload of client-server communications. For webapps, the "REST" idiom is increasingly seen as making the greatest efficient use of deployed HTTP infrastructure, but this still leaves the decision as to the delivered payload, which may form either rendered markup or semantic data.
Delivering semantic data (JSON/XML) promises to be able to deliver applications with the greatest dynamism and flexibility, but at the same time is certain to make greater demands on the complexity of client-side development, with the disadvantages discussed above. For a known design with free choice of development resources, delivering markup (simply as complete pages, or more dynamically using the "AHAH" strategy) is likely to be the most economical. For more flexible applications, whose function is still being explored, or liable to be extended in ways not specifically planned by a development team, the semantic approach is likely to offer benefits. Despite the "primary server payload" being delivered in data, however, it is important to isolate the markup within an overall design in a form which is easily amenable to design. Immature dynamic apps tend to focus on a "string-bashing" approach where markup, or even statements in a language, are assembled by raw concatenation of strings written in another language. These designs are extremely opaque, hard to maintain, and not amenable to work by a team composed of specialised or distributed workers.
Keeping markup isolated in a semantic data app implies the use of some form of client-side templating engine, operating on a set of payloads which are separated in function - as well as the semantic, variable data payload (JSON/XML), it must also operate on a set of more static, markup oriented "template" files. Ideally these templates are as close to the desired final markup in syntax as possible, to aid work by design specialists.
Mobility of function
However, it is likely that as a result of changing design goals, or simply changing maturity or complexity of the target application, that one or other of these two approaches will seem desirable at different times in the lifecycle. To this end, it is desirable that application functions should be as mobile as possible, in terms of their implementation on clients and servers, or indeed even on servers of different kinds.
These two functions fall into two main categories. Firstly we require mobility of rendering - it would be desirable that the same templates may be used on the client, as well as the server, with the selfsame rendering semantics. This requires a common templating format, and specified engine which is amenable to being implemented in a number of environments.
Secondly, we require mobility of UI servicing - that is, the code which is responsible for applying the effect of a UI update to a data model should also be as mobile as possible. This is to allow for a required "granularity change" in application function to be an easy code maintenance operation - to switch from a "largely static" application which operates by plain HTML forms, to a "chatty AJAX" application, which transmits fine-grained updates to a server-side model, to a "bulk AJAX" application, which pens up multiple user changes, perhaps over multiple views or even when working offline, for a single large-scale server update, by design, should be a change in configuration and not require development.
Motivating RSF Goals
Future RSF development will be continuing to work towards these goals. The RSF templating syntax is as close to (X)HTML as possible, differing by only a single attribute. This makes it easy to work on by designers with standard tools, and keeps rendering logic separate from design. Work has already begun in porting this algorithm to the client-side, with a working prototype completed. This will enable clients and servers to agree on a common set of templates, easing migration of rendering function from clients to servers, as well as enabling common deployment of a consistent widget set through multiple server-side environments, including not only non-RSF Java applications but also non-Java environments.
The following phase of development will concentrate on porting a further core framework function to the client-side, that of performing encoding and servicing of the data binding model. RSF's binding model is transparent and language neutral, and capable of being serviced in any form of environment. Allowing binding service on the client means that changing the data update model (changing granularity, or location of service) is just a configuration change, and does not require specific developer resource to change either markup templates or component tree specification.
This will provide a toolkit of functions that makes it easy for developers to construct their own dynamic and flexible "widgets" or applications, without being constrained by possibly inappropriately baked collections of requirements. As a simple example, a "data table" widget is a core component of many client as well as server-side frameworks - however, these are all constrained by the use cases of the original designers and are often very troublesome to customise or maintain in ways not originally anticipated by the designers. Further, in an attempt to cater for all possible use cases, these widgets become overburdened with configuration options, which often amount to writing snippets in a new custom language, rather than being able to concentrate on writing conventional code. A classic use case for both server and client-side widgets is "heterogeneous tables" - a table for which the markup is not a regular repeating structure, but requires for example some special rows, or even simply a radio button selection model for rows.
With a universal and transparent markup and binding model, developers would be freed up to simply specify application function, in the language and environment of their choice (Java, Javascript, PHP or further), in the confidence that changing goals or growing complexity would not pose a serious maintenance risk.
Housing within Sakai
The URL environments available within Sakai are documented at URLs within Sakai. The same choices which are available for AHAH dispatch (discussed within Helpers and Gadgets within Sakai are also available for JSON views. As of RSF 0.7.3M1, pure data views (JSON encoded or other) are available on an equal footing within RSF applications, and thus may form part of a coordinated suite of views hosted from an application fragment or "entity world".