A web server lies at the heart of a web application server, effectively providing the "transport layer" by handling incoming HTTP[18] requests and delivering files or dispatching handlers to generate content in response. The combination of Apache and mod_perl in particular provides a powerful, flexible and efficient platform for developing web applications and a base on which numerous other higher level tools have been built. By itself, the mod_perl solution is fairly low-level requiring reasonable understanding of the specifics of the request phases, including authentication, authorisation, configuration management, and so on.

There are numerous tools, most of which appear to be written in Perl and available from CPAN[19], which offer some kind of template processing facility. The Template Toolkit[20], the author's own small contribution to the state of the art, provides a general purpose template language for generating and manipulating content. It promotes a clear separation of concerns between data (Perl data), application (Perl code) and presentation elements (templates), encouraging a structured approach to reuse for the sake of simple but flexible construction and ease of subsequent maintenance.

Apart from a few other enlightened template modules which also promote a clear separation between data and presentation (e.g. HTML::Template[21]), the majority of modules adopt the approach of embedding Perl programming fragments directly into template documents. Text::Template[22] is one such example, an elegantly simple module which incorporates the full programming power of Perl into text templates. Embperl[23] is a more comprehensive system adopting this approach and tailoring it especially to the web application environment. HTML::Mason is yet another powerful and highly regarded tool which allows the construction of complex web applications by combination of smaller components which integrate Perl code and presentation elements.

However powerful this approach may be, it is widely regarded as being limited as to the extent and ease with which it allows larger systems to be adapted, extended and generally managed. Without a clear separation of concerns between application, data and presentation, it is not easy or even possible to change one of these aspects without concern for the others. This is generally not an issue when the approach is used for small document systems, applications or tasks, but can become a problem when used in more complex scenarios. It is possible to use this approach to build large systems that are well structured but it generally requires discipline, experience and a good appreciation of the underlying design principles.

XML is a semantic markup language. Unlike HTML which for the most part adorns plain text with special tags to indicate font size, colour, page placement and numerous other presentation details, XML is a generalised markup language which adorns plain text with special tags to indicate what something is, not what it looks like. In its most simple form, XML can be thought of as a flexible content and data interchange markup language which permits a clear separation of data/content from presentation style (and thus it also falls into the previous category). Furthermore, the innately extensible nature of XML means that it can also be adapted and adopted to almost any application area imaginable (however appropriate or inappropriate that might be - it is possible, for example, to use XML as a programming language in its own right, but that doesn't necessarily mean it's a good idea).

There are dozens of Perl modules available for processing XML using numerous different techniques. The XML::Parser[24] module provides low level access to Expat[25], while XML::DOM[26], XML::XPath[27] and XML::Grove[28], to name just three, provide facilities for higher-level access to XML content. Other modules such as XML::XSLT[29], XML::Sablotron[30] and XML::XPathScript[31], provide transformation techniques for applying style sheets or otherwise processing XML documents into different formats. Moving higher still into the realm of application code, the XSP concept allows XML "tag libraries" to embed into documents well-defined calls to reusable application components. At the top of the evolutionary ladder for XML processing in Perl, AxKit provides a full-blown, XML content delivery system and application server environment built around Apache/mod_perl.

Database abstraction is typically found at three levels. At the highest level of the application, tools like Tangram[32] and SPOPS[33] act as intermediary layers, marshaling data between the application code (e.g. Perl object classes) and persistent storage media (e.g. relational database tables). In doing so, they simplify a program's interface to an underlying data source and hide the specific details of implementation, thereby reducing application complexity and increasing portability.

At a slightly lower level, modules like DBIx::Recordset[34] and Alzabo[35] provide a generalised abstraction of a database implementation. Instead of directly writing SQL queries to access a database, calls are made to the intermediary layer to perform fundamental record-oriented operations like insert, retrieve, update and delete. The abstraction hides the specific details of the SQL queries generated, thereby promoting greater conceptual simplicity and portability across the different flavours of SQL implemented by database vendors, and also to non-SQL systems.

At the lowest level, the DBI[36] module and related DBD drivers provide an abstraction of the environments specific to particular database implementations. Its methods provide a consistent interface for connecting to, querying, retrieving data from, and otherwise controlling an underlying database system. DBI is the de facto standard module at this level.

Content management as a general concept relates to the process of adding, updating, deleting and otherwise maintaining content delivered by a web or application server. It typically embodies the concepts of content ownership, version control, preview facilities, application of presentation styles or handlers specific to content type, and so on.

If the first lesson learnt from the explosion of the World Wide Web into our collective consciousness was that "content is king", then the second was surely that creating and managing content is one of, if not the most costly and time consuming element of running a web site. Thus, a truly useful and effective content management system is widely regarded as a highly desirable item.

In the realm of the web server security falls into the processes of authentication and authorisation. HTTP provides a basic authentication facility which permits a user to identify him or herself to a web server by means of a user ID and password. HTTPS is a implementation of HTTP over the Secure Sockets Layer (SSL)[37] and offers a more secure and robust approach to authentication. Authorisation is the process of validating that an authenticated user has access to a particular page or resource on a web server.

At a higher conceptual level, there may be numerous different authorisation strategies deployed. For example: hierarchical security levels, access control lists, access tokens (keys), and so on. It might also be the case that a particular user can access part of a system while playing one of a number of different roles, and thus is expecting to be presented with different views (e.g. guest user, administrator, editor, reviewer, etc).

HTTP is a stateless protocol, meaning that no information is retained from one request to the next. However, there exist numerous "bolt-on" techniques for maintaining state across requests using browser cookies, by encoding request parameters in the URL, and so on. These create the appearance of a continuous session throughout which the state of various data items can be maintained. The Apache::Session[38] module provides an abstraction of such session data, allowing arbitrary data to be saved and retrieved from a variety of different backing stores (e.g. SQL database, plain file, Berkeley DB File[39], etc).

Caching strategies can be applied at various different levels to achieve significant improvements in real and perceived system performance. At the outermost level, entire web pages can be cached if there is a deterministic way to assert that the output content remains consistent for a given set of input parameters. This implies that for a given URL and optionally, a set of CGI[40] parameters, the same page content will always be returned and that there will be no side-effects caused by processing or otherwise generating that page. A counter example would be a handler for the submission of form data which causes a record to be added to a database.

On a similar basis, smaller sub-page components or services can also be cached according to the same heuristics. At lower levels still, database abstraction layers or other persistent storage frameworks may cache raw data to avoid reading/writing to or from a relatively slow secondary data store. Similarly, an XML document may be parsed into and then cached in Document Object Model (DOM)[41] form, avoiding the need to re-perform this relatively costly parsing operation for subsequent requests that utilise it.

To subvert the words of Thomas Edison, building an application server is 1% inspiration and 99% debugging. Good error handling, logging and general debugging facilities are therefore essential.

For any given application server, there may be a number of domain specific services or facilities defined. These elements tend to implement some bespoke processing to represent "business logic" or provide access to a domain specific resource.

In the simplest case, we may want to serve up regular, static HTML pages that require no other processing. However, as with many of these examples, we may wish to apply peripheral services like adding standard headers and footers, apply a caching or security strategy, and so on.

The delivery of structured content (e.g. XML pages), transformed according to a particular style sheet and/or method. We may be converting content to different formats (e.g. HTML, WAP), different styles (e.g. graphics or text only) or by different techniques (e.g. XSL[42] or XSLT[43])

Delivering pages that have been constructed by processing source templates (e.g. Template Toolkit, HTML::Template, etc). Similar to the XML transformation process described above, but likely to be less formal and possibly interacting with various data sources other than XML (e.g. SQL data, Perl program code, etc).

A variation of the previous categories, this would typically involve a single template being used to present records from an SQL database or other data store. One template is thus used to generate a number of virtual pages. The process may be a direct application of the data into a template (e.g. using the Template Toolkit with data provided externally or accessed directly via the DBI plugin) or by first transforming it into another format to utilse an established transformation path (e.g. using DBIx::XML_RDB[44] to convert SQL result data into XML for transformation via XSL, or some other method).

With the appropriate security model in place, this would entail the provision of a facility to allow database content to be added or updated by end users, external applications or remote servers. A simple example would the provision of an HTML form for adding/updating a database record. A more advanced scenario might have an application sending data to a server via an XML document for central collation, storage and reporting.

Identified earlier as a general aspect, this can also be defined in more concrete terms relating to specific functionality. In this sense, content management represents a facility which allows an end user to upload a new page, template, or sub-page template component to the server (i.e. anything that the server might later serve out). Again, this requires an adequate security model and the definition of an appropriate ownership relation between users and content.

The provision of services through which the server itself can be managed, allowing new components and server elements to be easily installed or updated.

An extension of data management, this describes the more general case of multiple servers, applications or agents interacting directly with each other. An example of this might have an organisation running multiple servers in different physical locations which should cross-transmit articles posted to a bulletin board application (e.g. similar to the USENET news transmission model).

This describes the situation in which a number of discrete steps are required to perform a particular task, possibly over an extended period of time, almost certainly across multiple requests. An example of this might include an online purchasing facility through which goods are selected, followed by checkout (listing selected goods and total price) and then payment. Another example is the request to be added to a mailing list which typically causes a confirmation email to be sent, and a reply expected before the user is subscribed.

As well as delivering pages online and running interactive services, it should also be possible to use these various techniques to generate pages offline in batch mode. For example, if a database table is only updated once a day/night and contains a limited number of records, then it may be preferable to generate and store static HTML pages for each record, rather than construct dynamic virtual pages online.

The World Wide Web is not the only application area for the client/server model. We may want to run a local document server which formats or generates content as PostScript for sending direct to a printer. Alternately, we might want to run a server entirely within another application, thereby providing a generalised method for building complex content from command line programs.

Ideally, the server should provide the facility for communicating and interacting with other programming languages, platforms or software. If we need to run a Java application or fetch data from a database running on a remote Windows machine, then it would be highly desirable if the server could handle the specific details of this.

Data comes from all kinds of places: text files, XML files, SQL databases, CGI parameters, global configuration parameters, and so on. It may be stored in various different formats and require all manner of different tools and techniques to retrieve and massage it into an appropriate format for use. But ultimately, it's just data.

Camelot treats these various different data sources as "resources". A resource is a piece of data, or more accurately, some kind of facility which provides access to particular pieces of data. Before you can use a resource you must acquire it. When you're done with it, you should release it.

A typical example of a resource is represented by the individual records in a database table. To acquire the resource, you fetch a particular row or rows from the table. When you're finished with it you free the memory used, notify the database, call a method on a statement handle, or do whatever your particular database system requires you to do to say you're finished. Most of the time, you can trust Perl or one of your database abstraction layers to do it for you.

The Camelot::Resource module implements a base class from which any resource objects are derived. To illustrate the previous example, the Camelot::Resource::DBI::Table::Row module provides access to a row in a table via DBI.

my $resource = Camelot::Resource::DBI::Table::Row(
    dsn   => 'dbi:mysql:camelot', 
    table => 'products', 
    key   => 'id'
);

To acquire an instance of this resource, the acquire() method is called, passing in any relevant parameters to uniquely identify the resource. In this case, it is a product identifier.

my $product = $resource->acquire('foo123');

This particular resource will translate the above request into a SQL query of the form:

SELECT * FROM products WHERE id = 'foo123'

It then submits this to the underlying DBI connection and then calls fetchrow_hashref() to return the first row selected.

In practice, acquired resources may be very simple data structures (e.g. hash array of data), complex objects (e.g. security tokens, session objects), data interface objects (e.g. database handle or abstraction layer), content data (e.g. XML DOM), and so on. By providing a uniform access method to these disparate data types through the acquire/release interface of a resource, a high degree of conceptual clarity can be achieved. Furthermore, it allows potentially complex data management techniques (e.g. caching) to be automatically applied to resources in a consistent manner, regardless of the underlying data type or facility.

Actors represent self-contained units of application logic. They perform actions which operate on data, usually in the form of resources that have been previously acquired. Unlike a resource which is considered to be a "read-only" piece of data, actors may have side effects that change the state of a system.

An example of an actor would be an object responsible for adding a record to a database. It might define that single task as a number of separate actions: validating the input data, performing calculations to provide additional field data, constructing a query and submitting it to a database handle. The elements of data that it operates on, and possibly even the database handle, would be provided as previously acquired resources.

In other cases, actors might be used to implement common libraries of functionality (e.g. CGI), manage broader aspects of the system (e.g. content or system management) or encode application specific functionality. In implementation terms, an actor is simply an object and an action is a method. When Camelot invokes an action it calls the method passing a reference to the Camelot::Request object and any other configuration parameters specified (more on that shortly).

A presenter is something which generates views. In the general case, a presenter is an implementation of, or a wrapper around another module which performs template processing (e.g. Template Toolkit), a transformation process (e.g. XSL) or some other mechanism for generating output. The different views that a presenter can present are effectively implemented as templates or style sheets that control the output process, determining what gets output and how.

Presenters are derived from the Camelot::Presenter class and implement a simple interface for presenting views via the present() method. As with resources, having a common interface ensures that Camelot can treat one presenter much the same as any other. The method is passed a reference to a hash array containing currently defined data items. This will typically include the various resources that have been acquired by a service and/or additional data items that have been provided or manipulated by actors.

A service is a self-contained operational unit which combines ones or more of the above items. It provides a container for them and defines a workflow as a series of actions to be taken when the service is invoked. Typically, a service will acquire some resources, perform some actions and then generate some output by presenting one or more views. Finally, it will automatically release any resources that have previously been acquired. In the event of all or part of a service failing to run as expected, custom error handling facilities may be specified which the framework will automatically invoke.

A service can be something little or something big. It may require no input data, perform no fundamental operations, or generate no output, but presumably it does at least one of the above. Or it may do something very complex, requiring many external resources, performing multiple operations, calling on several other services, and producing output from a number of different views. Camelot allows small services to be constructed that perform useful, reusable tasks which are then built up into larger services in different combinations.

For example, one service may be responsible for generating a menu, another for creating a view of a product from an SQL database record and an XML document, a third for managing a shopping cart, and a fourth for combining the output of all these other views into a single HTML page for sending back to the client.

As with resources, the consistent interface to Camelot services provides an opportunity for output to be automatically cached. Although this only applies to services that have no side effects, the design allows a complex service to be broken down into smaller services (which themselves become more reusable), to which different caching strategies can be applied.

A server is the overall framework in which services are provided. It includes functionality for mapping requests to services and overseeing the processing of these services. It also provides peripheral features like logging and error reporting.

A client communicates with the Camelot server via an interface module. Different interfaces may be provided for particular operating environments (e.g. Apache/mod_perl, CGI script, TCP/IP socket, GUI, command line, etc). The interface handles the specific details related to accepting a client request from the environment (e.g. an HTTP request received via Apache/mod_perl, notification from a GUI control, etc) and returning an appropriate response. This is the thin abstraction layer that prevents Camelot from being limited to just a web-based application server.

The interface module through which a client connects to a server constructs a request object to encapsulate the details of the request, store resources that are acquired by services, and also manage one or more output buffers that different services might generate. In some cases, requests may be thin wrappers around other objects (e.g. Apache::Request objects within the Apache/mod_perl environment). The request maintains a reference to the interface that created it so that output and/or status messages can be dispatched back to the client by the appropriate channel and in the correct format.

This example defines a resource to retrieve records from an SQL database table via DBI. The 'name' attribute specifies a unique name for the resource within the enclosing service and 'class' specifies the implementing class (this is implicitly prefixed by 'Camelot::Resource::' and thus expands to 'Camelot::Resource::DBI::Table::Row' - 'module' can be used in place of 'class' to avoid such a prefix being added). Additional parameters enclosed within the <resource>...</resource> block are passed to the resource constructor.

<resource name="zoo" class="DBI::Table::Row">
  <dsn>dbi:mysql:camelot</dsn>
  <table>animals</table>
  <key>id</key>
</resource>

This fragment defines an actor to implement some application logic. The 'name' attribute specifies a unique name for the actor within the enclosing service and 'module' defines the implementing class (no prefix added, unlike 'class' which for actors would add the 'Camelot::Actor::' prefix). One additional constructor argument is also provided.

<actor name="keeper" module="My::Zoo::Keeper">
  <brush>soft</brush>
</actor>

This fragment defines a presenter named 'tt2' which uses the the Template Toolkit via the 'Camelot::Presenter::Template' class.

<presenter name="tt2" class="Template">
  <include_path>/usr/local/camelot/zoo</include_path>  
  <pre_process>header.tt2</pre_process>
  <post_process>footer.tt2</post_process>
</presenter>

To pull it all together, we can now define a service named 'animal' which acquires an animal record from the 'zoo' resource (DBI table) using the value of the current data item 'animal_name' as an argument, and stores the returned data in the 'animal' item. This is then passed to the 'keeper' actor which performs some arbitrary processing on it, identified here as the action 'groom'. Finally the data is presented via the 'tt2' presenter, using the 'animal.tt2' template as a view.

<service name="animal">
  <acquire resource="zoo" name="animal" item="animal_name"/>
  <invoke  actor="keeper" action="groom" item="animal"/>
  <present presenter="tt2" view="animal.tt2"/>
</service>