The designers and implementors of a large or complex application can rarely anticipate all requirements future users will have on the application. Typically, users wish to be able to customize the user interfaces of applications according to their personal tastes or requirements, or they want to extend the functionality of an application (either by combining existing functions into new ones or by adding entirely new capabilities). This is especially true for applications used routinely, such as text editors, and for applications with a high degree of user interaction or with complex graphical user interfaces.
Certainly any application can be customized by modifying its source code and recompiling it. But this approach is often not feasible, as the source code of the application or the tools needed to recompile it may not be available. Even if it were feasible, it would be a time-consuming process; it would be hard to keep up with new releases of the application; and the coexistence of multiple, similar versions of the same application would become a general maintenance headache.
The alternative to this approach is not to ``hard-wire'' the entire functionality and all external aspects of an application in the source code at all, but to provide means to customize the application's behavior later by its users.
Many applications support at least simple methods for customization, such as command line options or configuration files. More powerful tools for customization are macro languages, command languages, or scripting languages that are typically found in text editors and word processors. Prominent examples of such customization and extension languages are the macro language of the now legendary TECO editor and, in UNIX, the macro language of the troff text formatter [Ossanna 1979] and the configuration language of the sendmail program.
Although many of these classic extension languages are quite powerful (some of them are full-fledged programming languages), they have a reputation of being ``cryptic'' and hard to understand and use by untrained users. The prevailing opinion seems to be that only experts can actually benefit from these types of extension languages (for example, people who have mastered the sendmail configuration language in all details are commonly appointed the status of a ``guru''). In fact, it can be observed that only very few users of the troff text formatter (whose macro language is reputed to be particularly cryptic) are using macro packages written by themselves; many users give up after some time and fall back on vendor-supplied macro packages or packages written by a ``troff guru.''
Experience also indicates that simplified or specialized extension languages often have more features added and grow until they resemble a full programming language. Such ``organically grown'' extension languages are likely to be contorted designs as they will consist of several levels of extensions glued on to their initial, more limited design.
Recently application designers have begun to abandon specialized and cryptic macro-style extension languages in favor of extension languages that resemble usual high-level programming languages, mainly languages with Algol/Pascal-style or Lisp-style syntax and semantics. Prominent examples of such high-level extension languages are TPU developed by DEC, the Ness language of the Andrew Toolkit [Hansen 1990], AutoDesk's CAD extension language (a dialect of Lisp), and Emacs-Lisp, the extension language of Richard Stallman's popular GNU Emacs editor [Stallman 1981, Lewis et al. 1990].
Emacs was the first wide-spread application to employ an already existing and widely used high-level programming language as its extension and customization language. Emacs-Lisp is a dynamically scoped dialect of Lisp with additional operations for text-editing. The approach taken by Emacs has been tremendously successful; users of Emacs have contributed a wealth of extensions written in Emacs-Lisp.
Note that Emacs-Lisp is not a scripting language. It is tightly interwoven with the application for which it provides extensibility. It also is somewhat inaccessible to the casual user, who is unlikely to have previous experience with Lisp-like languages. This can be contrasted with languages such as Tcl [Ousterhout 1990] and REXX [Cowlishaw 1985], whose underlying models are no less complex, but which are similar enough to well-known languages such as BASIC to present less of an obstacle to casual users. On the other hand, non-trivial extensions benefit from the structuring functionality inherent in general purpose programming languages such as Lisp.
Using Lisp or Lisp-style languages as extension languages seems to enjoy growing popularity; several applications besides Emacs now use dialects of Lisp as their extension language. This development has one disadvantage: the number of incompatible (but similar) extension languages is continually growing. Users have to learn a new language for each new application, and application writers keep implementing new extension language interpreters instead of reusing existing ones.
These problems can be solved by a general, reusable extension language implementation that application writers can include into their applications, an extension language kit. The main objective of the Elk project was to develop such an extension language kit and to make it freely available to encourage use by application writers.