felideon's road to lisp

uncommon experiences in web application development

  • Intro to UCW
    • 7
      21 Oct 2009

      hello uncommon world

      • Edit
      • Delete
      • Tags
      • Autopost

      Today's post will kick off a series of posts introducing UnCommon Web (UCW). It'll pretty much be my own Cliff Notes version of getting-started.txt, where I'll explore and write about topics I've found interesting or somewhat confusing.

      In this post, I'll discuss two aspects of CLOS I found interesting (read as "I did not understand"), as well as describing UCW's RERL protocol by creating a hello world application.

      CLOS newbie

      Not being too familiar with CLOS, I had to look into a few things after reading the getting started doc, including skimming through the two chapters on CLOS ("Object Reorientation") in PCL. I also found a nice, brief guide to CLOS if you really want a quick refresher.

      If you're an experienced Lisper, you may want to skip ahead to the section on Uncommon Web's RERL.

      Mixins
      One thing I see a lot in the code snippets class definitions that seem to inherit some sort of superclass with the -mixin naming convention. I really had no idea what this was or what it meant. Not being entirely convinced with Wikipedia's definition, I asked around in the #lisp channel:

      "What are mixins?"
      They're regular super-classes, but instead of being used to define a class hierarchy, you use them as a means of adding features to your class. -tic

      Mixins [are] an abstract concept involving the way multiple classes can be inherited from. There's no such thing as a "mixin" in Lisp: only classes and multiple inheritance. -sykopomp

      "When should I call something a mixin?"
      Usually, 'mixins' are just classes that you only wrote with the intent of adding behavior. In general, you don't use mixins directly, you inherit from them. -sykopomp

      Mixin-ness is an information intent. When the class is not meant to be [instantiated] (abstract), nor meant to be inherited from alone (not just a regular abstract base class). -pkhuong

      Thank you, kind sirs.

      Now let's look at an example. When we define a sample application class in UCW, such as:

      (defclass example-application (standard-application 
                                     cookie-session-application-mixin)
        ()
        (:default-initargs
          :url-prefix "/example/"))

      We are inheriting from both standard-application and cookie-session-application-mixin classes. the intent here is that example-application directly specializes the standard-application superclass—establishing an is-a relationship—but it also needs the features provided by the cookie-session-application-mixin. (In contrast, probably establishing a has-a relationship, if one could say "Example application has a cookie session feature.")

      What's up with the empty parens?
      Something else that got my attention in the code is the use of empty parens in the body of defclasses. The only thing supplied is a default-initarg for :url-prefix. What is url-prefix?

      With help from SLIME's M-. command, we can easily find that :url-prefix is a slot in the MINIMAL-APPLICATION class (defined in src/rerl/basic-classes.lisp) which is an indirect superclass of our example application as so:

      STANDARD-APPLICATION -> BASIC-APPLICATION -> MINIMAL-APPLICATION -> APPLICATION

      In this respect, here's how each subclass specializes its superclass:

      MINIMAL-APPLICATION
      Defines a slots called url-prefix which is a string specifying the start (prefix) of all the urls this app should handle. This value is used by the STANDARD-SERVER to decide what app a particular request is aimed at and for generating links to actions within the app.

      Also defines the following slots: request-context-class, server, lock, and debug-on-error.

      BASIC-APPLICATION
      Additionally inherits a mixin called APPLICATION-WITH-SESSION-HANDLING-MIXIN, which as you might guess, adds session handling capabilities to applications.

      Defines slots named charset and dispatchers. Kind of self-explanatory but see documentation strings in basic-classes.lisp for more info.

      STANDARD-APPLICATION
      Defines a slot named action-class. Actions and callbacks will be explored in a future post, but you can read about them in the getting started document.

      Also notice the empty parens when defining the "hello.ucw" entry point:

      (defentry-point "hello.ucw" (:application *example-application*
                                   :with-call/cc nil) 
          ()
        (format (html-stream (context.response *context*))
                "Hello World"))

      Now, this one isn't as obvious since DEFENTRY-POINT is a macro. But without looking at the code, it seems like that list form is used when you need to supply parameters to an entry point, as seen in this example:

      (defentry-point "hello2.ucw" (:application *example-application*
                                   :with-call/cc nil) 
          ((message "World"))
        (format (html-stream (context.response *context*))
                "Hello ~A" message))

      If left empty, you are defining an entry-point that does not need URL arguments.

      What is the RERL?

      From the getting started doc:

      UCW is based around the Request Eval Response Loop (RERL). The RERL is an extensible CLOS protocol that takes care of the details, while still allowing the programmer to hook in at any point. Because of this extensibility, UCW is suitable for all types of web applications. The more advanced features of Uncommon Web are especially suited for highly stateful applications with a complex control flow. The developer is free to use an much or as little of the RERL as desired for any given request. One can freely intermix a REST-style with a heavily stateful CPS approach where appropriate.

      (Emphasis mine, keep reading.)

      There is also another concise definition from the documentation in src/rerl/protocol.lisp:

      A UCW server sits in a tight loop waiting for incoming HTTP requests, elaborating them and then sending out the HTTP responses. This loop is called the "RERL" (Request-Eval-Response Loop) and the objects, methods and variables involved in the RERL are described here.

      In the documentation we can see that objects and methods for:

      • Backends
      • Servers
      • Applications
      • Rendering
      • Components
      • Request contexts
      • Sessions
      • Session Frames

      Are defined as being part of the RERL.

      This sheds some light on the phrase I emphasized above. For example, you don't have to use components or session frames when building web apps with UCW. You can even use components without sessions if you wanted to. (Thanks unknown_lamer.) In other words, the RERL gives you tools to build a web application in the style you need—anywhere from basic parameter passing through forms to continuation-passing style control flow.

      Backends, servers, and applications

      One thing to clarify is that the backend is the web server responsible for handling HTTP requests, whereas the server object (we'll call it the UCW server) is the abstracted "logical server". The UCW server's main responsibilities are to take request and response objects provided by the backend and run one iteration of the RERL, and to determine the application which should handle the request. (See more functionality provided by the SERVER class in protocol source code.)

      Note that you can use UCW on top of any backend, e.g. mod_lisp, but UCW has a decent internal HTTPD. This is another reason a logical server object comes in handy, as it even allows UCW to deal with multiple backends. (I'll try to write about this in a future blog post.)

      As far applications go, their main responsibility in UCW is to serve as containers for DISPATCHERS (e.g. ENTRY-POINTS), and as a place to store SESSION objects.

      Hello, world!

      Alright, it's time we write a simple hello world with UCW. We need to instantiate a backend server and application that will host our web application.

      Instantiating a server and application with Uncommon Web
      To start a server, we need to first make an instance of the STANDARD-SERVER class and pass it a BACKEND object. We can create a BACKEND object with the MAKE-BACKEND method and return it to the :backend slot of our STANDARD-SERVER instance.

      After this object is created, we can use the STARTUP-SERVER method to start it.

      (defvar *example-server* 
        (make-instance 'standard-server 
          :backend (make-backend :httpd :port 8080)))
      
      (defun start-example-server ()
        (startup-server *example-server*))

      Once you've evaluated the code above, go ahead and call START-EXAMPLE-SERVER at the REPL. You should see some output log from UCW, and if you go to your browser and navigate to http://localhost:8080, you should get a 404 error. This means we have the server up and running. If not, you would have gotten an error message from your browser stating something to the effect of not finding the server or not being able to establish a connection.

      The next step is to create an application. In order to create an application, you can define a new subclass that inherits from STANDARD-APPLICATION and any mixins you want to add, as well as specifying a default URL-PREFIX to match requests with. Once you instantiate the application you need to register it with the server created above.

      (defclass example-application (standard-application 
                                     cookie-session-application-mixin)
        ()
        (:default-initargs
          :url-prefix "/example/"))
      
      (defvar *example-application* (make-instance 'example-application))
      
      (register-application *example-server* *example-application*)

      If we try to browse to our site, we will still get a 404 error even if we try http://localhost:8080/example/. UnCommon Web matches the request against its default dispatchers, i.e. ACTION-DISPATCHER, but we have not defined any handlers. We thus enter the EVAL stage of the RERL.

      Creating your first entry point
      I think the sections on entry-points in the getting started doc are pretty straightforward. But to summarize, entry points are pretty much external URLs that are "bookmarkable, REST-style linkable resources exposed directly to the web." You can also think of an entry point as a page.

      (defentry-point "hello.ucw" (:application *example-application*
                                   :with-call/cc nil) 
          ()
        (format (html-stream (context.response *context*))
                "hello, uncommon world"))

      Please refer to getting-started.txt for more information on what this code is doing.

      Go ahead and evaluate DEFENTRY-POINT above to create the hello.ucw entry point and browse to http://localhost:8080/example/hello.ucw. A page is finally served, completing UCW's RESPONSE stage.

      Additionally, entry points can handle query strings. Suppose we wanted to change our greeting by passing in a string in the URL.

      (defentry-point "hello.ucw" (:application *example-application*
                                                :with-call/cc nil) 
        ((message "world"))
        (format (html-stream (context.response *context*))
                "hello, uncommon ~A" message))

      Here we specify an argument message with a default value of "world". If we go back to our "page" and specify a query string, such as in http://localhost:8080/example/hello.ucw?message=universe, "hello, uncommon universe" will be displayed. Otherwise, the default one will be displayed.

      Next Time: UCW Components

      For my next post, I plan on writing about Uncommon Web's components. I'll probably skip this section on creating an HTML form with YACLML, so you should read that in getting-started.txt on your own if you are interested. (It's pretty straightforward.)

      Hopefully I'll get some time to starting writing it sooner and for longer periods of time. My initial goal is to have at least two posts per month. I'll be posting updates, so you should follow me on twitter here if you are interested.

      • views
      • Tweet
      • Tweet
    • 3
      19 Sep 2009

      Preparing for the road ahead

      • Edit
      • Delete
      • Tags
      • Autopost

      UnCommon Web Documentation

      Let me make this clear from the beginning. Do not make the same mistake I made, which was to look for UCW documentation on the web. If you even happen to find any that doesn't 404, don't use it. The only up-to-date and relevant documentation for UCW is pretty much the source code and the getting-started.txt file included with the library. Above all, avoid the Trac site like the plague.

      That being said, it's now obvious that if you are the type of person who will only learn a new language/framework/technology as long as there are instructions every step of the way, UCW isn't for you. But then again, the same applies to Lisp.  Hopefully, this blog will help a few would-be web-lispers to take the plunge.

      What is UnCommon Web?

      Straight from the getting started file:

      UCW is a multi-paradigm framework for building Web based applications in Common Lisp. It has an extensible core, the RERL, that enables many methods and styles of web development, and can operate using almost web server as a front end, including a few that come built in.

      UCW includes a component oriented system allows both the graphical and the presentation logic to be easily reused and adapted, and has features that allow developers to write complex page flow as if it was a "regular" sequence of function calls.

      UCW has been used to develop all kinds of web sites and applications just using the standard library (ucw-standard) but also serves as an excellent platform for developing higher-level frameworks, such as lisp-on-lines, ucw_ajax and core-server.

      UCW is not a full-stack, all or nothing approach. UCW does not include form builders, Object/Relational mapping or any database support. UCW deals only with the web side of things, like dealing with HTTP requests, sessions, marshalling/unmarshalling of GET/POST variables and the like.

      Users are encouraged to explore the higher-level ucw-based frameworks once a solid grounding in UCW concepts has been acheived.

      I also recommend the features page from the UCW website.

      Why UnCommon Web?

      Not having used UCW yet, the question I am striving to answer is: Why is UnCommon Web the best web framework out there? For now I have faith that it actually is, but I need to get my hands dirty and experience why. I have some ideas in which UCW will help me come up with a solid and effective foundation for developing web apps. I see UCW the same way I see Lisp: powerful, expressive, and elegant.
       
      One reason my interest in UCW is particularly piqued has a lot to do with Drew Crampsie's lisp-on-lines (aka lol), which is built on UCW and abstracts Pascal Costanza's CLOS extension for context-oriented programming, ContextL.

      Homework—Light Reading

      One thing not clearly expressed in the getting started doc (although it becomes apparent once you see the code snippets), is how heavily based on CLOS (Common Lisp Object System) it is. I'd go as far as saying that it pretty much seems like an extension to CLOS for the web. Moreover, it uses the CLOS Metaobject Protocol (MOP) for defining components.
       
      That being said, I recommend the following light reading:

      From Practical Common Lisp:
      Object Reorientation: Generic Functions
      Object Reorientation: Classes
       
      From the Common Lisp Cookbook:
      Fundamentals of CLOS

      Additionally, if you're not familiar with the term or concept, you may want to read on what continuation-passing style (CPS) is, as UCW's control flow logic is implemented as a CPS interpreter (Common Lisp does not support native CPS). 

      Last but not least, if you would like to get a head start, you can go ahead and read the getting started doc which is included once you've downloaded the library. Following instructions in the next section, it will be located at /path/to/clbuild/source/ucw/doc/getting-started.txt.  (Update:  I recommend enabling org-mode for this file in Emacs.  Since it is in outline format, it will be easier to follow.)

      Installing UCW with Quicklisp

      (Updated) You should use Quicklisp to install UCW and its dependencies. If you haven't installed it already, now would be a great time.
       
      To download UCW and its dependencies once quicklisp is installed, run (ql:quickload "ucw" "puri" "rfc2388-binary") at the REPL and you should be good to go.

      Starting the demo

      Once you've installed UCW and all its dependencies, go ahead and launch SLIME in Emacs. Compile UCW and it's dependent packages, and enter the UCW package:

      CL-USER> (require 'ucw)
      CL-USER> (in-package :ucw)

       
      Then, load the demo and run it by typing the following:

      UCW> (load "/path/to/clbuild/source/ucw/demo/demo.lisp")
      UCW> (startup-demo)

       
      You should see something like this:

      1:02 UCW INFO Spawned new worker thread # 
      01:02 UCW INFO Spawned new worker thread # 
      01:02 UCW INFO Started standard-server # 
      with backend #


      Note the last line, where the app has started on the localhost server on port 9090. To try it out, open a browser window and go to http://localhost:9090/demo/index.ucw.
       
      Click around and enjoy! Take a look at the demo.lisp source code to see what's going on.  However, there are two things from the demo code that caught my attention that I needed some clarification on.  First, the usage of defcomponent over defclass (inconsistent with getting started doc). Thanks to unknown_lamer from #ucw, defcomponent is pretty much syntactic sugar for

      (defclass ... 
              (:metaclass standard-component-class))


      Second is that, being unfamiliar with CLOS, I wanted to find out why (defmethod render ((self ...)) ...) was being used so much.  The answer was provided by drewc:

      The reason you see SELF a lot in the UCW source code is historical more than anything, and I personally think it's bad style. In older versions of UCW, CALL was a macro that relied on a hidden lexical variable, SELF, that DEFACTION would set up. This meant that, unless you named the component SELF in the RENDER method definition, you couldn't do something like (<ucw:a :action (call ...)) in a RENDER method.
       
      CALL has since been completely re-written, but my fingers tend to still type (defmethod render ((self ...))) automatically some times, and a lot of older source code still uses the convention. I'd love to eradicate parameters named SELF from the UCW code, but it's not a major priority.

      In closing...

      Well they're you have it. You should be ready to start playing around with UCW, and you should be able to follow along with my posts. For next time, we can start writing a hello world program. 
       
      If you find any problems while toying around with UCW, there is a #ucw channel on freenode. (Tip: You can ask a question and then idle in the chat until someone notices and gives an answer.)
       
      Subscribe to my posterous blog with the link at the bottom of the page, or you should follow me on twitter here. Expect new blog posts every other week, if not sooner.

      • views
      • Tweet
      • Tweet
    • Search

    • Sites I Like

      • Planet Lisp
      • Quicklisp
      • qooxdoo
      • Hacker News
    • Tags

      • common lisp
      • ucw
      • uncommon web
      • web development
      • lisp
      • components
      • hello world
      • introduction
      • method-combination
      • render
      • ui
    • Archive

      • 2010 (1)
        • January (1)
      • 2009 (3)
        • October (1)
        • September (2)
    • Obox Design
  • felideon's road to lisp

    Hi, my name is Felipe and I am a Lisp hacker and web developer.

    Drop me a line at felideon+blog@gmail.com. You should also follow me on twitter here.

    46160 Views
  • Get Updates

    Follow this Space »
    You're following this Space (Edit)
    You're a contributor here (Edit)
    This is your Space (Edit)
    Follow by email »
    Get the latest updates in your email box automatically.
    Loading...
    Subscribe via RSS
    Twitter