Happstack, Fay, & Acid-State: Shared Datatypes are Awesome

Haskell is slowly moving onto the browser -- and that is very exciting. We have Fay, GHCJS, UHCJS, Haskell-like languages such as Elm, and more!

In this post I want to demonstrate two ways in which this is awesome.

1. we can now define a data-type once and use it everywhere

2. we can now define a type-checked AJAX interface between the browser and the server.

In this post we will be using:

• happstack-server - a modern Haskell based web application server
• acid-state - a native Haskell database system
• fay - a compiler which compiles a subset of Haskell to Javascript

The Bad Old Days

Let's first consider a more traditional system where we use happstack-server, a SQL database, and Javascript. If we have a value we want to store in the database, manipulate on the server, and send to the client, we need to manually create several representations of this value and manually write code to marshal to and from the various representations.

In the SQL database, the value needs to be represented via columns and relations in one or more tables. On the server-side we need to create an algebraic data type to represent the value. To transmit the value to the client we need to the convert the value into a JSON object. And then on the client-side we may then need to convert the JSON value into a Javascript object.

To switch between each of these representations we need to manually write code. For example, we need to write the SQL statements to update and retrieve the value from the database.

So in addition to the four representations of our data, we have 3 bidirectional conversions to manage as well:

SQL <=> ADT <=> JSON <=> JAVASCRIPT

Now let's say we need to make a change to our datatype -- we have to correctly update 10 different aspects of our code.

Because SQL and Javascript are outside of the domain of Haskell, we don't even get the help of the typechecker to make sure we have keep all the types in-sync.

A popular mantra in computer programming is DRY - "Don't repeat yourself". Yet, here we have to repeat ourselves 10 times!

In addition to keeping everything in sync, we still have the problem of having to think about the same data in 4 different ways:

1. as relational data
2. as an algebraic data type
3. as a JSON object
4. as a Javascript object

The Path to Awesome

The picture when using happstack-server, acid-state, and fay is radically different. In this system we define our data type as a nice algebraic data type which can be stored in acid-state, manipulated on the server, and sent to the client, where it is also treated as the same ADT. This definition occurs in once in a normal Haskell file that is shared by all three pieces of the system.

The data does still need to be serialized by acid-state and for communication to/from the client (via AJAX), however, this serialization is done entirely automatically via template haskell and generics.

mastermind

I have created a simple example of using happstack-server, acid-state, and fay to implement an interactive web 2.0 mastermind clone. The board updates all occur client side and communication is done over a typed AJAX communication channel.

You can find all the source code here:

http://hub.darcs.net/stepcut/mastermind

A demonstration of the game play is shown is this video:

http://www.youtube.com/watch?v=K2jdUlhX_E8

There are some bugs in the code, unimplemented features, etc. It seems to display correctly in Chrome, but not Mastermind. If any of these things bother you, feel free to submit patches. These issues, do not get in the way of the interesting things we want to demonstrate, and so they will likely remain unfixed.

The tree is organized as follows:

• MasterMind.Client.* - client-side Fay code
• MasterMind.Server.* - server-side Haskell code
• MasterMind.Shared.* - code that is shared between the client and server

shared types

MasterMind.Shared.Core contains the datatypes needed to define the state of the game board. There is not much to say about these types -- they are basically what you would expect to see for a game like mastermind.

In MasterMind.Server.Acid those types are stored persistently using acid-state. All played games are retained in the database, though there is currently no code implemented to browse them.

In MasterMind.Client.Main those same types (such as Color, Guess, and Board) are imported and used for the client-side interactions.

By virtue of the fact that everything fits together so seamlessly -- there isn't much to say. It looks like we just defined some normal Haskell datatypes and used them in normal Haskell code -- just like any other Haskell program. The interesting part is really what is missing! We've managed to eliminate all that manual conversion, having to think about multiple representation of the same data, javascript, SQL, etc, and left ourselves with nice, simple Haskell code! When we want to change the type, we just change the type in one place. If we need to update code, the type-checker will complain and let us know!

Best of all, we do not need to rely on special syntax introduced via QuasiQuotation. We define the types using normal Haskell data declarations.

There is a bit of Template Haskell code in the acid-state portions of the code. To create SafeCopy instances we use deriveSafeCopy. In principle this is not much different from the standard deriving Data, Typeable mechanism. However, for those that eschew Template Haskell, there is work on allowing SafeCopy to use the new Generics features in GHC 7.2.

There is also a Template Haskell function makeAcidic which would be a bit more difficult to remove.

The typed AJAX interface

Now that we have a way to share types between the client and server, it is relatively straight-forward to use those types to build a type-safe communication channel between the client and server.

At the end of MasterMind.Shared.Core there is a type:

> data Command
>     = SendGuess Guess (ResponseType (Maybe Row))
>     | FetchBoard      (ResponseType (Maybe Board))
>     deriving (Read, Show, Data, Typeable)

The Command type defines the AJAX interface between the server and the client. The constructors 'SendGuess' and 'FetchBoard' are commands that the client wants to send, and the ResponseType a is what the server will return.

It would be far more sensible to declare Command as a GADT:

> data Command r where
>     SendGuess  :: Guess -> Command (Maybe Row)
>     FetchBoard ::          Command (Maybe Board)

Unfortuantely, Fay does not support GADTs at this time, so we have to use a series of hacks to get the type safety we are hoping for. Language.Fay.AJAX (from happstack-fay) defines a type:

> data ResponseType a = ResponseType

This gives us a phantom type variable that we can use to encode the type of the response.

Looking at the Command type again, you will see that the last argument to every constructor is a ResponseType value:

> data Command
>     = SendGuess Guess (ResponseType (Maybe Row))
>     | FetchBoard      (ResponseType (Maybe Board))
>       deriving (Read, Show, Data, Typeable)

On the client-side we can use call to send an AJAX command:

> call :: (Foreign cmd, Foreign res) =>
>         String                    -- ^ URL to @POST@ AJAX request to
>      -> (ResponseType res -> cmd) -- ^ AJAX command to send to server
>      -> (res -> Fay ())           -- ^ callback function to handle response
>      -> Fay ()

For example:

> call "/ajax" (SendGuess guess) $ \mRow ->
>     case mRow of
>        Nothing    -> alert "Invalid game id"
>        (Just row) -> updateBoard row

You will note that the type signature for call is a bit funny. The type for the cmd argument is:

> (ResponseType res -> cmd)

instead of just

> cmd

But on closer examination, we see that is how the type-checker is able to enforce that command and response handler types match. When we actually use call we just leave off the last argument to the constructor, and the code is quite readable.

Also, note that call is asynchronous -- meaning that call we return immediately, and the handler will be called after the server sends back a response. That is why we pass in a callback function instead of just doing:

> mRow <- call "/ajax" (SendGuess guess) -- this is not how it actually works

We could create a synchronous version of call, however the underlying javascript engine is single-threaded and that could result in the UI blocking. We could probably give the appearance of a blocking call by using continuations in some fashion, but we will consider that another time.

On the server-side we use a pair of functions:

> handleCommand :: (Data cmd, Show cmd, Happstack m) =>
>                  (cmd -> m Response)
>               -> m Response
>
> fayResponse :: (Happstack m, Show a) =>
>                ResponseType a
>             -> m a
>             -> m Response

handleCommand decodes the AJAX request and passes it to a handler. fayResponse is used to convert a return value into a valid Fay response. The ResponseType a parameter is used to enforce type safety. So in the code we are going to have something like this in our top-level route:

>                    , dir "json"     $ handleCommand (commandR acid)

where commandR looks like:

> commandR :: AcidState Games
>          -> Command
>          -> ServerPart Response
> commandR acid cmd =
>     case cmd of
>       (SendGuess guess rt) -> fayResponse rt $ sendGuessC acid guess
>       (FetchBoard rt)      -> fayResponse rt $ fetchBoardC acid

We see that we pull the ResponseType value from the constructor and pass it to fayResponse, so that the type checker will enforce that constraint.

The command handlers have types like:

> sendGuessC :: AcidState Games > -> Guess > -> ServerPart (Maybe Row)

> fetchBoardC :: AcidState Games > -> ServerPart (Maybe Board)

Hopefully we can add GADTs to Fay soon, which will remove some of the boilerplate.

Cabal

If we want to use cabal to build and install our web application, then we need to tell cabal how to compile the Fay code to Javscript. I believe the long term plan is for Cabal to somehow directly support Fay packages. But in the meantime, this custom Setup.hs seems to do the trick:

Setup.hs for building Fay code

Note that in mastermind.cabal we have build-type: Custom instead of that standard build-type: Simple. You need to specify Custom or cabal will ignore the Setup.hs.

What Still Sucks

Fay is still very raw and buggy. In order to get this simple application working I had to file four bugs against Fay and commit several other patches myself. When the developers say that Fay is still alpha they mean it.

On the other hand, the Fay team was very responsive and fixed my issues quickly!

If you want to experiment with Fay, I highly recommend it -- but be prepared to run into some issues.

Programming in Fay is far nicer than Javascript. But, ultimately we still have to deal with the DOM model that the browser is based around. And, even in Fay, that still sucks (even with bootstrap and jQuery to help). However, now that we have a nice language to work with, we can hopefully create a nice Fay-based library for client-side UI development.

Conclusion

Fay (and friends) are definitely a huge step in the right direction. Things are still just getting started, but they are definitely set to revolution Haskell web programming. We already have mature solutions for web 1.0 programming such as happstack-server and reform. But, technologies like Fay are making it far easier to provide web 2.0 solutions with rich client-side functionality.

I have released happstack-fay on hackage which provides the glue code needed for AJAX communication.

In future blog posts I hope to cover three additional topics:

1. how to incorporate type-safe routing using web-routes

2. how to add client-side validation to reform using Fay

3. how to use HSX for client-side HTML generation

We would love to hear your feedback!