fDB: F# and JSON Database on Lucene – Part 1

I am experimenting with F# a bit and decided bring in lucene something I have used extensively in the past and vet through building out on Azure.

Couple of things about the approach.

1. HTTP Interface through a custom HTTP Handler written in F#
2. Wrappers for Lucene to save data, manage index, query
3. Open Source Project on Codeplex for the “AzureDirectory” extention to lucene.

First, let’s take a look at what the input mechnism will be:

http://domain.com/<catalog>/<action>?(<timeout>)<query>

We will need to write a function to take the relevant parts from the HTTP call, decode, and have the data ready for the core services of the platform to execute on. The below method returns a 4 item tuple of; TimeOut, Feature, Action, Parms – to use in the rest of the application.

type HttpHandler() =
    let GetHttpParts(rqst:HttpRequest) =
        let items = rqst.Url.AbsolutePath.Split('/')

        let ftr =
            if items.GetValue(1).ToString().Trim().Length = 0 then
                "index"
            else
                items.GetValue(1).ToString().ToLower()

        let (parms, timeOut) =
            match rqst.HttpMethod with
                | "GET" ->
                    if rqst.Url.Query.Length > 0  && (rqst.Url.Query.StartsWith("?(")=false) then
                        (HttpUtility.UrlDecode(rqst.Url.Query.Substring(1)), 0)
                    else
                        if rqst.Url.Query.StartsWith("?(") then
                            let s = rqst.Url.Query.Substring(2)
                            let pos = s.IndexOf(")")
                            let sTO = s.Substring(0, pos)
                            (HttpUtility.UrlDecode(s.Substring(pos+1)), Int32.Parse(sTO))
                        else
                            ("", 0)
                | _ ->
                    let rdr = new StreamReader(rqst.InputStream)
                    let s = rqst.Url.Query.Substring(1)
                    let pos = s.IndexOf(")")
                    let sTO = s.Substring(1, pos-1)
                    (rdr.ReadToEnd(), Int32.Parse(sTO))

        match items.Length with
            | 2 -> (timeOut, ftr, "", parms)
            | 3 | 4 -> (timeOut, ftr, items.GetValue(2).ToString().ToLower(), parms)
            | _ -> (timeOut, "index", "", parms)

Now we need to start to build out the ProcessRequest method of the HTTP Handler. Keep in mind, we want to enable the timeout to be respected. To do this, I am going to use the async capabilities available in Task Parrallel and the specific F# constructs that exists to wrap the work in an async computation and call RunSynchronously setting the timeout.

   interface IHttpHandler with
        member this.ProcessRequest(ctx:HttpContext) =
            let (timeOut, feature, action, parms) = GetHttpParts(ctx.Request)

            let result =
                let operation = async {
                    let results =
                       // do work here based on the Feature and Action Passed in to the HTTP Call
                    return results
                }
                let result =
                    if timeOut > 0 then
                        try
                            Async.RunSynchronously(operation, timeout=timeOut)
                        with
                            | exp ->
                                "ERROR: TIMEOUT " + timeOut.ToString()
                    else
                        Async.RunSynchronously operation
                result

            ctx.Response.Write(JsonConvert.SerializeObject(result))
            ctx.Response.End()

We will come back and write the code to go do some work based on the Feature and Action passed in. First let’s focus on building out the F# module and classes for the Lucene and Azure work in Part 2.

Advertisement