mebassett programming

Metaclasses, first class objects, and a lesson from SICP.

2012-02-08T16:44:00.000-08:00

This week my SICP Study Group is looking at functions as first-class objects. This means, among other things, that functions can be passed as arguments and returned as values of other functions. A few members of my group feel that the material covered is too theoretical and has little real-world value. I'm always a bit bemused by this, because I use ideas from SICP nearly every day, especially when it comes to higher-order functions. So I thought I'd use an example from my day job to show how this section of SICP is particularly useful.

Part of my job includes writing data-entry applications so that other team members can interact with my statistical models. I write these applications in Python, using a library called web.py. In this library, we can assign to each web request URI a class to handle it.

What's a Class?

It's something we haven't seen before in our study group! I don't want to explain it in detail (you can check Wikipedia), but I hope it'll suffice to say this:

A class can be thought of as a way to capture, isolate, and reuse state between a group of functions. "State" is another concept we haven't seen in my group! We've only written stateless, functional programs. "State" here means data that exists independently of the functions, but is sometimes used or modified by the functions to perform its computation (exempli gratia, the balance of a bank account might be state between a group of functions that affect one's bank account). For instance, say we have a web server handling requests from the internet. Each request needs to perform a series of computations related to that request, but the data for each request, id est, the state of the request, should be kept separate from each other. This is exactly why web.py uses classes for requests. We can define what data or state our groups of functions need to share, and we can make copies of those groups (called "instances" of the class) that won't interfere with each other's state.

What's a Class look like?

Classes in Python have a name, some data members, and some functions. Classes to handle web requests with web.py generally look something like this:

class RequestHandler:
    request_var1 = "data..."
    request_var2 = "more data..."
    def GET(self):
        i = web.input()
        #code to handle request...
        return html_output

Web.py will look for a function GET inside the class when the web request comes with GET headers. Each time the application gets a web request, it creates a new instance of the class, id est, a fresh copy of the data and functions, and calls the new GET method with the new data.

In this specific example, the class is really just a special "wrapping" around the GET function so web.py can use it. It won't hurt the reader who is unfamiliar with classes to regard them as functions for the remainder of this post.

Now, classes to handle web request are often a lot more complex than that. In my case, I have specific requests that should respond with data that contains the current parameters of my statistical model, and the data should be formatted for easy entry into a web application (exempli gratia, in JSON). I have about two dozen of these requests that ask for different aspects of the same object. Hence if I had to write them all out, each one would look something like this:

class YetAnotherRequestHandler:
    def GET(self):
        # code to process incoming request data and check for errors
        # 5-10 lines ***same for every request***.
        ...
        # code that's specific to this request, 2-3 lines.
        ...
        # code to transform output into JSON-ready format
        # another 5-10 lines that are **the same for every request***
        return json_text

I don't want to burden the reader with those 20 lines of code that are the same for each request (nor do I want to show my employer's source code or think of my own contrived example!) but I hope I can make the point clear: each of these two dozen request handlers, they share a significant amount of code. If I were to write them all out by hand, I'd have over 80% duplicated code. Boring. Hard to read. Messy to change all at once.

Higher-order functions...erm, Classes

What if, instead of writing two dozen classes that share the same code, we could somehow abstract the similarities and only write the difference? Say, we could write a series of functions that do the specific-to-request bit:

def processor_for_request1(args...)
    # 2 lines of code here
    return data
    
def processor_for_request2(args...)
    # another 2 lines of code here
    return data
    
# and so on...

And pass those functions into some machine that will turn them into classes? Well, we can! Luckily, in Python, functions and classes are first class citizens, so all the techniques in our section of SICP apply here. We can write a function that takes these processors as input and returns a class that uses them like so:

def request_class_creator(processing_function):
    class Abstract_Request:
        # code to process incoming request data and check for errors
        # 5-10 lines ***same for every request***.
        data = processing_function(args...)
        # code to transform output into JSON-ready format
        # another 5-10 lines that are **the same for every request***
        return json_text
   
Request_Handler1 = request_class_creator(processor_for_request1)
Request_Handler2 = request_class_creator(processor_for_request2)
# ...and so on

In this way, we only write once the code that is different, we can reuse the components that work, and we can easily see what the important parts of our program are. Additionally, we can easily write separate unit tests for each processing function and for the Abstract_Request handler. We obviously aren't covering unit testing in the SICP study group, but it's an important part of writing good software nevertheless.

While the language and the example is different, this is the exact same technique described in Sec 1.3 of SICP. In fact, it's where I learned it from. Python users call this technique a metaclass. Python includes a few constructs to expand how one might use this method, but I think it's safe to say that the basic ideas of it are well-explained in SICP.

Perceptrons in Lisp (A simple machine learning exercise)

2012-02-05T06:37:00.000-08:00

So having missed Stanford's Machine Learning course (mostly out of laziness - I'm sure it was great) I'm trying to learn this stuff on my own. I'm going through MIT's Machine Learning notes on OpenCourseWare. They're easy [for me] to digest without being insulting, and they help me avoid searching for "The right book" to learn from (a task that would delay my learning anything but make me feel busy).

After reading the first two lectures I decided I should stop and practice what I've learned: a simple perceptron learning algorithm.

What's a Perceptron anyway?

It sounds like a Transformer. It's actually a method of deciding whether an object belongs to a certain class based on a linear combination of that object's characteristics. Id est, say we have some data on atmospheric conditions, exempli gratia humidity H, wind speed W, et cetera and we want to classify those methods as to whether the conditions will produce a storm a not. We would have a linear term:

$a_1 H + a_2 W \ldots $

We want to choose the variables $a_i$ so that the above term is positive when we'll have a storm, and negative otherwise.

More generally, say we have a vector of characteristics $x$. We want to choose another vector $\boldsymbol\sigma$ so that the dot product $\textbf{x} \cdot \boldsymbol\sigma$ is positive when $x$ belongs to our class and negative otherwise. The perceptron is the function

$ f (\textbf{x}) = \text{sign}(\boldsymbol\sigma \cdot \textbf{x}) $

How do we find $\boldsymbol\sigma$?

Our learning algorithm will tell us how to choose that $\boldsymbol\sigma$. To do this, we need some training data. Id est, some vectors $\textbf{x}_1, \ldots, \textbf{x}_n$ along with labels $y_i$ where $y_i = 1$ if $\textbf{x}_i$ belongs to our class and $y_i = -1$ otherwise.

We're going to start with any ol' $\boldsymbol\sigma$, say just a vector with 1's in all positions. For $\textbf{x}_1$, we look at $f(\textbf{x}_1)$. If it equals $y_1$, we do nothing. If not, then we update $\boldsymbol\sigma$ with $\boldsymbol\sigma = \boldsymbol\sigma + y_1 \textbf{x}_1$. One can prove that this update rule will ensure that $f$ gets better at correctly classifying each $\textbf{x}_i$. We repeat this for all of our training data, and then we have a perceptron that's ready to classify some samples!

Let's see it in practice.

This is a pretty simple algorithm and won't take us long to implement, so let's try it. For our implementation, we're going to look at 25x25 pixel black and white images. We'll train a perceptron to identify which images are of the letter A and which aren't.

I [rather foolishly] created my own training data. I used GIMP to create 25x25 pixel images, saved them as an html table. Each cell of the table corresponded to a pixel, and GIMP saved them so that each td tag included a BGCOLOR attribute. I then used sed to convert each html file into a list of colors for each pixel. You can see the sed command on the github for this post.

I saved the resulting files with a ".img" extension, and it was pretty easy to write a racket function that would convert such a file into a vector:

 (define (img->vector filename)  
  (define filelist (with-input-from-file (string->path filename)  
            (λ ()  
             (define (iter file-list line)  
              (if (eof-object? line)  
                file-list  
                (iter (append file-list (list line)) (read-line))))  
             (iter '() (read-line)))))  
  (list->vector (map hexstr->number filelist)))

The perceptron itself is not even interesting. It's just a function that takes the sign of a vector dot product:

 (define (linear-classifier img)  
  (sign (dot sigma img)))

And the dot product function is just an application of the built-in vector map function:

 (define (dot v1 v2)  
  (apply + (vector->list (vector-map * v1 v2))))

Of course, the real "machine learning" bit is in the learning algorithm. This too is just a straightforward description of the above into lisp:

 (define (train-perceptron img-name)  
  (define label (filename->label img-name))  
  (define vec (img->vector img-name))  
  (cond ((= (linear-classifier vec) label)  
      (display "good!\n"))  
     (else  
      (set! sigma (vector-add sigma (scalar-mult label vec)))  
      (display "bad! updated sigma, though.\n"))))

Where vector-add and scalar-mult are more applications of the vector-map function in Racket's library:

 (define (vector-add v1 v2)  
  (vector-map +  
        v1  
        v2))  
 (define (scalar-mult s v)  
  (vector-map *  
        v  
        (build-vector (vector-length v) (λ (x) s))))

One final detail, I defined sigma as 625 element vector initialized to 1

 (define sigma (build-vector 625 (λ (x) 1)))

I had 25 training images for this perceptron, and it still wrongly identified my unseen images. I noticed that $\boldsymbol\sigma$ was still changing if I trained it repeatedly on the same data, and it didn't stabilize until around the tenth time. Maybe it needs more training data, maybe the perceptron is a bad algorithm for this task, I really don't know. In any case, after repeated training, it improved slightly, but still had a false negative. In any case, we have our first simple supervised machine learning algorithm. Code for this, including training data, is on my github.

There's to particular reason I used Racket/Lisp for this: it would have been just as straightforward to write it in Ruby, Python, Javascript, et cetera. I just happen to like writing in Racket.

Edit:

An intelligent commenter on Hacker News pointed out that there are some theoretical guarantees for the accuracy of your perceptron when it encounters new data. I probably should have covered this here, but because I'm out of time and eager to do other things, I'll just mention that one can prove that the learning algorithm will always settle on a specific $\boldsymbol\sigma$ after a finite number of training points $k$. Additionally, one can think of $\boldsymbol\sigma$ as a plane dividing points in space, where on one side of the plane are points in our class, and the other side are all the other points. Theoretically, there will always be a distance $d$ between the closest point in our class and the plane. $k$, of course, will depend on $d$; id est, the smaller your $d$, the harder the learning problem, and the more data you'll need to train. (To the best of my understanding!) The proof of all these things is pretty straightforward, so long as one is comfortable with basic geometry and vector-based arguments. No higher maths needed. In fact, it's covered in the second set of lecture notes!

Don't Censor the Web.

2012-01-18T14:21:00.000-08:00

Congress is considering two Orwellian-named laws, SOPA and PIPA, that are threatening free speech, internet security, and innovation.

This is a reminder to call your representatives in Congress and/or donate to the EFF today to help stop internet censorship.

Reading group for Structure and Interpretation of Computer Programs.

2012-01-12T12:50:00.000-08:00

I'm leading a study group in the computer science/software engineering classic: Structure and Interpretation of Computer Programs. The study group is part of the UCL Undergrad Maths Colloquium but students and professionals of any level are welcome to join. We start meeting on Thursday, 19 January at 7pm in London. You can find out more at the Study Group Page at the Colloquium's website.

Tales from the wild - isolation, incompetence, and database requirements

2011-08-26T05:48:00.000-07:00

I worked on my own or a led a team of less-experienced programmers throughout all of my professional experience. As a result, I am sorely missing guidance and help from other developers. While I normally have the confidence (arrogance!) to plow ahead with my own solutions, I am increasingly aware of my own incompetence, which goes unchecked without having a more senior - or at least equal - developer to pull me back into reality. For instance, here's an example of a problem I faced several months ago:

We needed to manage a list (>500 and growing) of events from a web app. Each event has a title, other general information, and a list of one or more date/place pairs. Each date/place pair has a vector of numerical info about the event. Users need a CRUD to edit all this, and it needs to keep track of each change. E.g.; a user needs to change a date without loosing the associated place and vectors, and it needs to log the change; or a user needs to change the numerical info associated to a specific event/place, et cetera. The user needs to do this from an ajaxy grid that displays the most recent version of the information (the user doesn't need to see revision history in this grid.)

How can we store this data in a way that we can retrieve it quickly? Ideally, I'd like to request all the data needed for the grid in a single SQL query. Of course, my language used in describing the problem puts many assumptions on a data storage solution already. Maybe I shouldn't be using SQL. Maybe a list of pairs isn't optimal.

My Original Solution

I wrote this solution 6+ months ago and am now embarrassed by it:
I have four MySQL tables: Events - id, titles, et cetera; EventDateBridge - a map between Event_id's and id's for date/time pairs (EventDateBridge_id); EventDates - revision history for date/place pairs with EventDateBridge_id, date, place, and time added; EventVectors - revision history for vectors associated to date/place pairs with EventDateBridge_id, time added, and the vector. To update an Event we just add a row to the EventVectors or EvenDates table. To build the grid: We need one query to grab all events. For each event we need another query to grab all EventDateBridge_id's on that event/place pair. Finally, we need two more queries - one to grab date/place and another to grab the vector.

Obvious problems: lots of "order by date_added desc limit 1" to get most recent revision, lots of sub-queries, several queries needed per Event to build the grid - this results in several thousand queries!

A Better Solution

A more sober programmer (myself, months later when returning to it) recommended this:
Three MySQL tables: Events as before; RecentEventDates - date_id, event_id, date/place pair and vector; EventDateHistory - date_id, event_id, date_added, date/place pair and vector. To update an Event, we update the RecentEventDates and add a row to EventDateHistory. To build the grid we run one a query on RecentEventDates (using a simple join to grab related event information). We can use our programming language to reassemble the result into a list of events, each event having a list of date/place pairs (otherwise, the result is a list of vectors including event info, date/place pairs, and the numerical vector.)

Am I missing some better solution? Am I missing a premade solution?

A Brief Intro to Writing Macros in Racket

2011-06-20T03:43:00.000-07:00

I plan to learn, among other things, more about lisp-like macros this Summer. I had a brief introduction to the Common Lisp macro system years ago, but was frightened away when I couldn't understand my code after taking a few days' break. I'm revisiting macros using Racket. While there are a handful of great references on Common Lisp macros (including Paul Graham's On Lisp, which I've yet to read!), I had trouble finding a basic introduction to Racket's more complicated macro system. The team at Racket/PLT Scheme have been researching macro systems for a few years now, and they've implemented a pretty sophisticated system: syntax objects are first class citizens, et cetera. Quite honestly, I don't understand any of it yet. But I thought it'd help to write down my thoughts and work thus far. Maybe it'll help someone else learn something, too!

Step 1: Getting to terms with Syntax Objects and Transformer Bindings

Lisp macros allow you to extend the syntax of the lisp language itself. Racket implements these macros by creating a syntax type to model the syntax you wish to implement. Essentially, a syntax object contains an unevaluated s-expression (lisp expression) along with some data about its scope. A syntax object is a first-class citizen, meaning it can be passed around to and fro functions like any other lisp object. You can extract data (e.g. particular symbols) from these syntax objects using pattern matching techniques, and then use that data to build a new syntax object. Id est, you can write a function that takes a syntax object and returns a new, transformed syntax object. To implement a macro, you describe the syntax and write a function to transform it. When Racket sees your new syntax, it transforms it as you described and then evaluates the resulting syntax object as an s-expression within its own scope. (At least, I think that's what happens!)

Pretend for a second that we wish to implement a while clause in Racket. We want to be able to write something like:

(while (< x 5)
(begin (display x)
(set! x (+ x 1))))

And have it display

1234

What we need is something to take the above s-expression and transform it into the following s-expression:

(define (loop)
    (if (< x 5)
        (begin
            (display x)
            (set! x (+ 1 x))
            (loop)
        )
        (display "")))
(loop)

Let's first think about how to turn the s-expression (while test body) into a syntax object. This is done with the syntax function, e.g.:

(syntax (while text body))

Racket provides a shortcut to "quote" syntax, id est, the above is equivalent to:

#`(while text body)

Let's go ahead and define a syntax object to use later in our examples:

(define stx #`(while (< x 5)
    (begin (display x)
        (set! x (+ x 1))
    )
))

But how do we transform this syntax object into another? As stated above, we'll use a sort of pattern matching to extract data from the syntax object, and plug this data into a new syntax template. Racket provides a function to implement this called syntax-case. syntax-case is a function that takes a syntax object, a list of "keywords", and a list of pairs of patterns and new syntax objects. It tries to match the syntax object you provided to one of the patterns listed, and returns the new corresponding syntax object. Those patterns can be tricky, and are best explained with examples, so let's try one: First note that our keyword is "while", let's try the code:

(syntax-case stx
(while)
[(while test body) #`body])

The (while test body) says that we're looking for an s-expression that starts with the keyword "while", and follows with two other s-expressions. It binds the syntax object corresponding to those s-expressions to the symbols "test" and "body", respectively. The new syntax object #`body is simply the body sample. Thus the above expression returns a syntax object for only the last s-expression in our while clause.

If we didn't list "while" in our keyword list, the expression would have still returned the same result. But instead of only matching while-clauses, the syntax-case expression would match anything, and simply store the first expression in a symbol named while.

Let's implement a proper transformation now. As we stated above, we want our while expression to transform into a recursive function that calls itself so long as test is true. Our syntax-case is thus:

(syntax-case stx (while)
    [(while test body)
    #`(begin (define (loop)
            (if test
                (begin body
                    (loop))
                (display "")))
        (loop))])

As you can [hopefully] see if you run this code through a REPL, we get a syntax object containing something similar to our desired s-expression. Now let's tell Racket to perform this syntax transformation. To do this, we'll use define-syntax. This statement works a lot like our normal define, except that it ties an identifier to a syntax transformation. Here's the code:

(define-syntax (while stx-obj)
    (syntax-case stx-obj (while)
        [(while test body)
        #`(begin (define (loop)
                (if test
                    (begin body
                        (loop))
                    (display "")))
            (loop))]))

Now the code

(let ((x 1))
    (while (< x 5)
        (begin (display x)
            (set! x (+ 1 x)))))

Works as expected. stx-obj captures the total expression, including the while clause that we defined. Since Racket detects this by itself, we actually don't need to define the while keyword. We can replace it with an empty keyword list and a "_" (underscore) pattern (this matches any syntax symbol, but doesn't bind the symbol to a new expression) like this:

(define-syntax (while stx-obj)
    (syntax-case stx-obj ()
        [(_ test body)
        #`(begin (define (loop)
                (if test
                    (begin body
                        (loop))
                    (display "")))
            (loop))]))

Rolling Our Own Object System

Now that we've got the basics of macros down, let's try a slightly more ambitious project: we're going to use macros to implement a basic message-passing object system. First, consider the following code:

(define (animal location health speed)
    (define (move new-loc)
        (set! location new-loc)
        (set! health (- health speed)))
    (define (dispatch m)
        (cond [(eq? m 'move) move]
            [(eq? m 'location) location]
            [(eq? m 'health) health]
            [(eq? m 'speed) speed]
            (else (error "unknown message"))))
    dispatch)

(define my-animal (animal 1 100 5))
(my-animal 'health)

     >> 100

((my-animal `move) 2)
(my-animal 'health)

     >> 95

This code implements a message-passing design pattern for my awkward model of an "animal" (perhaps for a game?). We can create new animal objects with the animal function that we defined, and then pass messages to it, emulating object-oriented programming.

Instead of writing this design pattern over and over again, we're going to create a macro to implement it for us. We want to write

(new-object Animal
    (location health speed)
    [(move (λ (new-loc)
        (set! location new-loc)
        (set! health (- health speed))))])

The pattern we wish to match is:

(_ name (value ...) [(func-name lambda-body) ...])

The ellipses (...) after "value" and "(func-name lambda-body)" denote that we expect a list of symbols, rather than just a single symbol. When we write the syntax template, we'll have to use the same ellipses to repeat the corresponding expression across the list. To transform the above code into our implementation, the code is:

#'(define (name value ...)
    (define func-name lambda-body) ...
    (define (dispatch m)
        (cond [(eq? m 'value) value] ...
            [(eq? m 'func-name) func-name] ...
            [else (error "unknown message")]))
    dispatch)

Combining all this together, we can write the following syntax transformation:

(define-syntax (new-object stx)
    (syntax-case stx ()
        [(_ name (value ...) [(func-name lambda-body) ...])
            #'(define (name value ...)
                (define func-name lambda-body) ...
                (define (dispatch m)
                    (cond [(eq? m 'value) value] ...
                        [(eq? m 'func-name) func-name] ...
                        [else (error "unknown message")]))
                dispatch)]))

This macro works fine for the above. But hold on! What if we want to implement setters for new objects? E.g.

((my-animal `set-health!) 500)

Implementing this bit is tricky...

Manipulating Syntax

...to do this, we'll need to take the syntax object for the symbol "health" and transform it into a new syntax object that contains the symbol "set-health!". We'll need to do this for the full list of members. But how can we? Let's take a break from the macros and play around with syntax objects for a bit. First, define the following:

(define stx #'(health location speed))
(syntax->list stx)

syntax->list does what it sounds like, takes a syntax object of a list and returns a list of syntax objects. Our idea is to continue transformations on that list until it becomes the following syntax object list:

#'(set-health! set-location! set-speed!)

Then we'll incorporate it into our macro definition. Racket provides another useful function, syntax->datum which will transform each individual syntax object of a symbol into the symbol itself. So our next step is

(map syntax->datum (syntax->list stx))

Now we have a list of symbols. We can transform this normally, e.g.

(map (λ (item)
(format "set-~a!" (syntax->datum item)))
(syntax->list stx))

We're getting closer, now we need to transform this list of strings back into a list of syntax objects of symbols. This is pretty straightforward, we just nest datum->syntax and string->symbol. Note the following about datum->syntax: A syntax object includes information about the scope of the s-expression it contains. We want our new syntax object to be within the same scope as the original syntax object. Luckily, datum->syntax will do exactly that. It takes both a syntax object and a symbol and returns a new syntax object containing that symbol within the scope of the first syntax object. Hence our code is

(map (λ (item)
        (datum->syntax stx
                (string->symbol (format "set-~a!" (syntax->datum item)))))
    (syntax->list stx))

That does exactly what we want it to. Now, let's re-visit the syntax template from our macro and see how to implement this idea for the object setters. Recall the macro:

We need to do an intermediate syntax transformation, to convert the "(values ...)" syntax object to the one we created earlier. We can do this with with-syntax. with-syntax takes a list of pairs of patterns and syntax objects and a template syntax object. It binds the bits of the syntax object to the symbols in the pattern, likesyntax-case, and transforms the template into a new syntax object. We're going to match our transformed list to a simple syntax pattern like thus

[(set-value ...)
    (map (λ (item)
        (datum->syntax stx
            (string->symbol (format "set-~a!" (syntax->datum item)))))
        (syntax->list #`(value ...)))]

Note that syntax->list is now acting on #`(value ...), id est, the syntax object of lists of symbols for object member names. Also note that the syntax object used in datum->syntax is the stx object from our macro definition. Our new template, with the new setters, looks like:

#'(define (name value ...)
    (define (set-value new-val)
        (set! value new-val)) ...
    (define func-name lambda-body) ...
    (define (dispatch m)
        (cond [(eq? m 'value) value] ...
            [(eq? m 'func-name) func-name] ...
            [(eq? m 'set-value) set-value] ...
            [else (error "unknown message")]))
    dispatch)

Now let's wrap it all up in our macro definition:

(define-syntax (new-object stx)
    (syntax-case stx ()
        [(_ name (value ...) [(func-name lambda-body) ...])
        (with-syntax ([(set-value ...)
                (map (λ (item)
                    (datum->syntax stx
                        (string->symbol (format "set-~a!" (syntax->datum item)))))
                    (syntax->list #`(value ...)))])
        #'(define (name value ...)
            (define (set-value new-val)
                (set! value new-val)) ...
            (define func-name lambda-body) ...
            (define (dispatch m)
                (cond [(eq? m 'value) value] ...
                    [(eq? m 'func-name) func-name] ...
                    [(eq? m 'set-value) set-value] ...
                    [else (error "unknown message")]))
                dispatch))]))

There you have it, folks. We've implemented a basic object system in less than 20 lines of code.

Comments, questions, corrections all welcome.

CSS3 Transition Effects in My New Website

2011-06-04T15:29:00.000-07:00

Having finished my MSci exams at UCL, I thought I would take this opportunity to "freshen up" the layout on my personal website. There was one nasty, looming problem with this idea, however - I suck at design.

Design work starts off a days-long cycle of me torturing myself. In stage 1, I accept/realize that I am bad designer with few capabilities and resign myself to "keeping it simple". Stage 2 sees me drawing half of a dozen paper sketches and mock-ups for this "simple design". These designs later inundate my hapless friends as I pester them for their opinions. By stage 3, I've accepted that half of my sketches are terrible, and that the other half require an on-screen mockup to judge. Thus I launch the Gimp and try to create a vector mockup, leading back to stage 1: realizing that I don't have the technical prowess to complete this design and resolving to do something simpler. After several days, this process takes its toll on my mental health and produces an unremarkable HTML product.

I went through this cycle this week (after promising myself that "freshening up" would only take a few hours) and settled on a design similar to my Reverse Job App awhile back. But when I began writing the actual HTML I decided I could torture myself a little more and complete the layout without any use of javascript or jquery. Instead, I relied solely on the new CSS3 gradient and transition effects. I decided to write this blog about the latter effect.

The CSS Effect

The front landing page of my has three boxes that link to the different sections of my website like such:

Each box has a neat hover effect where the shaded background drops down and changes color. While this effect is easy to implement in jquery, I was able to implement it in Firefox/Chrome/Safari without using one lick of javascript, and I'll show you how.

The HTML

< a href="http://www.blogger.com/maths" >
< div class="bg" >
<div class="filter" >
< p >maths. < /p >
< /div >
< /div >
< /a >

Two divs in an a tag? That's right. The top a tag is obviously the link. The outer div.bg tag gives us the background images and adjusts the size of the a element (this is possible without the div.bg, but I could only get the full background image to show if I used absolute positioning). The inner div.filter tag and p tag will be the subject of the transition effects. div.filter is a partially transparent box that gives us the shaded background. The p tag is the actual text of the link.

CSS Before Effects

Let's implement the above description.

* {margin:0; padding:0;}
a{
width:180px;
text-decoration: none;
}
a > div.bg {
height:216px;
width:180px;
border: 2px solid black;
background-image: url(maths.png);
}
a > div.bg > div.filter{
background-color: rgba(33, 33, 33, 0.5);
height:217px;
width:180px;
}
a > div.bg > div.filter > p{
display:block;
padding-top:90px;
padding-left:18px;
font-weight:bold;
color: #ffffff;
}

All pretty straightforward. Combining the above should yield a pretty simple website:

Now let's write down the CSS for the hover stage. We'll start by adjusting the above CSS to get an a tag with a darker filter that only covers the top of the image. This means we need to change the css for div.filter and p.

a > div.bg > div.filter
{
background-color:#000000;
background-color: rgba(0, 0, 0, 0.7);
height:145px;
margin-top:72px;
}
a > div.bg > div.filter > p{
display:block;
padding-top:18px;
padding-left:18px;
font-weight:bold;
color: #ffffff;
text-decoration:underline;
}

How do we get a hover effect? That's simple, we can combine these two descriptions using the :hover psuedo class. We'll use the first CSS we wrote, and append the second definitions after changing "a > " to "a:hover >", like so:

* {margin:0; padding:0;}
a{
width:180px;
text-decoration: none;
}
a > div.bg {
height:216px;
width:180px;
border: 2px solid black;
background-image: url(maths.png);
}
a > div.bg > div.filter{
background-color: rgba(33, 33, 33, 0.5);
height:217px;
width:180px;
}
a > div.bg > div.filter > p{
display:block;
padding-top:90px;
padding-left:18px;
font-weight:bold;
color: #ffffff;
}
a:hover > div.bg > div.filter
{
background-color:#000000;
background-color: rgba(0, 0, 0, 0.7);
height:145px;
margin-top:72px;
}
a:hover > div.bg > div.filter > p{
display:block;
padding-top:18px;
padding-left:18px;
font-weight:bold;
color: #ffffff;
text-decoration:underline;
}

CSS Transition Effects

We're nearly there! How do we get those cool transitions? The working draft for CSS3 provides two new attributes:

transition-property: [list of properties]
transition-duration: [number of seconds]

transition-property instructs the browser to do a default (read: linear) transition from the previous properties to the new properties in this definition. transition-duration is just the number of seconds the effect should last. You can read more about them at the Mozilla Development Network.

For div.filter, we're changing the background color, the height, and the margin-top, so we change the hover div.filter definition to:

a:hover > div.bg > div.filter
{
background-color:#000000;
background-color: rgba(0, 0, 0, 0.7);
height:145px;
margin-top:72px;
transition-property: background-color, height,margin-top;
transition-duration: 0.3s;
}

We'll want to add the same thing to the default div.filter definition as well, so we have a smooth transition from hover to non-hover states:

a > div.bg > div.filter{
background-color: rgba(33, 33, 33, 0.5);
height:217px;
width:180px;
transition-property: background-color, height,margin-top;
transition-duration: 0.3s;
}

Similarly, we need to change the padding-top property of the p tag, this is done with:

a:hover > div.bg > div.filter > p{
display:block;
transition-property:padding-top;
transition-duration: 0.3s;
padding-top:18px;
padding-left:18px;
font-weight:bold;
color: #ffffff;
text-decoration:underline;
}

But hold on! Since transition-property, et al, is still experimental, it requires a vender-prefix. E.g. for Firefox, it needs -moz-transition-property, et cetera, for webkit-based browsers, -webkit-, for opera, -o-. So our complete CSS is:

* {margin:0; padding:0;}
a{
width:180px;
text-decoration: none;
}
a > div.bg {
height:216px;
width:180px;
border: 2px solid black;
background-image: url(maths.png);
}

a > div.bg > div.filter
{
float: left;
background-color: transparent;
background-color: rgba(33, 33, 33, 0.5);
height:217px;
width:180px;
-moz-transition-property: background-color, height,margin-top;
-moz-transition-duration: 0.3s;
-webkit-transition-property: background-color, height,margin-top;
-webkit-transition-duration: 0.3s;
-o-transition-property: background-color, height,margin-top;
-o-transition-duration: 0.3s;

}
a > div.bg > div.filter > p{
display:block;
padding-top:90px;
padding-left:18px;
font-weight:bold;
color: #ffffff;
}
a:hover > div.bg > div.filter
{
background-color:#000000;
background-color: rgba(0, 0, 0, 0.7);
height:145px;
margin-top:72px;
-moz-transition-property: background-color, height,margin-top;
-moz-transition-duration: 0.3s;
-webkit-transition-property: background-color, height,margin-top;
-webkit-transition-duration: 0.3s;
-o-transition-property: background-color, height,margin-top;
-o-transition-duration: 0.3s;
}
a:hover > div.bg > div.filter > p{
padding-top:18px;
padding-left:18px;
font-weight:bold;
color: #ffffff;
text-decoration:underline;
-moz-transition-property:padding-top;
-moz-transition-duration: 0.3s;
-webkit-transition-property: padding-top;
-webkit-transition-duration: 0.3s;
-o-transition-property: padding-top;
-o-transition-duration: 0.3s;
}

There you have it, folks! CSS3 transitions for everyone! (except, possibly, IE users.)

Open Sourcing Sneffel

2011-03-06T06:35:00.000-08:00

I've decided to open source Sneffel and other projects.

I started web development in 2001 with an open source browser based strategy game called Solar Empire. The player community was generally excited by my work, as I brought some energy into a long-stalled development process. Except I didn't. I kept all my contributions to myself on my own server. I was 11 years old and was too embarrassed to have these "real" software developers look at and criticize my code.

What a shame. I missed a great opportunity to learn from others. Since then I've worked within various teams, but still largely stick to the habit of programming alone in a box. (This isn't 100% bad. Even at Universal, I spend time doing "exploratory" programming in Scheme.) So in an attempt to rectify years' worth of bad habits, I've decided to start open-sourcing my projects. All of them.

I create a github account (github rocks! I can't believe its taken me this long to sign up!) and am open sourcing my major personal projects. I expect to open source any future personal projects.

I know for the most part no one will care and no one will look at my code anyway. But there's always the chance someone will take a peek and want to contribute. And I'm sure I could learn a lot from said person. At the very least, the thought that someone else may be combing through my code will force me to write better.

You can follow my work at https://github.com/mebassett/

An idea for a new web app...

2011-01-12T16:20:00.000-08:00

I often have ideas for new web apps (though I'm thinking more about the iPad every day, I think I want one...) and I like to jot them down. But I remembered I haven't updated this blog in awhile, so I thought I could store them here.

I've been pretty busy lately with Python programming in data management/financial models for Universal, Scheme/Racket programming for myself, and a whole bunch of maths. I got this idea during a brief pause in my mathematical studies:

Essentially, its another Reddit/Hacker News app. The key difference is that you pick the community to vote on things. The problem I'm trying to solve is identification of key content from blogs and other alternative news sources. (The problem appeared to me in a comment on Hacker News, actually) RSS and other feeds don't do a good job if signalling important content the way the front page of a newspaper does. Social bookmarking communities rely on the "wisdom of the crowd" which doesn't necessarily pick out the best content (E.g. the top item is some link-bait spam-post about 10 things smart people should work on, while an important article about changing political and economic climates goes unnoticed.) This would solve that problem by giving "upvote" rights to people you select, e.g. friends you respect online and off. It could pull links from Twitter feeds you choose, for instance, as well as RSS, or simply whatever your "editors" pull in. Make sense?

Right, back to the maths... Hopefully I'll get a chance to write about my adventures in Racket, Python, HTML5, and iPad apps soon!

Matthew

Classes and prototypes in a new PHP OOP system.

2010-09-22T08:01:00.001-07:00

I found this post on Hacker News and thought I'd re-blog a link to it. Just because I thought it was so cool.

http://dhotson.tumblr.com/post/1167021666/php-object-oriented-programming-reinvented

Learning to build great software

2010-08-25T13:20:00.000-07:00

Whenever I build a new piece of software, I'm nearly always torn in two. One half of me, the side that got me to start work in the first place, says things like "this is great! this app will change the world! everyone will want to use it, you just need to get the word out!". The other side, which doesn't appear appear until after I can see a working prototype, is much less optimistic. He says things like "wow, this sucks. No one is ever going to use it. It's essentially useless. You shouldn't waste any more time on it".

As one might imagine, I have a difficult time deciding which half is right, and whether or not pursue a project further. But today I discovered a great tactic to determine; I've learned exactly how to figure out if my software is great or not:

Pick up the phone and call your ideal users.

Yes. It's that simple. I don't know how I missed it for so long. It seems so obvious now. But until my finances got dire and I was pursuing Sneffel as a business (Rather than just a programming project) full time without other options, I had never thought about it before I spent today finding distance learning schools who might be interested, calling them up, and trying to sell Sneffel to them.

I must say: it was really depressing work.

But I think I've been able to decide if Sneffel is worth pursuing further or not. And as much as it saddens me to say, I think it's time to move on to better projects.

Ouray – The Facebook Connect LaTeX editor Chat Room.

2010-06-12T16:29:00.000-07:00

I have two main personal projects running right now: Sneffel and Ouray (both named after places in Colorado.) The latter started as my 24-hour attempt at making a better “virtual learning environment” than Blackboard, Moodle, or other software I've seen educational institutions use. Inspired by Twitter and Yammer, I thought a micro-blogging format would do the job, and I put together a quick system that used Facebook Connect to allow a single-click sign on.

Of course, this was all before Open Graph and the other changes to Facebook's API. So naturally, when I went back into the code this week to make some changes, I had to see what was up. All I wanted to do was ask for facebook user's email address (I had not done so when they first signed up) so that I could send them notifications for the Ouray courses they've signed up for. Easy, right? Just set the “perms” field in the FBML, like so:

<fb:login-button perms="email" v="2" size="medium">Connect</fb:login-button>

Well, this is where the trouble starts. First, I was using the old PHP API library, and I wanted to ensure I was using the latest code. Second, as a result, I needed to look up documentation for the new API, and it was a bit fragmented.

The first thing I realized is that I needed to use a new authentication cookie, and to get this cookie, I needed to ensure I was using the latest javascript API. So I replaced the old facebook <script> tags and added the new:

<script src="http://connect.facebook.net/en_US/all.js"></script>
<script>
FB.init({appId: 'MYAPPID', status: true, cookie: true, xfbml: true});
FB.Event.subscribe('auth.sessionChange', function(response) {
if (response.session) {
//login
} else {
//logout
}
}); </script>

At the bottom of my pages as instructed. Now the FB.Event code is key to getting your app to respond to logins. It tells javascript to listen for a facebook event and act upon it with the function passed to it. I found two different events used for logins, “auth.login” and "auth.sessionChange", for seemingly no reason. Now the uses of each seem pretty clear from the names, but it would be nice to see the different responses; e.g., can I use sessionChange to detect a login? Yes I can. Did I find that out from Facebook? No I didn't. (see http://developers.facebook.com/docs/reference/javascript/FB.Event.subscribe for a list of FB events.)

The next step came in trying to access this new Facebook Cookie. Facebook's http://developers.facebook.com/docs/authentication/ provides the PHP function

function get_facebook_cookie($app_id, $application_secret)
{
$args = array();
parse_str(trim($_COOKIE['fbs_' . $app_id], '\\"'), $args);
ksort($args); $payload = '';
foreach ($args as $key => $value)
if ($key != 'sig')
$payload .= $key . '=' . $value;
if (md5($payload . $application_secret) != $args['sig'])
return null;
return $args;
}

Which worked quite well, assuming the first part was done right. From here you could grab the cookie data in an array (say $cookie = get_facebook_cookie...) and get going. The key bit about this cookie is that it carries an “access token” which you needed to pass to the Graph API in order to get the data you requested in the fbLogin perms field. Now the examples you see at http://developers.facebook.com/docs/guides/web#personalization would have you think the key is “oauth_access_token”. But the example at the bottom of http://developers.facebook.com/docs/authentication/ says its “access_token”. I was unfortunate enough to not immediately notice the difference, and was left wondering why my cookie didn't have the “oauth_access_token”. After an hour or so of fighting, I finally realized my var_dump contained an “access_token” and was able to catch the problem. Thus I was finally able to use code like:

$user = json_decode(file_get_contents( 'https://graph.facebook.com/me?access_token=' . $cookie['access_token']));

To get my facebook data. Just one final tip – in case anyone else is trying the same thing – don't use file_get_contents. Use curl. Like this:

$ch = curl_init('https://graph.facebook.com/me?access_token=' . $cookie['access_token']);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
$fbResponse = curl_exec($ch);
curl_close($ch);
$userDetails = json_decode($fbResponse);

I was also able to finish the LaTeX editor for Ouray (even transparent png outputs!) Thanks to ImageMagick and a great tutorial from http://www.linuxjournal.com/article/7870. Essentially, I just wrote a PHP class to do the batch scripting.

Enjoy!

more coming soon...

2010-06-07T04:53:00.001-07:00

watch this space.