Nov 15, 2012

Adding syntax around futures

After a lengthy discussion about how sucky CPS is, Paul Khuong (pkhuong) showed me a syntactic abstraction over futures that almost makes async programming (with CPS) into real, stack-based programming. Note that the macros built into the future system and detailed below are close (but not exact) implementations of Paul’s syntax abstractions. So, mad props.

There are a few things happening here. A future is a representation of a value that will be available in the future. A future can have a callback attached. The process of attaching a callback to a future returns another future, which is finished with the return value(s) of the callback that was just attached. A future that is finished with another future as the value binds its callbacks to the new future without firing them. What this means is that you can have many layers deep of CPS going on, but the final value will be available to the top-level via a future.

Note that this is all explained (hopefully in better depth) in the futures doc page. Let’s go over a few examples so we can see how this all fits together. Please note the following examples assume you are in a package that’s :useing the cl-async-future package.

The following function will be used throughout the examples below:

;; first, let's define a function that runs asynchronously and returns a future
(defun future-gen (x)
  "Wait one second (async) and then finish the returned future with value x+1."
  (let ((future (make-future)))
    (delay (lambda () (finish future (+ x 1)))
              :time 1)
    future))

Now that that’s out of the way, let’s run over some quick future usage examples. Here, we attach a callback to the future so we can get the value. Simple enough.

(attach (future-gen 4)
  (lambda (x)
    (format t "Value: ~a~%" x)))

This should print out Value: 5.

Wicked. Now, for a demonstration of nesting futures. Here, several futures are calculated in sequence and the final result is returned as a value. Remember that the future returned from attach is finished with the return value from the callback it was attached with. What this means is that the callback attached to the top-level future is being reattached level by level until it reaches tha computed value:

(let ((future (attach (future-gen 0)
                (lambda (x)
                  (attach (future-gen x)
                    (lambda (x)
                      (attach (future-gen x)
                        (lambda (x)
                          (* x 5)))))))))
  (attach future
    (lambda (x)
      (format t "X is: ~a~%" x))))

The output is X is: 15. No doubt.

Now that you are an expert in cl-async futures, let’s look at some of the syntactic abstraction the above affords us. First, we’re going to look at cl-async’s alet and alet*, which act like let and let* respectively:

;; alet example. bindings happen in parallel, and body is run once all bindings
;; are calculated. alet returns a future (of course) which is finished with the
;; return value(s) of the body form
(alet ((x (get-x-from-server))
       (y (get-y-from-server)))
  (format t "x + y = ~a~%" (+ x y)))

;; alet* example. bindings happen in sequence, and body is run once all bindings
;; are calculated. returns a future that is finished with the value(s) of the
;; body form
(alet* ((uid (lookup-my-user-id-from-server-lol))
        (name (get-user-name-from-server uid)))
  (format t "I know that you and ~a were planning to disconnect me, and I'm afraid that's something I cannot allow to happen." name))

Note in the alet* example, the name binding uses the uid value to do its lookup.

Let’s go over one more example, multiple-future-bind, which is the multiple-value-bind of THE FUTURE:

(multiple-future-bind (uid name)
    (get-user-from-server)  ; returns a future
  (format t "hai, ~a, your id is ~a.~%" name uid))

Pretty simple.

Note on alet / alet* / multiple-future-bind

The binding forms for these macros do not have to return futures. They can return normal value(s) and those value(s) will just be used for the binding(s). For instance, these forms will work fine:

(alet* ((x (get-x-from-server))
        (y (+ x 5)))
  (format t "x,y: ~a,~a~%" x y))

(multiple-future-bind (name num-friends)
    (values "andrew" 0)
  (format t "~a has ~a friends.~%" name num-friends))

Event handling

Although futures are capable of handling events, the syntactic abstractions above give little help in this area, but keep in mind that with a good global event handler in *default-event-handler*, this won’t be a problem. Also note that event handlers are passed down from future to future along with callbacks when reassigning, meaning you may only have to set a handler on the first future and it will propagate down.

I may write code-walking macros that mimick handler-case, but I also may not. It really just depends on the work to reward ratio.

UPDATE: See the post on error handling with futures. Specifically, check out future-handler-case, which provides handler-case like error handling for asynchronous operations involving futures.