Green threads and async programming
Not too long ago, another lisper, Stephen Goss, wrote a green threads library on top of cl-cont. It was recently converted to use cl-async’s futures, meaning it can now be easily operated with any drivers built on the same futures platform. It’s now possible to write async code without using CPS. Let’s dive in for a few examples!
Let’s create a synchronous delay (mimicking
sleep). Note that
are the cl-async-future and green-threads package nicknames, respectively:
;; we *must* use defun/cc or without-call/cc anywhere gt:wait-on is used, ;; otherwise wait-on will not function properly. (cl-cont:defun/cc fdelay (value &key (time 1)) (let ((future (asf:make-future))) (as:delay (lambda () (finish future value)) :time time) ;; this call blocks this green thread (assuming it's wrapped in one of ;; cl-cont's macros) while allowing the event loop to process more events. ;; once the future is finished, this green thread continues execution and ;; returns the values the future was finished with. (gt:wait-on future))) (as:start-event-loop (lambda () ;; start a green thread (gt:with-green-thread (format t "Start.~%") (format t "End: ~a.~%" (fdelay 5)))))
In the above example, our fdelay function creates a future, spawns a delay to
finish it at the given time, then yields the current green thread by using the
wait-on takes a future, and once that future is finished,
continues execution of the thread, returning the values the future was finished
Start. ;; 1s delay End: 5
Note how where we’d normally have to put the second
(format) call into a
callback function, we can now keep it in the same stack as the first
In other words, by using green-threads along with cl-cont, we’ve eliminated CPS!
As noted in the comments, it’s important to mention that any time
called, it must be wrapped in a cl-cont macro:
with-green-thread both do this for you. This macro transforms the code that
appears to be in the same stack (like the
(format ...) statements in the
example) into CPS. So we’re not creating “real” stacks/threads and yielding
them, we’re using cl-cont to make it appear as though we are, and green-threads
is very tactfully managing all the state for us.
We can do more than just delay though, we can use a library like drakma-async to “synchronously” grab a webpage:
(as:start-event-loop (lambda () (gt:with-green-thread (format t "Google's homepage: ~a~%" (gt:wait-on (drakma-async:http-request "http://google.com"))))))
Here, we asynchronously grab a web page (cl-async continues to process events
while we’re waiting on
http-request), and pass the result to
format as if
it’s a synchronous call.
With the absence of real continuations, coroutines, and green threads in lisp, it can be very difficult to give asynchronous programming a natural syntax, because by nature it’s all CPS. Using cl-cont and green-threads shrinks this gap significantly.
It’s important to note the limitations of this method. The main one is error and
condition handling. What looks like a normal stack is actually being transformed
into CPS, so catching and handling errors becomes suddenly more difficult. On
top of this, the future-handler-case
macro won’t do much when using green-threads'
wait-on because it doesn’t know
to look for it. It might make sense to make
wait-on a macro that expands to an
attach call so that errors can be handled more effectively.
Hopefully you now have a better understanding of how green-threads is a way to transform ugly CPS-style code into normal stack-based code. It’s not a perfect solution to the age-old problem, but it works well and, in my opinion, is a pretty badass project worth trying out…and because it now integrates with cl-async-future, it can be used very easily alongside any of your async programming.