Waking up at 6:30 AM has bestowed upon me a blissful, irritant-free period that is perfect for contemplation, so of course I used it for a few experiments instead.
With all the hoopla around Go concurrency, a surprisingly large number of folk have been pursuing similar avenues for writing nice, clean concurrent code, and analogues have sprung up all over the place – like Clojure‘s
core.async, for instance, which is around a year old and which I’m quite fond of.
In Python land, one of the available options is Rob Galanakis’ goless, which I’ve been tracking fairly closely. It provides Go-like primitives and works wonderfully out-of-the-box with PyPy, providing in-process coroutines with pretty much zero hassle.
As it happens, PyPy is now my default Python interpreter1, so
goless is a very nice addition to my toolkit, all the more so now that I’m starting to benchmark and profile most of my code as a matter of course – partly to figure out where the bottlenecks are, and partly to figure out ways to make it go faster2.
And it’s been quite entertaining, really – I’ve found a few interesting things here and there, and profile results from even trivial bits of code is quite informative.
; A straight port of the goless library example (runs fine with the PyPy backend) (import [goless [go chan]] [cProfile [Profile]] [pstats [Stats]] [functools [partial]]) (defn produce [msgs done count] (for [i (xrange count)] (.send msgs i)) (.send done)) (defn consume [msgs out name] (for [msg msgs] (.send out (% "%s:%s" (, name msg))))) (defn logger [out] (for [msg out] (print msg))) (let [[p (Profile)] [done (chan)] [msgs (chan)] [out (chan)]] ; enable profiler (.enable p) ; start a producer, three consumers and a logger (go produce msgs done 10000) (map (partial go consume msgs out) ["one" "two" "three"]) (go logger out) ; wait for completion, stop profiler and dump stats (.recv done) ; stop profiler (.disable p) (.dump_stats (Stats p) "out.pstats"))
Next up, applying this to a number of things I’m building around an MQTT broker…