Clojure on Martin_Trojer

Some thoughts on clojure.spec

Tue, 14 Jun 2016 00:00:00 +0000

Some of the readers of my Beyond Clojure blog series have asked about my opinion on clojure.spec, and if it solves Clojure’s ’type problem’. Implying whether its presence makes me look more favorably on Clojure. Here are some of my thoughts.

Beyond ClojureScript: Elm

Tue, 03 May 2016 00:00:00 +0000

This is a post in the Beyond Clojure blog series, in which a Clojure developer looks at typed languages for web app development. In this episode we look at front-end development in the language Elm.

Front end development targeting web browsers is a ghetto, everybody seems to agree. The core tools at our disposal are the amalgamation of ideas and accidents thrown together without much overall strategy. One positive development in recent years has been the drastic improvement of the JavaScript (JS) engines in popular browsers. They have now gotten so good that it’s a valid option to treat JS as a compilation target. This fact is one of the drivers behind the explosion of JS transpilers, there are now literally hundreds of languages that (either primarily or as an after-though) can compile to JS. These languages range from light syntax improvements (ala CoffeScript) to full blown languages with big runtimes and everything in between.

Beyond Clojure: Haskell

Thu, 21 Apr 2016 00:00:00 +0000

This is a post in the Beyond Clojure blog series, in which a Clojure developer looks at typed languages for web app development. This is by no means a complete survey of the Haskell web development landscape, rather a random collection of thoughts.

If you are interested in typed functional languages one stands taller than the rest. It’s impossible not to get sucked into the Haskell vortex, but why fight it? In spite of its reputation of being extremely hard to learn and even harder to master, there are several excellent resources out there and you are guaranteed to learn lots of very valuable lessons.

Beyond Clojure: Prelude

Tue, 19 Apr 2016 00:00:00 +0000

Here we are, after five years of learning and later doing Clojure full time, I’ve come to the point where I am seriously looking around for alternatives. I’ve gotten very comfortable working in Clojure, and it has and will continue to serve me very well. But getting comfortable has a flip-side, you stop caring.

Datomic Do's and Don'ts

Wed, 03 Jun 2015 00:00:00 +0000

I recently tried to use Datomic in anger in a project. Here are some things I learned.

Clojure and Emacs without Cider redux

Sat, 14 Feb 2015 00:00:00 +0000

It’s been a couple of months since I’ve stopped using Cider for Clojure development in Emacs. I find a simple ‘inferior lisp’ setup faster and more reliable. For a good summary of why one would consider not using Cider, see Luke VanderHart’s excellent summary.

Developing Clojure in the Cloud

Thu, 04 Dec 2014 00:00:00 +0000

Recently, I gave a talk at the Clojure eXchange 2014 titled ‘Developing Clojure in the Cloud’. I described a way of creating and using (Clojure) development environments inside VMs. I’ve been developing like this for the last year (spanning 2 projects).

Kebab-case keywords in nested Clojure data structures

Sat, 18 Oct 2014 00:00:00 +0000

Clojure and Emacs without Cider

Thu, 02 Oct 2014 00:00:00 +0000

I’ve been hacking Clojure for many years now, and I’ve been happy to rekindle my love for Emacs. The Clojure/Emacs toolchain has come a long way during this time: swank-clojure, nREPL, nrepl.el, and now Cider. The feature list is ever-growing, and every time you look, there are some new awesome shortcuts that will ‘make your day’.

Book Review: Clojure Cookbook

Sun, 06 Apr 2014 00:00:00 +0000

The Clojure Cookbook is part of the O’Reilly cookbook series. I’d describe this format as a ‘curated wiki in print’. The wiki analogy is especially true for this volume since its contents were contributed by some 60 different developers. It’s packed with small, bite-sized recipes for solving common problems in Clojure. This is useful for developers across the entire spectrum from beginner to expert.

The content is organized into 11 chapters, each containing a number of recipes. The chapter layout is clear and serves its purpose when looking for content. The book covers a lot of ground, from working with primitives and basic data structures to dealing with databases, writing web apps, and running Hadoop jobs. Each recipe comes with code and/or REPL examples, so it’s very easy and enlightening to ‘play along’. In contrast to other books with code snippets, the authors have made all recipes self-contained (no other projects/files need to be created and run for the examples to work), which makes it very easy to dive in at any point in the book.

The Clojure REPL; a blessing and a curse

Sat, 05 Apr 2014 00:00:00 +0000

All Clojure developers swear by their REPL; it’s one of the most powerful tools in our arsenal. Coming from traditional edit/compile/launch languages, it is also a great productivity boost. The Clojure community takes non-AOT (ahead of time compilation) to the extreme. By default, we ship Clojure source code in our development and production jars and thus leave compilation to the very last minute (when the program launches). This gives us lots of power and flexibility; if you’ve ever navigated into a library in Emacs and fixed a bug, re-evaled the form and carried on working, you know what I’m talking about.

Working with core.async: Blocking calls

Sun, 09 Mar 2014 00:00:00 +0000

You can’t do anything even remotely blocking inside go-blocks. This is because all the core.async go blocks share a single thread pool, with a very limited number of threads (go blocks are supposed to be CPU bound). So if you have hundreds/thousands of go blocks running concurrently, just having a few (a handful really) block – all go blocks will stop! For a more in-depth explanation see this previous post.

Working with core.async: Chaining go blocks

Sun, 09 Mar 2014 00:00:00 +0000

One particularly annoying difference between core.async and Go is that you can’t wrap function calls with the go macro. This is due to implementation details of core.async, which can only see the body ‘inside’ the macro and not the functions it may call. This is obviously not a problem if the called function doesn’t interact with any channels, but if it does, then you might be in trouble. I’ve touched on this subject in a previous post.

Working with core.async: Exceptions in go blocks

Sun, 09 Mar 2014 00:00:00 +0000

Dealing with exceptions in go blocks/threads is different from normal Clojure core. This gotcha is very common when moving your code into core.async go blocks – all your exceptions are gone! Since the body of a go block is run on a thread pool, there’s not much we can do with an exception, thus core.async will just eat them and close the channel. That’s what happened in the second snippet in this post. The nil result occurs because the channel we read from is closed.

Testing an API with Simulant

Sun, 29 Sep 2013 00:00:00 +0000

tl;dr I present a new Simulant example project, testing a simple web API.

All programs are simulation tested, at least once. — Stu Halloway

Simulation testing is an interesting field that has a lot going for it. While most website/API developers write many tests at the unit level (an old Rails habit), testing and understanding a whole system is often not done. Most systems we build nowadays consist of many (mega/micro) services/databases glued together. Getting a deeper understanding of how a system actually works and being able to simulate production-like scenarios is very useful.

Retrofitting the Reloaded pattern into Clojure projects

Sat, 07 Sep 2013 00:00:00 +0000

Stuart Sierra has done a great job with clojure.tools.namespace and the reloaded Leiningen template. If you haven’t heard about this before, please have a look at c.t.n’s readme and watch this presentation.

I have retrofitted this pattern into two rather large Clojure projects (20,000 and 5,000 lines) with several modules, and here are some of my findings.

Removing global state

The first step is to find all resources that need to be “lifecycled.” Typical examples are Jetty servers, database/message bus clients, etc. It’s common that these resources are in a (defonce server (atom ...)) form. I tend to grep for defonce and atom to find these items.

Non tail-recursive functions in core.async go blocks

Wed, 17 Jul 2013 00:00:00 +0000

I’ve been using various Go examples and tutorials to take a deeper look into core.async. The CSP pattern is very interesting and powerful; it’s a good move for Clojure to “throw in” with Go and push this style of programming.

core.async works at the s-expression level, where some other JVM solutions (Kilim, Pulsar) do the same at the byte-code level. The main benefit of doing these transforms at the s-expression level is that they are applicable to ClojureScript, where CSP can be a very neat way out of callback hell. David has written about this.

core.async and Blocking IO

Sun, 07 Jul 2013 00:00:00 +0000

Some time ago I wrote about Asynchronous workflows in Clojure. With the recent release and excitement of core.async, I thought it a good time to revisit that post.

While there are already some good examples and comparison-with-go posts out there, I’d like to focus on an area often misunderstood, namely async frameworks and blocking APIs (most commonly blocking IO). It’s important to understand the implications of blocking IO and its effects on ‘async code’, in this case core.async.

Datomic Bootstrap

Fri, 05 Jul 2013 00:00:00 +0000

A simple SQL scenario;

Comparing FP REPL Sessions

Tue, 04 Jun 2013 00:00:00 +0000

Functional programming is great; higher-order functions, closures, immutable data-structures, lazy sequences etc.

Most languages comes with a REPL (or ‘interactive’ prompt), where you can play with these features at your leisure. Dynamically typed languages are a bit more convenient in the REPL, but not by as much as you might think. Also, F# type providers closes the gap even further.

Here’s a typical, hit-a-JSON-endpoint-and-look-at-the-data session in Clojure;

$ lein repl

user=> (def res (slurp "http://www.bbc.co.uk/tv/programmes/genres/drama/scifiandfantasy/schedules/upcoming.json"))

user=> (require 'clojure.data.json 'clojure.walk)
user=> (def json (->> res clojure.data.json/read-str
 clojure.walk/keywordize-keys))

user=> (->> json :broadcasts (filter #(>= (:duration %) 6300))
 (map :programme) (map (juxt :title :pid)))
(["Lady in the Water" "b00l5wdn"] ["Lady in the Water" "b00l5wdn"] ["Lady in the Water" "b00l5wdn"] ["Lady in the Water" "b00l5wdn"])

Nice, clean and very powerful, virtually zero ceremony. Doing the same in Scala, is just a little bit more awkward;

Scala and me

Sat, 01 Jun 2013 00:00:00 +0000

This epic journey (yeah right) began at university with discovering the mighty SICP, still the best book on programming I’ve read (and let’s face it, the best I will ever read). After that profound experience, I kept an eye on the Lisp/FP world and wrote some toys in Scheme, ELisp, and OCaml every now and then. One thing that dawned on me was that none of these languages had much practical use; they weren’t very applicable to real-world software problems. While very clever and mind-expanding, they seemed mainly an academic exercise. There were zero jobs out there using these languages. Heck, hardly any of my peers had heard of or cared about them.

Clojure Hacking on the Samsung ARM Chromebook

Sat, 06 Apr 2013 00:00:00 +0000

I recently switched to the Samsung ARM Chromebook for all my laptop needs. The pitch is quite appealing: £200, dual-core ARM Cortex-A15s, good keyboard, totally fanless (CPU is passively cooled), good battery life, and 1kg weight. The one downside is its quite limited RAM size, just 2GB. But with a decent swap file, I’m running multiple JVMs (with Datomic, Elasticsearch, CLJS compiler, etc.) without any hiccups.

Out of the box, it runs ChromeOS, which I have to say is pretty stellar for browsing and consuming content on the web. It is also possible to run any Linux distribution you wish on this machine; Google has an outspoken strategy to enable it.

Flexible multi consumer/producer pipelines

Sat, 16 Mar 2013 00:00:00 +0000

TL;DR: Pipejine - a lightweight Clojure library for multi-threaded producer/consumer pipelines supporting arbitrary DAG topologies.

Recently, a colleague and I faced a problem where we needed to optimize the total running time of a complicated calculation. This calculation involved several asynchronous steps getting data from other systems (like Elasticsearch and other home-grown services), along with some number crunching and tallying up the results at the end. Here is a simplified example of the system:

The M Word

Sat, 24 Nov 2012 00:00:00 +0000

Some core.logic graph code

Thu, 27 Sep 2012 00:00:00 +0000

Some more Datalog

Tue, 28 Aug 2012 00:00:00 +0000

I’ve written about Datalog and Datomic recently. To conclude, here’s another post comparing execution speed with the contrib.datalog library by Jeffrey Straszheim. Clojure 1.4-ready source can be found here.

The example I’m using in my benchmark is a simple join between two relations. In Datomic/Datalog, it would look like this:

(q '[:find ?first ?height
 :in $a $b
 :where [$a ?last ?first ?email] [$b ?email ?height]]
 [["Doe" "John" "jdoe@example.com"]
 ["Doe" "Jane" "jane@example.com"]]
 [["jane@example.com" 73]
 ["jdoe@example.com" 71]])
;; #<HashSet [["Jane" 73], ["John" 71]]>

In contrib.datalog, the same query requires more ceremony. You can write it like this:

cKanren time!

Sun, 12 Aug 2012 00:00:00 +0000

Mr. David Nolen recently published core.logic 0.8.alpha2, with added cKanren (c for constraints) support. To celebrate this glorious event, I’m writing up some core.logic/cKanren stuff I’ve been looking at recently.

Enter the Queens

If you’ve followed this blog, you’ve perhaps seen my previous posts on solving N-Queens in core.logic (part 1 and part 2). How will this look and perform using the new shiny cKanren extensions in core.logic 0.8? Obviously, there are many (new) ways to solve this problem. Here’s a core.logic-styled version of the solution described in the cKanren paper (please read paragraph 4.2 for an in-depth explanation):

Untying the Recursive Knot

Mon, 23 Jul 2012 00:00:00 +0000

Here I present a couple of examples of the functional design pattern “untying the recursive knot.” I’ve found this useful on several occasions, for instance, when breaking apart mutually recursive functions. This material was inspired by Jon Harrop’s excellent Visual F# for Technical Computing.

First, let’s look at a simple factorial implementation using direct recursion:

(defn fact [n]
 (if (= n 0) 1
 (* n (fact (dec n)))))

We can break the direct recursive dependency by replacing the recursive calls with calls to a function argument:

Replicating Datomic/Datalog queries with core.logic, take 2

Tue, 17 Jul 2012 00:00:00 +0000

This is a follow-up to my previous post on datalog-equivalent queries in core.logic.

Here I present an alternate way to do the unification and join inside core.logic (without having to use clojure.set/join). It uses the relationships/facts API in core.logic, described here. First, let’s consider this Datomic query:

(q '[:find ?first ?height
 :in [[?last ?first ?email]] [[?email ?height]]]
 [["Doe" "John" "jdoe@example.com"]
 ["Doe" "Jane" "jane@example.com"]]
 [["jane@example.com" 73]
 ["jdoe@example.com" 71]])
;; #<HashSet [["Jane" 73], ["John" 71]]>

In core.logic we start by defining the relationships between our 2 datasets:

Replicating Datomic/Datalog queries with core.logic

Mon, 16 Jul 2012 00:00:00 +0000

I’ve been toying with Datomic recently, and I particularly like the power of its query language (~Datalog). Mr. Halloway showed a couple of months ago how the query engine is generic enough to be run on standard Clojure collections, gist here. Here is an example from that page of a simple join;

(q '[:find ?first ?height
 :in [[?last ?first ?email]] [[?email ?height]]]
 [["Doe" "John" "jdoe@example.com"]
 ["Doe" "Jane" "jane@example.com"]]
 [["jane@example.com" 73]
 ["jdoe@example.com" 71]])
;; #<HashSet [["Jane" 73], ["John" 71]]>

A question you might ask yourself is how can you use core.logic to do the same kind of queries? It turns out that it’s pretty straightforward, and also very fast. Core.logic provides some convenient helper functions for unification that we are going to use. Here’s an example of how to get a binding map for some logical variables over a collection;

N Queens with core.logic, take 2

Wed, 11 Jul 2012 00:00:00 +0000

This post is a follow-up to my previous post on N Queens and core.logic, in which I tried to find solutions using “pure” logic (without arithmetic goals) and basic miniKanren/Reasoned Schemer building blocks.

After excellent feedback and hints from David Nolen (big thanks), I present a greatly simplified (and faster) way of using core.logic to find all solutions. Credit also goes to good old Bratko.

First, let’s fix the safeo function (and def-subo macro). In miniKanren, you can use arithmetic goals given two prerequisites: the fresh variable must be bound to a finite (number) space, and we must use project to bind the values. This means we can get rid of subo altogether.

N Queens with core.logic, take 1

Sat, 07 Jul 2012 00:00:00 +0000

I’ve been “hammock-reading” the excellent Reasoned Schemer book these last couple of months, on my quest to develop a gut feel for when logic programming, as defined by miniKanren/core.logic, is applicable.

My first attempt is to apply it to a problem where (as it turns out) miniKanren isn’t a good fit: n-queens. What you really need for this problem in the logical programming world is something called constraint logic programming (CLP), which is implemented (for example) in cKanren. The good people at core.logic are working on integrating CLP and cKanren in core.logic in version 0.8, so I intend to revisit this problem as that work progresses.

Some thoughts on logging

Tue, 03 Jul 2012 00:00:00 +0000

Have you ever tried to log from a multi-threaded program? Have you tried to make sense of log output when multiple subsystems were logging to it? Have you tried to do average latency calculations based on that log file?

If you’re reading this blog, I’m guessing you answered yes to a couple of the questions above.

There are multiple problems here: multiple producers (race conditions), out-of-order logs, conflated subsystems in the same logs, etc. You have to put a lot of effort into your log post-processor to make any sense of the data, decorating it with various metadata to make it possible at all.

Distributed Actors in Clojure

Thu, 17 May 2012 00:00:00 +0000

Here’s another post on a topic that has been discussed since the dawn-of-time: is there a nice and idiomatic way to write Erlang/Actor-style distributed programs in Clojure? There have certainly been a few attempts, but Rich’s post (above) still holds true today.

First, some clarification: I am not primarily thinking about number-crunching, map/reduce-style stuff, where Clojure has a pretty good story:

Akka and the Erlang legacy

I am trying to write programs that solve problems in the areas where Erlang typically excels, such as:

Enumerate N Queens solutions

Sun, 25 Mar 2012 00:00:00 +0000

Adding Live Unit Feeds to Frinj

Wed, 14 Mar 2012 00:00:00 +0000

A couple of weeks have passed since I pushed Frinj to github and blogged/tweeted about it. The response has been pretty awesome, one highlight being when @stuarthalloway showed me a frinj+datomic example gist on the #datomic IRC channel. In short, the Clojure community is #badass.

Frinj comes with a big database of units and conversion factors, and while many conversion factors are “eternal”, others aren’t. Exchange rates, for instance, have to be kept up to date to be relevant. The Frinj unit database was designed to be updatable, both for usability when doing various calculations and for rates that constantly change. This is the reason the frinj.calc namespace exposes the (frinj-init!) function to reset the unit database to a known baseline (in case you write over some factors, etc.). Clojure’s support for atomically updating state is ideal for this purpose; the calculator’s state is kept in a number of refs and, thanks to the STM, always kept consistent.

Announcing Frinj, a practical unit of measure calculator DSL for Clojure

Sat, 03 Mar 2012 00:00:00 +0000

I am proud to announce a new Clojure project called “Frinj”.

Frinj is a practical unit-of-measure calculator DSL for Clojure.

Key features:

Tracks units of measure through all calculations allowing you to mix units of measure transparently
Comes with a huge database of units and conversion factors
Inspired by the Frink project
Tries to combine Frink’s fluent calculation style with idiomatic Clojure

Full source code available on github.

To whet your appetite, head straight over to the sample calculations page to see what Frinj can do!

ASCII Mandelbrot Set

Tue, 21 Feb 2012 00:00:00 +0000

Some thoughts on Clojure performance

Sat, 18 Feb 2012 00:00:00 +0000

Edit: This post recently re-surfaced on hacker news and caused a bit of a stir, mainly because of a slightly sensational/misleading title (was “Why is Clojure so slow?”). I wrote this before Rich Hickey’s Clojure/Conj 2011 keynote was published, in which he talks about most of my concerns (and outlines possible solutions).

Clojure is great in many ways, but one thing it can’t be accused of is being particularly fast. What I mean by fast here is the speed at which Clojure programs execute. This is a well-known issue in the Clojure community and has been discussed on the mailing list and Stack Overflow.

Scheme as an external DSL in Clojure

Sat, 28 Jan 2012 00:00:00 +0000

This is a follow-up post to my previous “Scheme in Clojure” post.

This time we implement a Scheme interpreter as an external DSL. This means that we consider the DSL as completely foreign to the host language, so we need to write our own parser (or reader as it’s called in Clojure) and interpreter. I have to admit that this is a bit of an academic exercise because the internal DSL version I wrote about previously is both smaller (less code) and faster (as fast as any other Clojure code). However, this can serve as an example of how to write parsers in Clojure and it also highlights how elegant and succinct such a parser/interpreter can be. And of course, it’s pretty darn fun :-)

Asynchronous workflows in Clojure

Thu, 22 Dec 2011 00:00:00 +0000

Asynchronous workflows is a very powerful feature of F#, and recently I wanted to explore the state of the JVM and in particular Clojure when it comes to replicate the functionality. In this post I’ll share some of my findings and I’ll include some background material to explain the problems.

Let’s start with an example of a webclient using “async” in F#.

let download url = async {
 let request = HttpWebRequest.Create(Uri(url))
 let! response = request.AsyncGetResponse()
 use stream = response.GetResponseStream()
 let! res = asyncReadToEnd stream
 return res
}

The magic here is that you can write continuation-style code in a sequential manner. This combines the scalability of asynchronous programs with the readability of sequential code. So, what lessons can we learn from this code and how would we do this with the JVM and Clojure? First of all, this is not the same as using futures over blocking calls;

Scheme as an embedded DSL in Clojure

Tue, 29 Nov 2011 00:00:00 +0000

If you give someone Fortran, he has Fortran.

If you give someone Lisp, he has any language he pleases.

– Guy Steele

Replace Fortran with whatever language you are currently using, and the quote still holds true today. Lisp has been around for a long time, and it’s built-in flexibility is still unmatched by other languages. In this post we will look at key Lisp concepts such as code-is-data and powerful macro semantics.

Tail Calls in F#, Clojure and Scala

Sun, 20 Nov 2011 00:00:00 +0000

I recently looked into Tail Call Optimisation/Elimination (TCO) and the implications for 3 modern languages, namely F#, Clojure and Scala. In this post I share my findings. If you’re new to the subject or just looking into some of these languages I hope this post can be of some use to you. I will mix code snippets in the 3 languages freely (and without warning! :)

TCO is a well documented topic in books and articles about functional programming, and the TCO in .NET and the lack thereof in the JVM has been debated “to death” on various programmers’ boards. I don’t intend to add any fuel to the fire here, rather some background and practical implications.