Get Inside Unbounce

Subscribe

Our curvy road to Clojure

curvy_roadThe Clojure language is spreading like wildfire at Unbounce. We just had three people attending Clojure West and being indoctrinated with the latest and greatest news about this language and its ecosystem. So we’re very serious about it! But this commitment didn’t happen overnight: it took us a while and some experiments (at times painful) to come to the conclusion that Clojure was the right tool for us.

Historically Unbounce started as a Ruby and Java shop, but it would be a mistake to reduce us to these two languages. We are all polyglot programmers at heart, and very curious ones on top that! Though there are some benefits in having two heterogeneous platforms and environments (like for example from a security standpoint), it also has drawbacks: when nothing can be shared between platforms, everything has to be done twice in two different ways, and this from coding to deployment.

vertx_blackSo we’ve been looking for ways to reduce this divide and increase collaboration and sharing across our teams. To that end, we’ve explored using Vert.x as a common platform, since it is a polyglot platform on the JVM, with many language bindings (including Ruby). The module system of Vert.x 2 was particularly attracting to us, as it allows composing applications from multiple independent modules written in any languages and communicating with each other via an event bus (thus low coupling). Vert.x clean model for worker components was also very attractive to us, as it solves one of the common headaches with Ruby applications. Finally the fact that Vert.x has all its networking built on top of Netty was also a major decision point for us, since we had a great experience building our new Page Server on this solid library.

Unfortunately, after months of efforts (leading to one application in production and several in development), this approach didn’t work out for us and failed to deliver what we were expecting from it. Let me make it very clear that this was not a Vert.x issue per se, but more a problem with its ancillary tools:

  • babel-crashing-downDespite ruby-maven’s heroic efforts to give a Ruby flavour to the Java tool chain, the development process was slow and cumbersome. Even after hiding most of the build complexity in a Maven parent POM, our developers had to go through long build processes to bring all the necessary Gems, JARs and modules in order to prepare the application for startup, which was very frustrating.
  • Some Gems were not thread safe and, thus, were causing issues when used in the multi-threaded context of the JVM. We’ve started to look at JAR equivalents of these Gems, since jRuby has a great interoperability with Java, and had some positive results but at the expense of making the development experience less familiar for Ruby developers.
  • jRuby performances were pretty bad, consistently one order of magnitude slower than MRI or the equivalent Java implementation. We’ve tried the recommended performance tweaks but to no avail. It’s very possible that with more efforts or maybe by using the upcoming version of jRuby, these problems could have been solved, but for us that was the last nail in the coffin for this attempt.

At this point, we were still very keen on using Vert.x so we’ve started to investigate other languages it supports, trying to find one that could appeal to both our Ruby and Java developers. Initially we thought that Scala was the most promising language because of its dual nature (object oriented and functional): our Ruby and Java developers would not have to give up on objects while gaining the benefits of functional programming.

Glorious day!

But then something happened, something as beautiful as a celestial shower of rainbow-coloured parenthesis: our Rubyists expressed a strong interest in Clojure.

It was a surprise: letting go of objects was an option? But what a great surprise! Indeed, Clojure bindings for Vert.x are excellent. So we tried porting a Vert.x Ruby application to its Clojure equivalent and the results were great: much less code, a cleaner application structure and significantly improved performances.

But then something else happened. In the process of migrating to Clojure on Vert.x, we realized the following:

  • We were not using much of Vert.x core features,
  • We were actually fighting with the way Clojure verticles were loaded and isolated,
  • We were (again) suffering from using the Java toolchain for building a Clojure application as a Vert.x module.

clojure-iconTherefore, we ended up taking the drastic decision of letting go of Vert.x while keeping Clojure. For us, the wins with Clojure are plenty:

  • The language is concise but explicit (no AOP magic nor monkey patch madness), functional and immutable, and comes with perfectly crafted core libraries – simple made easy!
  • Access to a lot of excellent open source libraries and all the Java ecosystem as well, thanks to a great interoperability with Java,
  • We can re-use our common code library that is used extensively by our Java and Scala projects: having access to our own battle hardened code is invaluable,
  • Excellent tooling, like the Leiningen build tool, and editor support for everyone (Emacs, vim, IntelliJ and Eclipse).

So were does all this leave us? In a pretty great position, considering:

  • Clojure wrappers around our common Java code have been written for the major features we needed (configuration loading, logging, error reporting, AWS integration…),
  • We’ve open sourced a few Clojure projects,
  • One Clojure application has made its way to production and is already dealing with all our public API traffic. More applications are coming soon!

Our road to Clojure was curvy but, in hindsight, we couldn’t have come to this ending without having first given our best shot at a polyglot platform. The reality of unified polyglot development is that it’s hard because programming involves way more than just a language: build tools, libraries, application design and architecture are hard, if not impossible, to harmonize. Picking up a single language and its ecosystem is a more pragmatic approach, and the one that will lead to the best development experience.

We’ve recently set afoot our initiative towards a microservice architecture. Though it won’t be the only tool in our toolbox, Clojure play a prominent role at Unbounce in this effort to rebuild our infrastructure in a more modular and less coupled fashion.

David Dossot
Director of Software Architecture

  • Carl Schmidt

    It took me a while to overcome the painful memories of my Lisp days back in university and the prejudices built up after many years of OOP, but I’m starting to understand the appeal and practicality behind functional approaches. Looking forward to seeing how this plays out for us!

    • Dylan Paris

      Once you’ve gotten used to solving problems in an immutable, functional way, it’s jarring to go back. Any time I have to work with our non-Clojure codebases, I get stressed out! So much mutation!

      Great talking with you at Clojure/West, Carl. Best of luck to you and your team!

  • Eric MacAdie

    Just out of curiosity, is there any reason you never looked at Groovy? It seems like it would be a good fit for companies that use both Java and Ruby.

    • http://david.dossot.net David Dossot

      In their pre-Unbounce lives, several of us have had experience with Groovy, from short embedded scripts to full blown Grails apps ; thus we have a good grasp of the language.

      There’s no doubt that Groovy has some very strong points, including closures, progressive typing, null-safe property navigation and tons of smart extensions to the JDK. But the fact the language is mutable by default and supports meta-programming (expando meta-class), has the capacity to complicate things. If you’ve worked on a non-trivial Grails application, I’m sure sure you know what I mean.

      Moreover, from a pure perception standpoint, despite 12 years of existence, Groovy seems to have stayed a niche language. Its tumultuous past and uncertain future, doesn’t help deciding to bet the house on it. The addition of closures and streaming API to Java, the rise of Rx, makes a massive shift towards Groovy less likely in the future.

      Considering all of these, Clojure seemed a safer choice. There’s a strong language theory behind it. Its community is vibrant, opinionated and pragmatic, a rare combination in our industry.

      We’ll see in the future if our bet was right or not. In the meantime, what a great ride :)

      • orubel

        Yeah totally. Netflix, LinkedIn, Target, Disney, etc… all niche companies in the niche fortune 500 using a niche language. Huh?!!!

        No seriously though… when Scala or Clojure can solve for the API architectural cross cutting concern, call me. Until then, I’ll keep solving that issue on Spring and Grails ;)

  • http://alexott.net Alex Ott

    just fyi: I added feed for clojure tag to the Planet Clojure

    • http://david.dossot.net David Dossot

      Thanks Alex, very appreciated!