Running Luminus on Dokku

Luminus provides a great way to get up and running with a Clojure web application. However, building your app is only half the work. Once you've got your app working, the next step is to host it somewhere so that the users can access it.

Cloud platforms, such as AWS, are a popular choice for deploying large scale solutions. On the other hand, VPS services like Digital Ocean and Linode provide a more economical alternative for small scale applications. The downside of running your own VPS is that managing it can be labor intensive. This is where Dokku comes in. It's a private PaaS modelled on Heroku that you can use to provision a VPS.

Let's take a look at what's involved in provisioning a Digital Ocean droplet with Dokku and deploying a Luminus web app to it.

set up the server

Let's create a droplet with Ubuntu LTS (18.0.4 at the time of writing) and SSH into it. We'll need to add new APT repositories before we install Dokku.

  1. add the universe repositorysudo add-apt-repository universe
  2. add the key wget -nv -O - https://packagecloud.io/dokku/dokku/gpgkey | apt-key add -
  3. add the Dokku repo echo "deb https://packagecloud.io/dokku/dokku/ubuntu/ bionic main" > /etc/apt/sources.list.d/dokku.list

Once the repositories are added, we'll need to update the dependencies and install Dokku.

  1. update dependencies sudo apt-get update && sudo apt-get upgrade
  2. install dokku apt-get install dokku

Once Dokku is installed, we'll create an application and a Postgres database instance.

  • create the app dokku apps:create myapp
  • install dokku-postgres plugin sudo dokku plugin:install https://github.com/dokku/dokku-postgres.git
  • create the db dokku postgres:create mydb
  • link the db to the app dokku postgres:link mydb myapp

We're now ready to deploy the app.

Create a new Luminus application

Let's create a Luminus application on your local machine.

  1. lein new luminus myapp +postgres
  2. cd myapp

Let's update the app to run migrations on startup by updating the myapp.core/start-app function to run the migrations.

(defn start-app [args]
  (doseq [component (-> args
                        (parse-opts cli-options)
                        mount/start-with-args
                        :started)]
    (log/info component "started"))

  ;;run migrations  
  (migrations/migrate ["migrate"] (select-keys env [:database-url]))

  (.addShutdownHook (Runtime/getRuntime) (Thread. stop-app)))

Next, we need to update env/prod/resources/logback.xml to use STDOUT for the logs:

<?xml version="1.0" encoding="UTF-8"?>
<configuration>
    <statusListener class="ch.qos.logback.core.status.NopStatusListener" />
    <appender name="STDOUT" class="ch.qos.logback.core.ConsoleAppender">
        <!-- encoders are assigned the type
             ch.qos.logback.classic.encoder.PatternLayoutEncoder by default -->
        <encoder>
            <charset>UTF-8</charset>
            <pattern>%date{ISO8601} [%thread] %-5level %logger{36} - %msg %n</pattern>
        </encoder>
    </appender>
    <logger name="org.apache.http" level="warn" />
    <logger name="org.xnio.nio" level="warn" />
    <logger name="com.zaxxer.hikari" level="warn" />
    <root level="INFO">
        <appender-ref ref="STDOUT" />
    </root>
</configuration>

Deploy the application to Dokku

We're now ready to deploy the app. First, we'll need to create a Git repo and add the app contents to it.

  1. git init
  2. git add .gitignore Procfile project.clj README.md src/* env/* test/* resources/*
  3. git commit -a -m "initial commit"

Next, we'll add the remote for the Dokku repository on the server and push the project to the remote. Dokku will automatically build the project once it's pushed, and deploy the application when the build is successful.

  1. git remote add dokku dokku@<server name>:myapp
  2. git push dokku master

The app will be pushed to the server where it will be compiled and run. If everything went well you should see output that looks something like the following:

...
-----> Building with Leiningen
       Running: lein uberjar
       Compiling sample.app
       2019-01-18 01:10:30.857:INFO::main: Logging initialized @6674ms to org.eclipse.jetty.util.log.StdErrLog
       Created /tmp/build/target/myapp-1.0.1.jar
       Created /tmp/build/target/myapp.jar
...
=====> web=1
...
-----> Waiting for 10 seconds ...
-----> Default container check successful!
-----> Running post-deploy
-----> Configuring myapp.<server name>...(using built-in template)
-----> Creating http nginx.conf
-----> Running nginx-pre-reload
       Reloading nginx
-----> Setting config vars
       DOKKU_APP_RESTORE:  1
=====> 8dc31ac11011111117f71e4311111ca5962cf316411d5f0125e87bbac26
=====> Application deployed:
       http://myapp.<server name>

To http://<server name>:myapp
   6dcab39..1c0c8b7  master -> master

We can check the status of the application in the logs by running dokku logs myapp command on the server. The output should looks something like the following.

JAVA_TOOL_OPTIONS defaults based on dyno size. Custom settings will override them.
Picked up JAVA_TOOL_OPTIONS: -Xmx300m -Xss512k -XX:CICompilerCount=2 -Dfile.encoding=UTF-8
2019-01-19 19:09:48,258 [main] INFO  myapp.env -
-=[myapp started successfully]=-
2019-01-19 19:09:50,490 [main] INFO  luminus.http-server - starting HTTP server on port 5000
2019-01-19 19:09:50,628 [main] INFO  org.xnio - XNIO version 3.3.6.Final
2019-01-19 19:09:51,236 [main] INFO  org.projectodd.wunderboss.web.Web - Registered web context /
2019-01-19 19:09:51,242 [main] INFO  myapp.core - #'myapp.config/env started
2019-01-19 19:09:51,243 [main] INFO  myapp.core - #'myapp.db.core/*db* started
2019-01-19 19:09:51,243 [main] INFO  myapp.core - #'myapp.handler/init-app started
2019-01-19 19:09:51,244 [main] INFO  myapp.core - #'myapp.handler/app started
2019-01-19 19:09:51,249 [main] INFO  myapp.core - #'myapp.core/http-server started
2019-01-19 19:09:51,249 [main] INFO  myapp.core - #'myapp.core/repl-server started
2019-01-19 19:09:51,250 [main] INFO  myapp.core - running migrations
2019-01-19 19:09:51,257 [main] INFO  migratus.core - Starting migrations
2019-01-19 19:09:51,418 [main] INFO  migratus.database - creating migration table 'schema_migrations'
2019-01-19 19:09:51,992 [main] INFO  migratus.core - Running up for [20190118214013]
2019-01-19 19:09:51,997 [main] INFO  migratus.core - Up 20190118214013-add-users-table
2019-01-19 19:09:52,099 [main] INFO  migratus.core - Ending migrations

You should now be able to check your application in the browser by navigating to http://<server name>.

Set up HTTPS

As the last step we'll set up HTTPS for the application using dokku-letsencrypt plugin. We'll set the app to run on the root domain on the server.

  1. add the root domain to the app dokku domains:add myapp <server name>
  2. remove the subdomain from the app dokku domains:remove myapp myapp.<server name>
  3. install the plugin sudo dokku plugin:install https://github.com/dokku/dokku-letsencrypt.git
  4. set the email for renewal warnings dokku config:set --no-restart myapp DOKKU_LETSENCRYPT_EMAIL=<your email>
  5. add HTTPS to the app sudo dokku letsencrypt myapp
  6. set up auto-renew for the certificate dokku letsencrypt:auto-renew

That's all there is to it. The application is now deployed to the droplet, it's hooked up to the database, and it's using Let's Encrypt SSL/TLS Certificates.

Any further updates to the application simply involve committing the changes to the local Git repo and pushing them to the server as we did with our initial deploy.

I recommend taking look at the official documentation on the Dokku site for more information about Dokku. I think it provides an excellent solution for running your VPS. If you're evaluating different options for deploying your Clojure apps give Dokku a look.

Permalink

I’m hoping not to choose, though I’m 100% Clojure right now.

I’m hoping not to choose, though I’m 100% Clojure right now.

I would say that Elixir (and the infrastructure, such as Mix and Hex) really embrace a number of developer affordances (first class REPL, great documentation, outstanding exception output) that is not nearly as developed in the Clojure world, even with third party add-ons.

I would also say that Clojure excels at data manipulation and is more concise at such things; Elixir atoms somewhat combine Clojure keywords and symbols, and some very useful ideas in Clojure (such as keywords acting as functions to extract a value from a map, or all the nil-punning in Clojure) don’t have a precise analog in Elixir … on the other hand, for structs, you have dotted name syntax to access values in nested associative structures.

In terms of BEAM vs. JVM, I think it is worth noting that the BEAM is not as powerful as the JVM, yet for many things, BEAM gives better throughput. For example, the bane of performance in JVM is “stop-the-world” GC … BEAM doesn’t have that problem, because each lightweight process has its own small, unique heap, and memory can’t be directly shared between processes.

The core abstraction of Erlang and Elixir … that every interaction between code inside a process and the outside world (other processes, the file system, etc.) takes the form of messages in the process’ inbox is … really cool. Like Clojure, Elixir can operate very quickly because it doesn’t get bogged down in locks and synchronization primitives.

Clojure code tends to focus on a data structure, then use and reuse existing primitives ( get, assoc-in, map, reduce, etc.) to manipulate the data. What I’ve seen so far in Elixir is more of an emphasis on macro-driven DSLs and code over pure data (pure data behind the DSL).

From a Clojure perspective, there’s the prevalence of macros to inject functions into a module, often to allow a module to implement a behavior (set of functions) without implementing all of them. This is uncommon in Clojure (having a macro def more than one thing, or anything not very obvious, is rare), and might be addressed differently, using records and protocols.

Clojure can be “sluggish” to start; you have to pay for JVM startup costs, then loading a lot of pre-compiled Clojure classes, bootstrapping of the REPL itself (more Clojure code) and any other (often unknown and invisible) related code (`clj` might get you to a prompt in a fair chunk of a second, but starting a REPL from within Cursive can take many seconds). Once you have the Clojure REPL running, you try to leave it running for hours if not longer. Elixir has a compile phase, but then the actual startup is extremely quick, so having to restart the REPL is not so onerous.

I’ve found in Clojure I rarely use a debugger the way I did in Java; more likely, I’ll add debug logging or output to a function and reload that function in the REPL. I’m not sure whether the same flow works in Elixir, or how useful the debugger is.

Finally, from my cursory involvement (playing around, attending ElixirConf) it feels like the core maintainers of Elixir are more actively involved in the community … at the very least, there’s more of a public roadmap and Jose has outlined very specifically where the language should go, and where it should not: a gradual move of emphasis from the language and core libraries to external libraries.

Permalink

Episode 012: Embrace the REPL

Christoph complicated development by misunderstanding the REPL.

  • We go back to the roots of Christoph’s experience with Clojure…
  • How do I run Clojure code?
  • The REPL is fun for evaluation, but how do you run a “real” program?
  • Experience in other languages: edit the file, compile, rerun the program
  • Mentality: get through the edit, compile, run loop as fast as possible!
  • Autobuilder is the logical end.
  • “Where’s my autobuilder in Clojure?!”
  • The REPL model is fundamentally different.
  • “[The REPL] is like cutting the Gordian Knot. You change the problem and that’s how you solve it.”
  • “The reason why the problem I wanted solved, wasn’t ‘solved’, is because nobody has that problem because they do things differently here.”
  • Tactic 1: edit, save, restart the REPL
  • “Restarting the REPL isn’t super slow, but let’s just say it’s not instantaneous.”
  • Tactic 2: edit, save, run (use 'the.namespace :reload) in the REPL
  • “Now I have a REPL history of use statements!”
  • Problems:
    • forgetting to reload dependent namespaces
    • loading dependencies in the wrong order
    • old definitions built up
  • Big Problem: function renames left the old version around, so accidental calls using the old name produced no errors and ran old behavior!
  • Back to quitting the REPL to clean out the cruft. Ugh!
  • Discovered clojure.tools.namespace! Reloads everything and cleans out the cruft!
  • Tactic 3: edit, save, (use '[clojure.tools.namespace.repl :only (refresh)]), (refresh)
  • Problem: refresh would purge itself!
  • “I don’t know why it took me so long to discover the magical user namespace!”
  • The REPL will automatically use the user namespace.
  • “I can put code into a user namespace and it will be there for me.”
  • Christoph would switch namespaces in the REPL without even stopping to wonder what namespace he started in.
  • “It’s like walking out of a door and not even thinking about the fact you’re in a building. Oh wait! What room did I just leave?”
  • Tactic 4: make sure refresh is in the user namespace, edit, save, (refresh)
  • However, Christoph still didn’t understand the REPL.
  • Just thought the REPL was for:
    • reloading the code and restarting
    • evaluating snippets of code
    • inspecting stuff
  • Nate’s big ah-ha moment: “Not only is my application inspectable, it’s fungible. I can change it in place as it’s flying along! I don’t have to restart it!”
  • Christoph’s hint that there was still more came from seeing comment blocks in code and reading about vim-fireplace.
  • “There’s a new room you can explore. That room is editor integration with the REPL.”

Clojure in this episode:

  • use
  • clojure.tools.namespace.repl/refresh
  • clojure.pprint/pprint

Related projects:

Permalink

Journal 2019.3 - spec, clojure.main errors

spec

I spent almost the entire week completing the large chunk of refactoring I’ve been doing in spec-alpha2. At this point, I think everything is put back together. I’ve updated the spec-alpha2 readme to include some information on using the lib as a git dep, namespace changes, and a summary of things you might find that are different.

At this point, there’s not many visible advantages for a spec user yet. Most of the changes are in the implementation and will affect those either implementing their own specs or wanting to do spec creation without using the spec op macros.

For the latter use case, more dynamic spec uses often ran into issues wrapping the spec op macros to do something like use a set to generate a s/keys spec. Doing so required use of wrapper macros, eval, etc. You can now use the new entry point s/spec*, a function, to generate a spec from data. spec* takes a spec op form, which must use fully-qualified symbols (or you can use the new explicate function to qualify a form).

(require '[clojure.spec-alpha2 :as s])

(defn make-keys-spec [keys]
  (s/spec* (list `s/keys :req (vec keys))))

(s/def ::a int?)
(s/def ::b keyword?) 
(s/explain (make-keys-spec #{::a ::b}) {::a 1})
;; #:user{:a 1} - failed: (contains? % :user/b) 

This is a new functional data interface below the op form macros, which takes data, primarily in the form oflists and symbols.

Additionally, we are starting to formalize what it means to implement your own custom specs. Most of that has existed, but it hasn’t been something we were willing to commit to but we’re moving in that direction. The Spec protocol has been pulled out to a new clojure.spec-alpha2.protocols namespace and implementing a new spec consists of two parts. First, creating a spec op macro that explicates (fully qualifies) the form, and then emits a call to the spec* functional interface. Second, you must provide an implementation of the create-spec multimethod, which is a routing layer under spec* that takes the form and returns an implementation of the Spec protocol. I’m not going to go through the details of this, but everything in spec-alpha2 is now implemented this way so there are copious examples.

I’ll be continuing to work on spec next week, but not sure exactly what yet!

clojure.main errors

This is actually from two weeks ago, but I forgot it last week and it was important. A new user on Slack reported a somewhat messy error message and I suggested that they should move to Clojure 1.10 to get the benefit of the new error reporting. But they were already on 1.10! I dug in a bit more.

The errors were being reported from a ring server started under lein. As I examined the stack trace, it became clear that clojure.main was being used as the main runner by Leiningen to run the ring server. And here’s the important part - we did a ton of work to better report errors from the clojure.main repl, but we did not extend that work to clojure.main -m, -e, etc modes!

I feel dumb for missing this in 1.10. This affects lots of ways that people interact with Clojure, even through other tools like Leiningen. For example, when you are AOT compiling source is being read and you can encounter spec macro errors, but you’ll still get old Clojure 1.9 style errors and a big (useless) stack dump, rather than all the stuff we did for the REPL.

Anyhow, I wrote this up at CLJ-2463 and even hacked up a proof of concept patch and this will be a high priority for 1.11. My big questions are around the stack trace - I think in read/compile/macroexpand cases this (usually) has little value and could be omitted, but some users (language implementors or macro writers) may occasionally want that stack to debug something, so maybe that could have an override switch. Also, we are currently printing just the Clojure file name, without the path, in the error message. With a full Java stack trace, there’s more info there, but with just the file name (“core.clj”), we’ve left out critical context. So, need to think through these questions a bit more, but overall this is a small change with a big impact.

Clojure Survey

One last reminder! The 2019 State of Clojure survey is open until Tues Jan 22nd. Please complete it if you use Clojure!

Non Clojure stuff I enjoyed this week…

Here’s two instrumental things I ran into this week and enjoyed. First, shibyua (ft San Holo) by Covet, propelled by guitarist Yvette Young using tapping and open tunings. Here’s also a video of just her doing this song.

Second, something a bit more raucous - Joy on Fire and their submission for Tiny Desk. I dig their sound - drums, bass, and sax.

Permalink

What you need to know about the Gilded Rose interview problem

A few years ago, I interviewed for a Dev Ops position at a pretty prominent CI shop in San Francisco. Part of the interview process involved a take-home code-assignment (of course). This particular assignment is known as the Gilded Rose.

The Gilded Rose isn’t a fizzbuzz. It’s not “Implement a Trie” in whatever language you want. The Gilded Rose is a problem that involves adding some functionality into the worst code that a developer could ever write. I don’t mean any offense to whoever created the Gilded Rose problem, bad code is entirely the intention! They want to see how you work through a terrible codebase to add new functionality.

This example will be in Clojure, as thats what I did this exercise with at the time, but it’s worth mentioning that the Gilded Rose exists for almost any programming language. If you find yourself facing this question, you’ll be left with usually two files.

The README

The README for this project is quite long. It’ll list stipulations for this problem that you have to follow for it to be successful. Some are downright stupid (more on that in a moment), but that’s the reality of actually working in real code. You’ll find those there, too. Anyways, here’s an example of what you might find in the Gilded Rose README (also, if you’ve ever played WoW, you might notice some references here).


Hi and welcome to team Gilded Rose.

As you know, we are a small inn with a prime location in a prominent city ran by a friendly innkeeper named Allison. We also buy and sell only the finest goods. Unfortunately, our goods are constantly degrading in quality as they approach their sell by date.

We have a system in place that updates our inventory for us. It was developed by a no-nonsense type named Leeroy, who has moved on to new adventures. Your task is to add the new feature to our system so that we can begin selling a new category of items.

First an introduction to our system:

  • All items have a sell-in value which denotes the number of days we have to sell the item
  • All items have a quality value which denotes how valuable the item is
  • At the end of each day our system lowers both values for every item

Pretty simple, right? Well this is where it gets interesting:

  • Once the sell by date has passed, quality degrades twice as fast
  • The quality of an item is never negative
  • “Aged Brie” actually increases in quality the older it gets
  • The quality of an item is never more than 50
  • “Sulfuras”, being a legendary item, never has to be sold or decreases in quality
  • “Backstage passes”, like aged brie, increases in quality as it’s sell-in value approaches; quality increases by 2 when there are 10 days or less and by 3 when there are 5 days or less but quality drops to 0 after the concert

We have recently signed a supplier of conjured items. This requires an update to our system:

  • “Conjured” items degrade in quality twice as fast as normal items

Feel free to make any changes to the update-quality method and add any new code as long as everything still works correctly. However, do not alter the item function as that belongs to the goblin in the corner who will insta-rage and one-shot you as he doesn’t believe in shared code ownership.

Just for clarification, an item can never have its quality increase above 50, however “Sulfuras” is a legendary item and as such its quality is 80 and it never alters.

Analyzing the Requirements

There’s a lot going on in this README, so I’m going to break this problem apart piece by piece. However, the interesting parts of this README aren’t the bullet points contained within it. It’s the content not bulleted (in fact, Im going to leave those bullet points out, scroll back up and read them if needed).

  • You’re making an update to this system to support conjured items. That’s it.
  • the “item” function can NOT be changed. Don’t touch it.
  • Legendary Items are an edge case. Sulfuras breaks the rules.
  • This is implied but not directly stated: YOU NEED TO WRITE TESTS.
  • YOU NEED TO WRITE TESTS FIRST.

If you do this for an interview, you’ll likely be told, “Oh don’t spend more than an hour or two on it.” Which is fair. If you don’t get anything else done in that hour or two, write tests and try to refactor. That is actually what this interview question is all about — the conjured items (the addition) is not substantial compared to ensuring code works before and after refactoring.

Why do I keep mentioning refactoring? Like I said at the beginning, this code is a nightmare. You’ll be given code like this (Clojure, sorry everyone else, but you should still be able to recognize that this is NOT good code):

(ns gilded-rose.core)

(defn update-quality [items]
  (map
    (fn[item] (cond
      (and (< (:sell-in item) 0) (= "Backstage passes to a TAFKAL80ETC concert" (:name item)))
        (merge item {:quality 0})
      (or (= (:name item) "Aged Brie") (= (:name item) "Backstage passes to a TAFKAL80ETC concert"))
        (if (and (= (:name item) "Backstage passes to a TAFKAL80ETC concert") (>= (:sell-in item) 5) (< (:sell-in item) 10))
          (merge item {:quality (inc (inc (:quality item)))})
          (if (and (= (:name item) "Backstage passes to a TAFKAL80ETC concert") (>= (:sell-in item) 0) (< (:sell-in item) 5))
            (merge item {:quality (inc (inc (inc (:quality item))))})
            (if (< (:quality item) 50)
              (merge item {:quality (inc (:quality item))})
              item)))
      (< (:sell-in item) 0)
        (if (= "Backstage passes to a TAFKAL80ETC concert" (:name item))
          (merge item {:quality 0})
          (if (or (= "+5 Dexterity Vest" (:name item)) (= "Elixir of the Mongoose" (:name item)))
            (merge item {:quality (- (:quality item) 2)})
            item))
      (or (= "+5 Dexterity Vest" (:name item)) (= "Elixir of the Mongoose" (:name item)))
        (merge item {:quality (dec (:quality item))})
      :else item))
  (map (fn [item]
      (if (not= "Sulfuras, Hand of Ragnaros" (:name item))
        (merge item {:sell-in (dec (:sell-in item))})
        item))
  items)))

(defn item [item-name, sell-in, quality]
  {:name item-name, :sell-in sell-in, :quality quality})

(defn update-current-inventory[]
  (let [inventory 
    [
      (item "+5 Dexterity Vest" 10 20)
      (item "Aged Brie" 2 0)
      (item "Elixir of the Mongoose" 5 7)
      (item "Sulfuras, Hand Of Ragnaros" 0 80)
      (item "Backstage passes to a TAFKAL80ETC concert" 15 20)
    ]]
    (update-quality inventory)
    ))

Tips for the Gilded Rose

I’m not going to share my solution with you — where’s the fun in that? Also, you can just browse Github if you must know. However, I will leave you with some tips.

  • Sulfuras is an edge case, but refactor it in such a way that you can add new edge cases (Perhaps Thunderfury, Blessed Blade of the Windseeker)
  • Use as many private functions as you find makes the public function readable, in Clojure, it can mean quite a few.
  • Consider using overloading, pattern matching, or value matching for items that have specific requirements (Brie, Concert Tickets)
  • Consider storing items in a map with a lambda that contains their “update-quality” behavior function.
  • Write tests first. Especially if your interviewer is asking you to commit this to Github. They will look at your commit history, because no-one (myself included) should touch this monster without adequate test coverage first.
  • Use comments throughout your code. The idea here is to see if you can take an abomination and make it usable for other developers while not breaking anything.
  • If you’re using Clojure for your project, this problem is an excellent one to consider Multimethods.

That’s all I have for you! Best of luck and have fun with it!

If you’re interested in learning more about Clojure, you can find my other Clojure related posts here!

The post What you need to know about the Gilded Rose interview problem appeared first on Brad Cypert.

Permalink

Reinforcement Learning : Exploration vs Exploitation : Multi-Armed Bandits


;; Reinforcement Learning : Exploration vs Exploitation : Multi-Armed Bandits

;; I'm reading the excellent:

;; Reinforcement Learning: An Introduction
;; by Richard S. Sutton and Andrew G. Barto

;; The book's website, on which is available a complete pdf, is here:
;; http://www.incompleteideas.net/book/the-book.html

;; In Chapter 2, they introduce multi-armed bandits as a simplified model problem

;; On the basis that you don't understand anything you can't explain to a computer, I thought I'd code it up:

;; Here is a 2 armed bandit
(defn bandit [action]
(case action
:arms? [:right :left]
:right (if (< (rand) 0.5) 4 0)
:left (if (< (rand) 0.2) 5 0)
:oops!!))

;; We can ask it how many arms it's got, and what they're called
(bandit :arms?) ; [:right :left]

;; And we can pull those arms. Rewards are variable.
(bandit :right) ; 4 ; 4 ; 4 ; 0 ; 0 ; 0 ; 0
(bandit :left) ; 5 ; 0 ; 0 ; 0 ; 5 ; 0 ; 5 ; 0

;; Once we pull an arm, we'll have an action/reward pair
(bandit :right) ; 4
;; the pair would be:
[:right 4]

;; Here's a function that yanks an arm at random, and gives us such a pair
(defn random-yank [bandit]
(let [a (rand-nth (bandit :arms?))]
[a (bandit a)]))

(random-yank bandit) ; [:left 0]
(random-yank bandit) ; [:right 4]

;; And a utility function to take the average of a sequence. We need to be able to provide a default value if the sequence is empty.
(defn average
([seq default] (if (empty? seq) default (/ (reduce + seq) (count seq))))
([seq] (average seq 0)))

;; with some examples
(average [1 2 3 4 5]) ; 3
(average (list) 10) ; 10
(average (list 1) 2) ; 1
(average [] 100) ; 100


;; If we just pull arms at random we get an average reward of about 1.5

(float (average (map second (repeatedly 1000 #(random-yank bandit))))) ; 1.49

;; Since we can see the code for this particular bandit, we know that
;; the expected value of pulling the right arm is 2 (a half-chance of
;; a reward of 4) and the expected reward for the left arm is 0.2*5 = 1

;; So if we were seeking to maximize reward, we'd probably be best to pull the right arm all the time.

(float (average (map bandit (repeat 10000 :right)))) ; 1.9912
(float (average (map bandit (repeat 10000 :left )))) ; 0.985


;; The interesting question is, if we don't know how the bandit works, how should we design an algorithm that gets the most reward?
;; (Or at least do better than yanking arms at random!)

;; One thing our algorithm is going to have to do is keep some state to record what happens.
;; Let's start by recording the results of all pulls to date:

;; At first, we know nothing, so we can set up a table to represent that we know nothing
(defn initial-state [bandit]
(into {} (for [k (bandit :arms?)] [k (list)])))

;; We haven't pulled either arm yet
(initial-state bandit) ; {:right (), :left ()}

;; When we get a new action reward/pair, we'll add the result to our state
(defn update-state [state [action reward]]
(update-in state [action] #(conj % reward)))

;; here are some examples of using update-state
(update-state {:right (), :left ()} [:right 2]) ; {:right (2), :left ()}
(reduce update-state {:right (), :left ()} [[:right 2] [:left 3] [:right 4] [:right 5]]) ; {:right (5 4 2), :left (3)}

;; here's how we can use it to record the result of ten random yanks
(reduce update-state
(initial-state bandit)
(repeatedly 10 #(random-yank bandit))) ; {:right (4 4 0 0 0), :left (0 0 0 0 5)}


;; Once we actually have some data, we can make estimates of the expected rewards

;; mapvals applies a function to every value in a map, returning a new map with the same keys
(defn mapvals [m f] (into {} (for [[k v] m] [k (f v)])))

;; examples
(mapvals {} inc) ; {}
(mapvals {:a 1} inc) ; {:a 2}
(mapvals {:a 1, :b 2} inc) ; {:a 2, :b 3}
(mapvals {:a 1, :b 2, :c 3} #(* % %)) ; {:a 1, :b 4, :c 9}


;; In the book, Q_t(a) is the current estimate (at time t)
;; We'll use as our estimate of the value of an action the average value seen so far, or zero if we have no information

(defn Q [state] (mapvals state #(average % 0)))

;; examples
(Q '{:right (5 4 2), :left (3)}) ; {:right 11/3, :left 3}
(Q '{:right (5 4 2), :left ()}) ; {:right 11/3, :left 0}
(Q (initial-state bandit)) ; {:right 0, :left 0}
(Q (update-state (initial-state bandit) (random-yank bandit))) ; {:right 0, :left 2}

;; let's check that we get roughly what we expect in the long run
(Q (reduce update-state (initial-state bandit)
(repeatedly 10000 #(random-yank bandit)))) ; {:right 9832/5015, :left 1027/997}


;; If we have estimates of the value of each arm, then a good way to
;; use them is to pull the arm with the highest estimate.

;; This is called 'exploitation', as opposed to 'exploration', which
;; is when you try things you think may be suboptimal in order to get
;; information

;; The 'greedy' action is the one with the highest expected value. Of
;; course there may be more than one greedy action especially at first.

;; To help with this, another utility function:

;; max-keys finds the keys with the highest value in a map, and returns a list with just these keys and values
(defn max-keys [m]
(let [slist (reverse (sort-by second m))
[_ max] (first slist)]
(take-while #(= (second %) max) slist)))

;; examples
(max-keys {}) ; ()
(max-keys {1 0}) ; ([1 0])
(max-keys {1 0, 2 0}) ; ([2 0] [1 0])
(max-keys {1 0, 2 1}) ; ([2 1])
(max-keys {1 0, 2 1, 3 -1 , 4 -3, 5 2, 6 2}) ; ([6 2] [5 2])

;; if there is a tie for the greedy action, we can choose at random between the candidates
;; And so we can go from estimates to greedy action like this:
(defn greedy-action [estimates]
(first (rand-nth (max-keys estimates))))

;; examples
(greedy-action '{:right 10, :left 3}) ; :right
(greedy-action '{:right 10, :left 3 :centre 20}) ; :centre
(greedy-action '{:right 10, :left 3 :centre 3}) ; :right
(greedy-action '{:right 3, :left 3 :centre 3}) ; :right

(greedy-action (Q '{:right (5 4 2), :left (3)})) ; :right
(greedy-action (Q '{:right (), :left (3)})) ; :left
(greedy-action (Q (initial-state bandit))) ; :left

;; after a lot of random pulls, the greedy action should reliably be the one with the highest expected payoff
(greedy-action (Q (reduce update-state (initial-state bandit)
(repeatedly 10000 #(random-yank bandit))))) ; :right

;; OK, so we have our stage set, a way of recording what's happened, and some helpful functions defined.


;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;

;; Our first try at a learning algorithm will be 'by hand', as it were.

;; We'll always make the 'greedy' choice.

;; At first, we have no records to go on
(initial-state bandit) ; {:right (), :left ()}

;; expected values for both levers are therefore zero
(Q (initial-state bandit)) ; {:right 0, :left 0}

;; so the greedy action will get chosen at random
(greedy-action (Q (initial-state bandit))) ; :left

;; in this case, we've chosen :left, and the bandit's response is
(bandit :left) ; 0

;; we record it
(update-state (initial-state bandit) [:left 0]) ; {:right (), :left (0)}

;; and we have a new state
'{:right (), :left (0)}

;; new estimates
(Q '{:right (), :left (0)}) ; {:right 0, :left 0}

;; and again, we choose at random
(greedy-action (Q '{:right (), :left (0)})) ; :left

;; the bandit is not feeling very generous
(bandit :left) ; 0

(update-state '{:right (), :left (0)} [:left 0]) ; {:right (), :left (0 0)}

;; new state:
'{:right (), :left (0 0)}

;; new estimates
(Q '{:right (), :left (0 0)}) ; {:right 0, :left 0}

;; this time we choose :right
(greedy-action (Q '{:right (), :left (0 0)})) ; :right

;; and the bandit pays out!
(bandit :right) ; 4

(update-state '{:right (), :left (0 0)} [:right 4]) ; {:right (4), :left (0 0)}

;; the greedy action will be :right now, because we have evidence that right is better.
(greedy-action (Q '{:right (4), :left (0 0)})) ; :right

;; You get the idea......

;; Let's automate that....

;; Given a state and a bandit, we decide an action and the bandit
;; responds, producing an action/reward pair, and a new state

(defn greedy-algorithm [bandit state]
(let [action (greedy-action (Q state))
reward (bandit action)]
[[action reward] (update-state state [action reward])]))


(greedy-algorithm bandit (initial-state bandit)) ; [[:left 0] {:right (), :left (0)}]

;; To get something we can iterate:

(defn step [[[a r] state]]
(greedy-algorithm bandit state))

(iterate step [ [:dummy :dummy] (initial-state bandit)])

;; ([[:dummy :dummy] {:right (), :left ()}]
;; [[:left 5] {:right (), :left (5)}]
;; [[:left 0] {:right (), :left (0 5)}]
;; [[:left 0] {:right (), :left (0 0 5)}]
;; [[:left 0] {:right (), :left (0 0 0 5)}]
;; [[:left 0] {:right (), :left (0 0 0 0 5)}]
;; [[:left 0] {:right (), :left (0 0 0 0 0 5)}]
;; [[:left 0] {:right (), :left (0 0 0 0 0 0 5)}]
;; [[:left 0] {:right (), :left (0 0 0 0 0 0 0 5)}]
;; [[:left 0] {:right (), :left (0 0 0 0 0 0 0 0 5)}]
;; [[:left 0] {:right (), :left (0 0 0 0 0 0 0 0 0 5)}]
;; [[:left 0] {:right (), :left (0 0 0 0 0 0 0 0 0 0 5)}]
;; [[:left 5] {:right (), :left (5 0 0 0 0 0 0 0 0 0 0 5)}]
;; [[:left 0] {:right (), :left (0 5 0 0 0 0 0 0 0 0 0 0 5)}]
;; [[:left 0] {:right (), :left (0 0 5 0 0 0 0 0 0 0 0 0 0 5)}]
;; [[:left 0] {:right (), :left (0 0 0 5 0 0 0 0 0 0 0 0 0 0 5)}]
;; [[:left 0] {:right (), :left (0 0 0 0 5 0 0 0 0 0 0 0 0 0 0 5)}]

;; In this case, the greedy algorithm happens to get a payout on its
;; first try, and decides that it will pull that arm for ever. It
;; never even tries the other arm.

;; Try again:

(iterate step [ [:dummy :dummy] (initial-state bandit)])
;;([[:dummy :dummy] {:right (), :left ()}]
;; [[:right 0] {:right (0), :left ()}]
;; [[:right 0] {:right (0 0), :left ()}]
;; [[:left 0] {:right (0 0), :left (0)}]
;; [[:right 4] {:right (4 0 0), :left (0)}]
;; [[:right 4] {:right (4 4 0 0), :left (0)}]
;; [[:right 4] {:right (4 4 4 0 0), :left (0)}]
;; [[:right 4] {:right (4 4 4 4 0 0), :left (0)}]
;; [[:right 4] {:right (4 4 4 4 4 0 0), :left (0)}]
;; [[:right 4] {:right (4 4 4 4 4 4 0 0), :left (0)}]
;; [[:right 0] {:right (0 4 4 4 4 4 4 0 0), :left (0)}]
;; [[:right 0] {:right (0 0 4 4 4 4 4 4 0 0), :left (0)}]
;; [[:right 4] {:right (4 0 0 4 4 4 4 4 4 0 0), :left (0)}]
;; [[:right 0] {:right (0 4 0 0 4 4 4 4 4 4 0 0), :left (0)}]
;; [[:right 4] {:right (4 0 4 0 0 4 4 4 4 4 4 0 0), :left (0)}]
;; [[:right 0] {:right (0 4 0 4 0 0 4 4 4 4 4 4 0 0), :left (0)}]

;; In this case, it tried the right arm a couple of times, then had a
;; go with the left arm, then went back to the right arm, won a
;; payout, and then got hung up on pulling the right arm repeatedly.



;; We've got a couple of problems here!

;; First is that the algorithm has clearly got into a state where it
;; always pulls the left arm (in the first case), and the right
;; arm (in the second case).

;; It can't be doing the right thing in both cases.

;; Secondly the state is growing linearly, as the algorithm remembers
;; all previous results. That's giving us algorithmic complexity
;; problems and the calculation will get slower and slower, and
;; eventually run out of memory.













Permalink

The Clojure Style Guide Redux

Prelude

Recently I’ve noticed that the Community Clojure Style Guide has turned 6! According to git log the project started its life on the 3rd of January, 2013!

commit 3abbcfb8b536999d0b1d64b531a9c57b49b4562a
Author: Bozhidar Batsov <bozhidar@batsov.com>
Date:   Thu Jan 3 10:09:53 2013 -0800

    Initial commit

Time sure flies fast!

After the initial announcement of the guide, I never really revisited the topic, so it seems to me that this “birthday”, is as good of an opportunity as any, to do so.

Impact

The main purpose of a style guide is always to educate (beginners) and align (experts). It showcases in a concise form the convention of a language and how to effectively use it. Its existence also spares you from the burden of having to make a lot of trivial decisions yourselves and makes it easier for people to work on many different projects (provided those projects applied the same style principles).

Did the Clojure style guide achieve its primary objectives? From my perspective it seems that the guide had a good impact on the community - I’ve often seen it cited here and there, which means that people are using it. I can only guess how many people are actually abiding by it, but I’m feeling optimistic in this regard.

The impact was not as big, however, as I had originally envisioned, because I never got to the second step I had planned - namely writing a lint-like tool that enforces the guide automatically. I pretty much wanted to replicate what I had done for Ruby with the Community Ruby Style Guide and RuboCop. I recall I spent a lot of time thinking about the name of the lint tool, but I never actually found time to start it.1

Despite my failure, others managed to fill some of the gap - I know that zprint has a community formatting mode and I think that cljfmt has defaults that are similar to what’s outlined in the style guide.

Current Status

The work on the guide has stagnated a bit in the recent years, as I’ve been very busy with work and other projects, and there haven’t been many contributors to the project. When the guide was announced a lot of people helped with growing and polishing it, but after a while the work grinded almost to a halt.

I’d really love for us to find a few more editors to help with the guide (and more occasional contributors, of course), so that the guide can be completed and always kept up-to-date with the best practices in the Clojure community. Having more editors should also dispel any notion that despite its name (community guide), the guide reflects my personal stylistic preferences, as opposed to those of the broader Clojure community.

Future Goals

So, I guess now you’re wondering how exactly can you help, right? Let me give you a few ideas - some goals I set out to achieve and never got to accomplishing.

Cohesion

One of the problems of the guide today is that certain sections are just stubs, compared to others. All sections should have equally good rationales, examples and so on. Currently that’s certainly no the case for sections like State Management, Macros, Documentation, etc.

A good style guide is a cohesive style guide. It’s a style guide with good style.

Coverage of Newer Clojure Features

Clojure introduced many new features after the guide was created that never made their way to it. Some things that come to mind are transducers, specs, reader conditionals.

We should totally fix this!

Better Rationales

I’m a big fan of Stuart Sierra’s series on Clojure Do’s and Don’t. They are similar to a style guide, but the articles there go into great detail when it comes to the rationale for the suggestions in them.

While a style guide has to be concise by definition, it also has to be clear. The Clojure style guide can certainly take a page from Stu’s book and improve the rationales of many of its suggestions.

I also think that he has covered many of topics that are not covered in the style guide presently, so it’d be nice if we borrowed those as well.

So, what’s “better rationale” to begin with. Well, unfortunately that’s really subjective, but let me try to illustrate this with a few examples. Here’s a rule with a weak rationale (or poor examples):

Leverage comp when doing so yields simpler code.

;; Assuming `(:require [clojure.string :as str])`...

;; good
(map #(str/capitalize (str/trim %)) ["top " " test "])

;; better
(map (comp str/capitalize str/trim) ["top " " test "])

Is this really simpler? Why not use threading?

And here are a few examples a consider better - either because they are self-explanatory, or they have better rationale attached to them:

Prefer condp instead of cond when the predicate & expression don’t change.

;; good
(cond
  (= x 10) :ten
  (= x 20) :twenty
  (= x 30) :thirty
  :else :dunno)

;; much better
(condp = x
  10 :ten
  20 :twenty
  30 :thirty
  :dunno)

Prefer case instead of cond or condp when test expressions are compile-time constants.

;; good
(cond
  (= x 10) :ten
  (= x 20) :twenty
  (= x 30) :forty
  :else :dunno)

;; better
(condp = x
  10 :ten
  20 :twenty
  30 :forty
  :dunno)

;; best
(case x
  10 :ten
  20 :twenty
  30 :forty
  :dunno)

Don’t use the interop syntax to construct type and record instances. deftype and defrecord automatically create constructor functions. Use those instead of the interop syntax, as they make it clear that you’re dealing with a deftype or a defrecord. See this article for more details.

(defrecord Foo [a b])
(deftype Bar [a b])

;; good
(->Foo 1 2)
(map->Foo {:b 4 :a 3})
(->Bar 1 2)

;; bad
(Foo. 1 2)
(Bar. 1 2)

Note that deftype doesn’t define the map->Type constructor. It’s available only for records.

I hope you get the point.

Looking Forward

Once the goals I’ve mentioned above have been dealt with, I’m considering to submit the style guide to https://clojure.org. Hopefully Alex, Rich and Stu would consider it a valuable addition to the official documentation.

I’d also love to find the time to write that lint tool I mentioned earlier, but I doubts that’s going to happen in the foreseeable future.

Epilogue

If you’ve got any generic feedback about the style guide (structure, content, examples, whatever) - I’d love to hear it! I’m looking forward to working with all of you on making the guide better and more useful to all the members of the Clojure community!

Keep hacking and keep editing!

  1. I think my front-runners for the name were Caliber and Zealot

Permalink

Senior Software Engineer | Scala, Clojure at Otravo (Full-time)

Are you passionate about software development? Do you believe in making a difference? We have a challenging role within our highly skilled team of dé VakantieDiscounter!

What are you going to do?

Because we build all our systems in-house with our own integrated development teams, we could really use your knowledge and skills, helping us leverage technologies such as

  • Apache Spark, for distributed data processing
  • Clojure and ClojureScript/React.js, for powering our website
  • Scala, Akka, and Play for a scalable stateless REST back-end
  • Kafka and Solr, for our data management platform
  • A/B experiments, for sensible decision making

You will be pairing to write lots of (clean) code that goes to production continuously and will be spending ± 10% on private projects to find the next big thing

Skills And Qualifications

You are a seasoned software developer (5+ years) and have solid experience with one of the technologies we use. You know functional programming and have a passion for learning and sharing your knowledge.

We consider it a plus if you have experience with:

  • Big Data, Spark, Solr/Elasticsearch
  • Cats / Scalaz / Shapeless functional programming libraries
  • React, Play framework
  • AWS, Docker or OpenStack

What do we offer?

Otravo is an informal yet professional company, beautifully situated at the Amsterdam canals. We use agile development and best practices to deliver working software with little overhead, fast time to market and high quality.

The package we offer contains the following benefits:

  • A good salary 40.000 - 65.000 gross per year;
  • 30 paid holidays;
  • Opportunities to attend conferences, training sessions, workshops, etc.;
  • Discount on your holiday trips (of course :)
  • Vibrant company culture;
  • Support with relocation (visa, housing, 30% tax ruling)
  • Free snacks and amenities; cake every sprint;
  • Free use of company gym.

About Otravo

Otravo means Online Travel Organisation and is market leader in the Benelux, Scandinavia and South Europe of online air travel sales. Travellers can book airline tickets to worldwide destinations via self-service and online, and for both leisure and business trips. Otravo offers travellers a wide selection of options, everything from worldwide flights, rental cars, hotels and sun holidays to all possible dynamic travel combinations.

Otravo is the parent company of several well-known travel brands. Brands like Vliegtickets.nl, Vliegtickets.be, WTC.nl, Schipholtickets.nl, de Vakantiediscounter, Flygstolen, Tripmonster and Greitai and most recently the brand Travelgenio are all part of Otravo. Every day over hundred thousands of visitors are welcomed to our websites and since 1983 millions of people have travelled through our brands. Otravo is a very healthy and innovative company with a growing yearly turnover of over 1,5 billion euro’s and a team of more than 500 professionals.

Get information on how to apply for this position.

Permalink

CIDER 0.20 (Oslo)

Hot on the heals of yesterday’s article) I’ve got a new one for you! This one is going to be short and sweet - just a few notes about the release of CIDER 0.20 (Oslo).

I guess some of you are surprised by the news - after all wasn’t CIDER 0.19 (Raleigh) released only a couple of weeks ago? Usually CIDER releases are (many) months apart. Well, while that’s normally the case, I’ve always wanted to do more focused and frequent releases and it seems that I managed to succeed this time around.

CIDER 0.20 includes very few changes overall.1 I’ll cover them briefly here.

Cleaning up Shop

CIDER (or cider-nrepl to be precise) dropped support for nREPL 0.2 (a.k.a. tools.nrepl). Now you’ll need nREPL 0.4+ (ideally 0.5.3+), but that’s not a big deal as at this point both Boot and Leiningen ship the modern nREPL. Just make sure you’re on Boot 2.8.2+ or Leiningen 2.8.3+.

I’ll be gradually removing support for nREPL 0.2 from all nREPL extensions I’m maintaining in the months to come. I’d advise other people maintaining nREPL extensions to do the same.

Native nREPL Pretty-printing

CIDER used to rely on a custom middleware for result pretty-printing for many years now, but nREPL recently added native support for pretty-printing values, and this rendered our custom approach redundant. The biggest change in CIDER 0.20 is that it switches to nREPL’s built-in pretty-printing API.

Some other related changes:

  • Support for zprint has been added.
  • We’re passing to the print engines (e.g. zprint) their native options, instead of wrapping their invocations in dynamic bindings (e.g. *print-length*, *print-level*, etc).
  • Pretty-printing is enabled in REPL buffers by default.
  • You can pretty-print results in scratch buffers using C-u C-j.

There’s now some work underway in nREPL to support streamed printing of values. That’s going to be a really great improvement of the current situation as it would mean faster feedback for the clients and no possibility for “invisible” printing of infinite/huge sequences on the server-side.2

CIDER now has a new section of its manual dedicated to pretty-printing. You should definitely check it out!

Note that the new pretty-printing requires for you to be on nREPL 0.5+. If you’re not, you’re simply going to get results that are not pretty-printed.

Misc

I’ve spent a bit of time reorganizing CIDER’s manual and breaking down some ridiculously big sections (e.g. Using the REPL, Configuration, etc) into smaller chunks. I think you’ll find the new layout easier to navigate, but there’s still a lot of room for improvement.

A handful of small changes and bug improvements also made it in this release, but there’s nothing worth highlighting there. Just the usual quality of life improvements you’re bound to expect from each new release.

On a related note - there’s also a new clojure-mode release, that you should certainly check out!

CIDER Next

I don’t have a specific plan for the next release - neither in terms of features nor in terms of a timeline.

I’ll be taking some time off from working on open-source now that the new CIDER is out, and I hope that in the mean time the work on the next nREPL release will be done and I’ll just have to incorporate all that awesomeness into CIDER when I’m back.

As some of you probably know, Christophe Grand has joined nREPL’s team and is working on bringing some amazing features from unrepl to nREPL:

(Extremely) Exciting times ahead! I cannot thank Christophe enough for the amazing work he has been doing on nREPL recently! People like him are the heart and soul of the Clojure community and the reason why I love being part of it!

Epilogue

The year had an explosive start for me - so many project release, so much open-source work done! I was really eager to start “clean” and finish many of the initiatives that have been dragging throughout the past year. It was pretty painful and exhausting, but I got to a point where I did everything I felt I needed to do (for the time being) and now I’m looking forward to some respite and re-energizing myself.

While, I’m gone the development won’t stop, of course. In my absence the other CIDER and nREPL Core Team members will be reviewing PRs and cutting releases. I hope you’ll surprise me with many great developments next time I open my GitHub!

Enjoy CIDER Oslo (ir)responsibly! Keep hacking!

  1. https://github.com/clojure-emacs/cider/releases/tag/v0.20.0 

  2. Those are hard to spot and result in out of memory errors in nREPL. 

Permalink

Copyright © 2009, Planet Clojure. No rights reserved.
Planet Clojure is maintained by Baishamapayan Ghose.
Clojure and the Clojure logo are Copyright © 2008-2009, Rich Hickey.
Theme by Brajeshwar.