test2junit Version 1.4.0 Released

This is a bit delayed announcement of the release of test2junit 1.4.0. test2junit lets you “Emit Clojure test output in Junit XML format and optionally automatically invoke HTML generation.”

With some version 1.3.x of test2junit, I added a bit more colorful command line output. What I did not consider back then was that this output may “break” test2junit in cases in which its output is redirected, e.g., when running it in a continuous integration environment.

With version 1.4.0, I added an option to silence the test2junit command line output. To silence the output, set “:test2junit-silent true” in your project.xml.

I hope this update is useful for you and fixes issues for people for which the 1.3.x versions caused trouble. If you have constructive feedback, please let me know.

Permalink

JOB: Software Developer

Location: Anywhere within the U.S.
Target Start Date: May 15, 2018
Salary Range: $82,000 to $130,000 per year (salary offers vary by experience and location-specific cost-of-living adjustment)
Benefits:
Vision, dental, & medical insurance; 403(b) retirement savings plan; generous minimum vacation policy; parental leave; long-term disability; employee assistance program
Level: Multiple levels, mid through senior

At Democracy Works, we believe voting should fit the way we live. To that end, we build technology for both voters and election administrators that simplifies the process and ensures that no voter should ever have to miss an election.

TurboVote, our first service, helps voters register, stay registered, and cast a ballot in every election, from municipal to national. TurboVote signed up its millionth voter in 2016 by building the largest college, nonprofit, and corporate voter engagement coalition in the country, including 176 campuses, companies like Starbucks, Univision, Facebook, Google, Snapchat, and dozens more. Our other work includes the Voting Information Project, whose polling-place data received 123 million impressions in 2016, an Election Technology Cooperative to provide affordable, voter-centered technology to election administrators, and Ballot Scout, which tracks absentee ballots through the mail, providing transparency in the vote-by-mail process and making it easier to follow up when things go awry.

These products are the work of our ten-person developer team. Most of our development involves writing microservices in Clojure running in Docker containers on Kubernetes and hosted on AWS. These services communicate over RabbitMQ and store their data in Datomic. Our users primarily interact with web apps written in ClojureScript and re-frame. We also have projects that use JavaScript, Node, React, Python, and PostgreSQL. We hope you have experience with some of these technologies and are excited to get experience with the rest.

We pair program, collaborate with product managers, and make sure our efforts deliver value to voters. We rotate roles and projects on our team so that everyone gets a variety of experience and working relationships and can bring their unique strengths to as wide a swath of our work as possible.

To apply:
Send a short email with resume, addressed to Chris and Wes, at work@democracy.works with the subject line “Will code for democracy” to begin the application process. Please include how you found this job listing. We also encourage all applicants to state their preferred pronouns when applying for any job opening at Democracy Works. Qualified candidates who meet the above requirements will have the opportunity to complete an anonymized skills evaluation before we schedule an interview. Based on the application, evaluation results, interviews, and reference checks, offers will be made to one or more finalist applicants.

Applications will be accepted and interviews will be conducted on a rolling basis.

Democracy Works is committed to diversity and inclusion in everything we do and aspires to have a team which is representative of the voters we serve. When hiring, we practice proactive outreach to top talent that’s underrepresented in our sector (including Latinx, Black, AAPI, and Indigenous candidates), and we offer every candidate an anonymized skills evaluation, to reduce implicit bias and resume-dependency in our process. We're a woman- and gay-founded startup, and promote an inclusive culture that stands against racism, sexism, homophobia, and ableism (to name a few). To be explicit, we strongly encourage applicants of all races, ethnicities, political party associations, religions (or lack thereof), national origins, sexual orientations, genders, sexes, ages, abilities, and branches of military service. Feel free to contact work@democracy.works if you have any questions about our commitment to inclusion or about general hiring practices.

Permalink

Dutch Clojure Days 2018 round-up

As you may recall, I had enthusiastic expectations and resolutions for Dutch Clojure Days. My first Clojure-only conference, my first proper face-to-face with the community. How could I not be excited?

On Saturday 21st at 8:30 am sharp we were at the TQ building’s reception, greeted by Carlo Sciolla. A couple of words on the venue: a simple but elegant building, close to Dam Square and right in front of a fascinating flower market. The conference happened on the fourth floor, with a balcony to enjoy the outstanding view on the city, and food and drinks for everybody. My first, huge “thank you, DCD!” goes to the vegetarian option which was palatable for a vegan, but let’s keep the cheering and the hand-clapping for the end.

Eleven speakers were waiting for us. Vijay Kiran set the stage and the playful mood of the day, leaving soon room to Alex Yakushev. “Embrace the JVM” was a talk to treasure. Observability, performance profiling, memory inspection. I am by no means a JVM expert, however the tools Alex showed us will definitely help me get a better understanding of the machinery behind Clojure.

Simon Belak was up next talking about transducers and statistical analysis. This was probably the hardest one for me. I haven’t found a way to appreciate the value of transducers yet, and statistical analysis is not my strongest skill. But I still appreciated the concept of sketch algorithms and I will hunt histograms pretty soon.

Srihari Sriraman with “Practical Generative Testing Patterns” blew my mind and, if you fancy ratings and such, was the highlight of the day. We all know test.check is good, but the approach of Srihari to automation, seeding relevant data and testing plausible behaviours left me eager to grab my keyboard and implement something similar.

After lunch we were treated to one more talk before the lightning sessions. Wilker Lúcio explained the beauty and easy-of-use of GraphQL, an interesting alternative to REST for better APIs.

The lightning talks kicked off with some magical REPL-debugging from Valentin Waeselynck. scope-capture looked promising, and I can only hope for an integration with CIDER. Dr Roland Kay reminded us the usefulness of clojure.spec, although if I had to base my opinion of clojure.spec on his talk, it roughly looked like the type-system Clojure is missing. No trolling intended. Thomas van der Veen hit the MQTT broker pedal, mixing Java and Clojure, but I am still not sure I got the purpose of the experiment aside from the sake of learning. Ray McDermott closed the lightning sessions with an amazing browser-driven, multi-user REPL he is devising which can make live pair-programming scattered around the world a breeze.

The last three talks reflected experiences of using Clojure for business. Josh Glover, Philip Mates and Pierre-Yves Ritschard shared with us the journeys of their companies and projects and how designing, developing and testing have only improved since their move to our beloved language.

Drinks followed before a bit of REPL-driven comedy courtesy of Ray McDermott. Suffice it to say we sang the Clojure version of Bowie’s “Rebel Rebel” aptly entitled “REPL REPL”. If you weren’t there, well, you don’t know what you missed.

Dutch Clojure Days left me with the impression that the Clojure community is alive and hard-working, and its heart is in the right place. Ideas flourish, projects boom, boundaries get stretched. We can only be thankful to the DCD stuff for being able to set up such a pleasant event, asking us only to join them and share our passion together.

Permalink

Website routing made simple and easy

I was kidding myself for a long time. I thought of myself as an engineer, I was making complex billion user web apps, I was ENGINEERING the front end and the back end, of course I was. Except I wasn't.

I make websites.

When I finally came around and saw myself how other people see me, I reached website maker enlightenment. I stopped using complex js frameworks, complex backend api tech, I quit graphql, I quit REST APIs, I quit doing stupid stuff, and I started finishing projects.

I want you to finish your projects too, and the first step is to dump those complex frontend frameworks, and maybe embrace the simplicity of clojure.

How exactly is routing made simple or easy. WTF does that even mean?

I'll show you. Here's a route all by itself

[:get "/" home]

Here's two routes

[[:get "/" home]
 [:get "/@:name" profile]]

This is a route in coast on clojure

(ns your-project
  (:require [coast.gamma :as coast]))

(defn home [request]
  {:status 200
   :headers {"Content-Type" "text/html"}
   :body "Hello world!"})

(def routes [[:get "/" home]])

(def app (coast/app routes))

(app {:request-method :get :uri "/"}) ; => Hello world!

Routes in coast are a vector of vectors, the routes that are on top get matched first.

(defn hello [request]
  {:status 200
   :headers {"Content-Type" "text/html"}
   :body "hello world!"})

(defn goodbye [request]
  {:status 200
   :headers {"Content-Type" "text/html"}
   :body "goodbye, cruel world!"})

(def routes [[:get "/" hello]
             [:get "/" goodbye]])

(def app (coast/app routes))

(app {:request-method :get :uri "/"}) ; => hello world!

If you don't want to write vectors all day, you can use coast's route functions

(ns routes
  (:require [coast.router :refer [get post put delete wrap-routes]]
            [coast.responses :as res])
  (:refer-clojure :exclude [get]))

(defn home [request]
  "welcome!")

(defn profile [request]
  (str "hello, " (get-in request [:params :name])))

(def routes (-> (get "/" home)
                (get "/@:name")))

(def app (coast/app routes))

Here's a more complete example for something like auth with buddy

(ns routes
  (:require [coast.router :refer [get post put delete wrap-routes]]
            [coast.responses :as res]
            [controllers.home :as c.home]
            [controllers.users :as c.users]
            [buddy.auth])
  (:refer-clojure :exclude [get]))

(defn wrap-auth [handler]
  (fn [request]
    (if (buddy.auth/authenticated? request)
      (handler request)
      (res/forbidden
        "I'm sorry dave, I can't let you do that."))))

(def auth (-> (get "/users/:id" c.users/show)
              (wrap-routes middleware/wrap-auth)))

(def public (get "/" c.home/index))

(def routes (concat public auth))

Routing in coast on clojure is meant to be easy and meant to be easy to understand. Hopefully I've given you a glimpse into just one aspect of how making websites, not complex over-engineered front end heavy web apps can be made simple and easy.

If you're picking up what I'm putting down, give coast on clojure a try!

Permalink

Bringing order with Clojure's sort-by

It is unavoidable, really.

Any data eventually needs to be sorted for presentation. Most of the times we’re very lucky and we could lean on the implicit order of data returned from the database, or we can decorate that HTML table with DataTables and get sorting for free.

Implicit sorting is a crutch that I’ve relied on many times, and I’m sure you have too.

Implicit sorting in RDBMS systems

What many of us observe when using, say MySQL, is that the rows get returned to us in an order we're familiar with. The insertion order. Using InnoDB tables this generally means ascending by numeric auto-incrementing primary keys. For MyISAM it is strictly insertion order. Other storage engines might have different properties.

Already it is clear that within one RDBMS system we can have multiple behaviours depending on the underlying storage engine used by the table.

Long story short, we have no gaurantees and this can easily nip us in the behind if we're not careful.

Leaving it to the caller

Knowing now that we cannot depend on the source to sort the data for us, it is almost certainly up to the caller to sort the data. This often comes in the form of a query parameter, be it an ORDER BY clause in a SQL statement or a query parameter to an API.

More bespoke sorting tends to happen in the consuming code, not at the source, and this is where we'll look at what Clojure offers us.

Sorting data with Clojure

Clojure is great at taming data of all kinds, here I just want to explore a few ways to get some order to your data using simple functions.

Example data

For the examples below I’m going to be working with some invoice data. Each invoice looks something like this:

{:invoice/number "ACMEINV00001"
 :invoice/date "2016-11-25"
 :invoice/total-before-tax 100
 :invoice/tax 10
 :invoice/items [...]}

I’ll leave it up to you to imagine how rich these data structures can become in a real invoicing system.

Getting started with sort-by

The first requirement could be to sort a vector of invoices by total. These is relatively simple with sort-by:

(sort-by :invoice/total-before-tax invoices)

The first argument to sort-by should be function, and since keywords are functions of maps you can just specify the key in the map to be used. One caveat, the value of the entry in the map must be comparable.

That gets you a new vector, with the smallest invoices first and the valuable ones at the end. Hardly useful for business, so lets flip it around by supplying a comparator function too:

(sort-by :invoice/total-before-tax > invoices)

Now we’re cooking with gas! The most valuable invoices are now at the head of the list! Need the top 10? Just take what you need:

(take 10 (sort-by :invoice/total-before-tax > invoices))

How did this happen? Clojure compared the values returned by :invoice/total-before-tax using the > function.

Sorting by composite keys

The next requirement might be to sort by :invoice/number and :invoice/total-before-tax. Imagine the idea is that when two invoices have the same total that they are then sorted by their invoice number to show some kind of implied order.

This is where our friend juxt comes in. juxt accepts a list of functions and returns a new function, that when called, returns the results of all the original functions in a vector.

(def head-and-tail (juxt first last))
(head-and-tail [1 2 3 4 5]) #=> [1 5]

Here you can see that juxt applied first, and last, to the supplied list of numbers and gave us the head and the tail of the list. Keen readers might have just figured out where I’m going with this.

sort-by can compare these vectors too, so we can sort our invoices like this:

(def total-before-tax-and-number (juxt :invoice/total-before-tax :invoice/number))
(sort-by total-before-tax-and-number invoices)

Now the results will be sorted from lowest value invoice to the highest, with the invoice numbers in order too. So we're halfway there.

Mixing the order (or composing functions)

The results of the previous example doesn't make much sense. How can we combine sorting the invoice amounts in descending order and have the invoice numbers run sequentially when there is an overlap?

One possible solution is to use comp, and make a new function that will return the value of :invoice/total-before-tax as a negative number. comp works by accepting a list of functions and returning a new function, which when called, calls the arguments from right to left and passing the result of the previous call to the next one, starting with the parameter when called.

An example will be worth a thousand words:

(require '[clojure.string :as str])
(def up-and-reverse (comp str/reverse str/upper-case))
(up-and-reverse "elloh") #=> "HELLO"

;; or

(str/reverse (str/upper-case "elloh")) => "HELLO"

In order to get the negative total we can just comp together - and :invoice/total-before-tax like this:

(def negative-total (comp - :invoice/total-before-tax))
(negative-total invoice) #=> -100

If this right-to-leftness of comp bothers you, you could also simply declare it as an anonymous function which wraps a thread-first functional pipeline: #(-> % :invoice/total-before-tax -).

And using our new friend juxt we can simply roll it up like this:

(def negative-total-and-number (juxt negative-total :invoice/number))
(sort-by negative-total-and-number invoices)

And now we have a list of invoices sorted by total from most to least valuable, and where the totals match the invoice numbers follow a progression.

Another variation would be to sort by number of items on an invoice. This can be achieved by composing count and :invoice/items together:

(sort-by (comp count :invoice/items) > invoices)

And you'll have the invoice with the most items in at the head of the list.

Wrapping up

Although my examples are a bit contrived, the power that comes from composing functions in these intuitive ways are nearly endless. This works equally well for predicate functions used by filter, remove and many others.

It seems many small and composable functions will end up serving you better in the long run!

Thanks

A big thanks for Robert Stuttaford for helping to review this post

References & further reading

Cover image by Willi Heidelbach — Creative Commons Attribution-Share Alike 3.0 Unported — Wikipedia

Permalink

Passing around components with reagent and Semantic UI

I've been happily using Semantic UI React since I first wrote about it more than a year ago. Everything in the previous post still holds true, the only thing that has changed is the version number of the semantic-ui-react package.

The CLJSJS community has been great at keeping things up to date, and I can't recall any breaking changes in the components that I've been using.

One thing I unknowingly skipped in the previous article was passing around React components as arguments for other components. I don't think I realized at the time it was possible. I was learning the minimum viable React through re-frame, which got me very far (and continues to do).

This oversight was noticed by others though. A few people reached out for advice in private, and on StackOverflow. Where I fell short other community members helped out in the Clojurians Slack.

The Problem

Looking at the tabs example in the Semantic UI React docs, you encounter this:

import React from 'react'
import { Tab } from 'semantic-ui-react'

const panes = [
  { menuItem: 'Tab 1', render: () => <Tab.Pane>Tab 1 Content</Tab.Pane> },
  { menuItem: 'Tab 2', render: () => <Tab.Pane>Tab 2 Content</Tab.Pane> },
  { menuItem: 'Tab 3', render: () => <Tab.Pane>Tab 3 Content</Tab.Pane> },
]

const TabExampleBasic = () => (
  <Tab panes={panes} />
)

export default TabExampleBasic

The same thing occurs in several places, including popups.

So the question really is how do we pass along our reagent component along as a React component?

The Widget

Sticking with my tradition of building over engineered GitHub widgets, here are some tabs in action:

The source code is available on GitHub at kennethkalmer/re-frame-semantic-ui-react-github-tabs.

Here is a truncated version of the tabs in the above video:

(ns github-repo-widget.views
  (:require [reagent.core :as reagent]
            [re-frame.core :as re-frame]
            [github-repo-widget.events :as events]
            [github-repo-widget.subs :as subs]
            [github-repo-widget.ui :as ui]))

(defn- readme-tab []
  (let [loading? @(re-frame/subscribe [::subs/repo-readme-loading?])
        readme   @(re-frame/subscribe [::subs/repo-readme])
        pane     (ui/component "Tab" "Pane")]

    [:> pane {:loading loading?}
     [:div {:dangerouslySetInnerHTML {:__html readme}}]]))


(defn- stats-tab []
  (let [loading? @(re-frame/subscribe [::subs/repo-info-loading?])
        pane     (ui/component "Tab" "Pane")]

    [:> pane {:loading loading?}
     ;; ...
     ]))


(defn- repo-tabs []
  (let [panes [{:menuItem "Readme"
                :render #(reagent/as-component [readme-tab])}
               {:menuItem "Stats"
                :render #(reagent/as-component [stats-tab])}]
        tab (ui/component "Tab")]

    [:> tab {:panes panes}]))

The solution

Reagent gives us reagent.core/as-component, which is exactly the interop we need to turn our Reagent component into a React component for these cases.

In the case of the tab panes, Semantic UI React expects a function that returns a component as a value for the render property. In other places it expects the component directly, as seen with popups:

(defn info-icon
  ([message]
   (info-icon {} message))

  ([options message]
   (let [popup (component "Popup")
         icon  (component "Icon")]

     [:> popup
      {:trigger (reagent/as-component [:> icon
                                          (merge {:name "info"}
                                                 options)])}
      " "
      message])))

Here the trigger property expects another component, not a function that returns the component.

In close

Being able to use Semantic UI React directly in ClojureScript without going through some insane incantations or rituals is a testament to the amazing work done by the Reagent contributors.

It is also a testament to Clojures pragmatic approach of embracing the language/environment that hosts it.

Permalink

Nothing public.

Nothing public. Just use (defn ^:export my-fnc []) to expose to JS. Check our Mori from David Nolen. I can always hope on Zoom if you have specific questions.

Permalink

test-doubles: A small spying and stubbing library for Clojure and ClojureScript

As you may know from a previous post I’m working for GreenPowerMonitoras part of a team that is developing a challenging SPA to monitor and manage renewable energy portfolios using ClojureScript.

We were dealing with some legacy code that was effectful and needed to be tested using test doubles, so we explored some existing ClojureScript libraries but we didn't feel comfortable with them. On one hand, we found that some of them had different macros for different types of test doubles and this made tests that needed both spies and stubs become very nested. We wanted to produce tests with as little nesting as possible. On the other hand, being used to Gerard Meszaros’ vocabulary for tests doubles, we found the naming used for different types of tests doubles in some of the existing libraries a bit confusing. We wanted to stick to Gerard Meszaros’ vocabulary for tests doubles.

So we decided we'd write our own stubs and spies library.

We started by manually creating our own spies and stubs during some time so that we could identify the different ways in which we were going to use them. After a while, my colleague André Stylianos Ramos and I wrote our own small DSL to create stubs and spies using macros to remove all that duplication and boiler plate. The result was a small library that we've been using in our ClojureScript project for nearly a year and that we've recently adapted to make it work in Clojure as well:

I’m really glad to announce that GreenPowerMonitor has open-sourced our small spying and stubbing library for Clojure and ClojureScript: test-doubles.

In the following example written in ClojureScript, we show how we are using test-doubles to create two stubs (one with the :maps option and another with the :returns option) and a spy:

We could show you more examples here of how test-doubles can be used and the different options it provides, but we’ve already included a lot of explained examples in its documentation.

Please do have a look and try our library. You can get its last version from Clojars. We hope it might be as useful to you as it has been for us.

Permalink

Capital One shutdown Level Money last year.

Capital One shutdown Level Money last year. It’s doubtful any of the code was brought to other projects.

Using the package approach with JavaScript interop is still a good option to start working clojurescript into an organization without having to get the entire company to align on a tech stacks.

Since Triforce, I’ve built another app that’s all Clojurescript in React Native and AWS Lambdas.

Permalink

A Typed-Macro Writer’s Toolkit

This is a transcript of a PL Wonks lightning talk.


Ok. So this talk is going to be about providing a toolkit to macro writers in Clojure, to be able to communicate to the type system.

Here’s the setting. On the left, we have the type system. It finds macros it needs to expand, but it has no idea how these macros work — all the type system has is the ability to expand macros, and then make a best-effort on type checking the expansion. On the other side, we have the macro authors, who, sure, are willing to teach the type system how the macro actually works, but, currently, there’s no way to actually do that.

So in this talk, I’m proposing that we provide an extensible interface to Typed Clojure’s internals to help it be more expressive and usable.

And how can Typed Clojure be made more expressive and usable? Well, by provide macros that yield better static type error messages, macros that require less type annotations, and macros that are optimized for type checking performance, rather than runtime performance.

So first, let’s talk about how we can write macros that yield better static type error messages.

Let’s look at this simple when macro — it’s like an if but the else branch is implicitly nil. So let’s imagine we’re trying to type check the expanded if form as a Number.

And we’ve found that the else branch is actually nil. It should be a Number.

So, what we’d like the type system to do is blame the original when expression.

But what it actually does is blame the original when macro — basically blame the macro-writer. The type system needs more information as to who to blame.

The solution here is to allow macro-writers to provide blame labels to the type system. For example, in the else branch here, the red box symbolizes an annotation the macro-writer can use. It says, if the else branch does not conform to the expected type, then blame the entire when expression.

Next, to argue that an extensible interface to Typed Clojure’s internals helps make it more expressive and usable, we’re going to look at how we can make macros require less type annotations.

So here’s a for comprehension. It increments '(1 2 3) to '(2 3 4). You’ll notice there’s two annotations — one for the input, and one for the output.

How might we eliminate the output annotation, knowing the output of this for comprehension is very complicated, with nested loops and local mutation.

To set the stage, let’s say the expected type for this for comprehension is a sequence of symbols. That means, the surrounding program expects it to be sequence of symbols — clearly it’s a sequence of numbers, so some type error is going to happen. But, who’s going to be blamed?

Let’s imagine the expansion is a bunch of code, with an increment form somewhere in it.

What we really want to do is propagate this Sym expected type into this increment form, because this will correctly blame the responsible user-written code.

And, unsurprisingly, our solution is exactly this: give the macro writer the ability to take an expected type from a type checking run, deconstruct it, and then propagate it to where error messages will be optimized.

And finally, providing an extensible interface to Typed Clojure’s internals helps us write macros that yield simpler checks.

For example, here’s our for comprehension. It doesn’t have any annotations. How can we get away with this?

Well, we just have a simplified expansion: instead of hundreds of lines of complex code, we have a single function call. Now, we basically have two macros: one optimized for runtime performance, and one optimized for type checking performance.

In an optional type system like Typed Clojure which does not optimize programs, we can keep the runtime-optimized expansion and throw away the type-checking optimized expansion (after we’re done). Clojure programs routinely reload code (and therefore reexpand macros), so this might be a feasible approach for Typed Clojure — perhaps less-so for systems like Typed Racket.

So finally, to conclude, we’re providing an extensible interface to Typed Clojure’s internals to help it be more expressive and more usable.

And how is it more expressive and more usable? Well, we can write macros that have better errors, macros that require less annotations, and macros that expand to code that is faster to type check.

Thanks!

Permalink

Copyright © 2009, Planet Clojure. No rights reserved.
Planet Clojure is maintained by Baishamapayan Ghose.
Clojure and the Clojure logo are Copyright © 2008-2009, Rich Hickey.
Theme by Brajeshwar.