What is the onion architecture?

Part 2 of the functional architecture series. When we’re structuring our functional software, we want to isolate the actions from the calculations. We can do that using the Onion Architecture, which has layers like an onion. The center of the onion is your domain model, then around that are your business rules. Finally, around that is your interaction layer, which talks with the outside world, including the database, web requests, api endpoints, and the UI.

Transcript

Eric Normand: What is the onion architecture? Hi, my name is Eric Normand. These are my thoughts on functional programming.

This is part two of a three-part series I’m doing on functional architecture. What does functional programming have to say about architecture? We went over the first thing, which was the stratified design. This is where you build layers of meaning on top of more fundamental layers of meaning. You’re building meaning up.

This onion architecture also has layers. That’s how onions are used for these layer metaphors, but these are in circles. Instead of top-down you’re going outside in. How does this thing work?

In a functional onion architecture, you’re going to have really three layers. The inner layer is where you put your domain model. This is a functional implementation of your domain model. By functional I mean its calculations and data, pure functions and immutable data. This is how you represent the ideas in your domain.

If you’re doing accounting software, this is where you implement the rules of accounting. How do I define a transaction? What are the operations I can do on transactions? All that goes in your domain model.

The second layer is where you put your business rules. If you’re an accounting business, and you’re developing software to help you run your business, there’s going to be rules that aren’t really set by how accounting systems work like everyone uses the same double entry bookkeeping accounting system.

How much money do you charge for an overdraft on an account? How much is the monthly fee on an account? How long do you let an account stay open before you close it if there’s no activity on it? These are rules that businesses like banks and accounting firms have that aren’t really accounting, but they’re part of their business. These are things that change a lot, a lot more than accounting, that is.

Accounting rules have been around for hundreds of years, but every business has different policies about how they deal with late fees and things like that. Those go in this other layer. Just like in the stratified design, you’re putting stuff that changes more frequently on an outer layer. Inner layer, stuff doesn’t change.

Now, these business rules are also pure functions with immutable data. We haven’t done any actions yet. They’re built on top of the accounting rules. They can use the concepts inside the domain model.

Third layer is where you start doing actions. This layer is interacting with the outside world. This layer is where you’re going to put your web server that converts a web request into either business rule, operation or a domain operation. It’s going to pierce through the layers and operate on the model.

That’s going to have some effect again on the outside world. That’s going to bubble out. It’s all pure on the inside. Once it starts bubbling out, this outer interaction layer is going to start doing stuff. It’s going to start sending emails. It’s going to store stuff somewhere. It’s going to make API requests. It’s going to respond to the web request.

Let’s look at how this onion architecture does some very common operations. We already talked about the web request. I’m going to do it again, but I’m going to put it off. Before we do that, let’s look at the persistence layer. This is the database.

A lot of apps are built with the database at the center. The onion architecture is built with the database on the outside, because the database…This is what architecture is about. Architecture is about isolating yourself from your mistakes, isolating the mistakes, not yourself, isolating the different mistakes that you could make from each other.

You have to make a lot of decisions when you’re building software. What database do we use? What language do we choose? How do we process a certain message? Do we need a cue? Can we do it as a monolith, or as micro services? There’s all these questions. All of them might be wrong. Certainly, some of them are going to be wrong. They can’t be right all the time.

We want to isolate those changes so that for instance, if I made a mistake in database, I don’t have to do a whole rewrite. I’ve done so much right. I made one mistake. Do I really have to start over? I’m sorry, this light is really bad on the video.

Do I really have to start over? It seems unreasonable. Then a lot of systems you do the database, the queries and the SQL statements are all smeared all over the code. You haven’t isolated it. If you ever wanted to change the database, let’s say, you go from SQL to no SQL, oh man, you’re going to have to go every line of code inspected.

Did I make an assumption here about the database? What’s going on? You want to just move all that out because the database is something you can make a mistake on. It’s much more likely that you choose the wrong database, than you choose the wrong accounting system, the accounting process.

The accounting process, it’s been around for hundreds of years. It’s not something that there’s any real debate on anymore. How do you do this? Database is on the outside. It means when your system decides something needs to be stored, this is a business rule that says, “I need to store this. That this check came in.”

The business rules generate some representation in immutable data of that storage. In [inaudible 8:31] world, they call this a command object. It’s just some representation of this thing that needs to be stored. When that bubbles out to the outer layer, the interaction layer, something grabs that and says, “I know how to store that,” and puts it in the database. That’s it.

The business rules decide what gets stored, but they don’t do the storing. This is an example of a pattern that I’m going to have, actually, I have this on my to-do list, an episode about this. It’s a dumb name, so I might change the name. I’m calling it the plan then act pattern. In plan then act, you make a plan, and then you act.

Usually, we write software in an imperative way. We write software that makes decisions as we go. You loop through all the checks. You get a query from that. You query the database. You get all the checks. You loop through all of them. You figure out which ones inside the for-loop.

You figure out which ones need to send an email about. You don’t know how many emails you’re going to send until you get to the end of the loop, and then you’ve sent them already. That’s how imperative programming tends to work. You don’t have to do it that way, but that’s a very common pattern.

In the plan then act situation, what you do is you have a pure function that takes all of the checks, and the immutable data. It goes through those checks and makes a plan, I need to send this email, this email, this email, this email, and whatever. Now you have a list of emails, also immutable data. That’s a pure function from checks to emails.

Then you have your outer layer, your interaction layer, that’s what does the query, it calls this business rule that determines the plan for what emails to send. It does a query, it gets immutable data. This is the current state of the checks that we have.

It calls that pure function that’s in the business rule layer. It returns a list of emails to send. Then the interaction layer iterates through that and sends them, but there’s no decisions anymore. There’s no decisions to be made at that point. It’s simply acting.

You’ve extracted all of the decisions out into your plan. That’s the idea of the plan and then act instead of acting and planning intertwined. That’s how persistence happens. The business rules will bubble out a plan, and part of that plan is what needs to get stored.

How does a web request happen? Well, the interaction layer has a web server with an open port, etc. The web request comes in, it converts that into some kind of request that the business rules understand. This is a transfer from one account to another. It represents that as data, pure data.

Then it calls the business rules for that. The business rules will return, well this is the new state that has to be stored, or now we have to send an email. It’s just a plan for what to do. That interaction layer is just going to iterate through the plan and do it.

It’s actually dumb, and that’s a good thing because you want the business rules to decide what happens. You don’t want the email server to decide what happens, the email system.

This is part two. I’m going to talk about part three, in the next one, more architecture. The reason I only have three is because I think only have three thoughts on functional architecture. I’d love to have more, so please, if you know something, I would greatly appreciate it.

I’d love to get into the discussion with you. On Twitter, I’m @ericnormand. On LinkedIn, just search for Eric Normand. Or on email, if you want some longer form discussion, I love that too, I’m eric@lispcast.com.

Subscribe, if you’re on YouTube, you can get notified of new videos if you subscribe and click the little alarm, the little bell icon. Please subscribe on podcast. Tell your friends about it. Awesome, thanks so much. See you next time.

The post What is the onion architecture? appeared first on LispCast.

Permalink

Ontology/Knowledge Engineer

Ontology/Knowledge Engineer

Reify Health | Boston, MA
remote
Clinical trials keep getting harder. We need to work smarter.
$100000 - $150000

At Reify Health, we are building a more creative healthcare system. We envision a world where every potential therapy, if safe and effective, is available to the patients who can benefit.

Our healthcare system relies on clinical trials to develop new, potentially life-saving treatments for patients. But clinical trials continue to be slow, unpredictable, and expensive. Reify Health’s product helps both the research leaders driving forward clinical trials and the doctors and nurses who care for the patient participants.

As we scale adoption of our product domestically and internationally, we will accelerate world-class clinical research and unlock innovation. By joining our data team, you will play a foundational role in supporting this mission by organizing Reify’s growing datastreams under a unified ontology to support the development of probabilistic inference systems and a host of other intelligent features.

Your Responsibilities

  • Work with the data team to organize growing temporal/atemporal data into structured knowledge and integrate this knowledge into a customized clinical ontology relevant to Reify Health’s sites and sponsor stakeholders
  • Develop systems to support the maintenance and expansion of our integrated ontology including, but not limited to, a query engine, API layer, and monitoring system
  • Design and develop systems to support supervised and/or unsupervised knowledge discovery and classification Integrate knowledge systems with a growing Kappa-style data architecture and existing application architecture supporting StudyTeam
  • Support the integration of knowledge architecture into direct/probabilistic inference engines which support intelligent recommendations and insight generation for clinical sites and sponsors
  • Become intimately familiar with HIPAA, GDPR, and other applicable regulatory frameworks and how they influence our architecture and development decisions
  • Become comfortable working across a stack which uses Clojure, Python and (Postgre)SQL
  • Frequently communicate your efforts to Data Team Lead and other technical/non-technical stakeholders (product and clinical teams) in clear written, verbal, or presentation form
  • Live our data philosophy, which focuses on ethical decision making, being aware of how biased data (and assumptions) can affect results (and people), and being laser-focused on business needs

What Will Make You Stand Out

  • A history of practical application in real-world systems (rather than theoretical) and the ability to make intelligent trade-offs in a business context (rather than purely technical)
  • Deep familiarity with existing relevant ontologies and taxonomies in the clinical and/or medical space, such as EPIC, UMLS, SNOMED, MedDRA, ICD-10, etc. so you are informed about which individually/in combination (if any) are suitable for our needs
  • Familiarity with modern semantic standards
  • Previous practical experience and/or familiarity with technologies similar to OWL, PR-OWL, SPARQL, and graph databases
  • Previous experience developing and growing knowledge systems and integrated ontologies in a rapidly growing startup
  • Expertise in graph algorithms, probabilistic inference/logic, Bayesian methods
  • M.S./PhD in ontology engineering, knowledge engineering, or a related field
  • Understanding and experience with distributed systems
  • Relevant published work (academic, blog posts, open-source contributions)
  • Previous experience with functional programming languages/philosophy (or existing Clojure chops!)
  • Experience in a startup environment (as a remote employee using video/chat collaboration tools, if you’d like to work remotely)
  • Relevant experience in a healthcare/health-tech company

Compensation & Benefits

  • Competitive Salary and Stock Options: Compensation varies from mid-level to very senior and is commensurate with your experience.
  • Comprehensive Health and Wellness Coverage: 100% premium coverage for you (and >50% for your dependents) for: a top-tier health plan covering you in all 50 states (with option of HSA for medical expenses and as investment vehicle) dental, vision, disability (short-term and long-term), and basic term life insurance (for your entire tenure at Reify). We enable 24/7 access to doctor by phone or online via telemedicine coverage.
  • Company-provided Workstation: You have the option of getting a brand new Macbook Pro (or similar, if desired) laptop if you’d like to use a separate computer for work.
  • Location Flexibility & Transportation: For those working out of Boston, we provide: a free monthly public transportation pass (and are located 2-3 minutes from South Station); unlimited coffee, infused water, and more (provided by WeWork); flexibility to work from home as needed. For those working remotely: you can work from anywhere in the U.S. compatible with an EST work schedule. Additionally, we’ll fly remoters in for our quarterly “remoters’ week”, filled with fun activities, good food, and many opportunities to get to know your colleagues better.
  • Vacation and Holiday Flexibility: Generous paid-time-off policy that accrues with your tenure at Reify which includes holiday flexibility and parental leave

We value diversity and believe the unique contributions each of us brings drives our success. We do not discriminate on the basis of race, religion, color, national origin, gender, sexual orientation, age, marital status, veteran status, or disability status.

We also completely eschew the “bro culture” sometimes found at startups.

Note: We are currently only considering US citizens or Green Card holders. Thanks!

Permalink

Data Analytics Engineer

Data Analytics Engineer

Reify Health | Boston, MA
remote
Clinical trials keep getting harder. We need to work smarter.
$100000 - $150000

At Reify Health, we are building a more creative healthcare system. We envision a world where every potential therapy, if safe and effective, is available to the patients who can benefit.

Our healthcare system relies on clinical trials to develop new, potentially life-saving treatments for patients. But clinical trials continue to be slow, unpredictable, and expensive. Reify Health’s product helps both the research leaders driving forward clinical trials and the doctors and nurses who care for the patient participants.

As we scale adoption of our product domestically and internationally, we will accelerate world-class clinical research and unlock innovation. By joining our data team, you will play a foundational role in supporting this mission by building and expanding powerful internal reporting and analytics tools that allow all Reify teams to benefit from rich data-driven decision making.

Your Responsibilities

  • Work closely with internal customer, product, and engineering teams to design, develop, iterate, and expand internal reporting and analytics tools
  • Act as a mentor and guide to internal team members learning to build/expand their own analytics dashboards using SQL, Python, etc.
  • Develop and support processes for securely and consistently organizing reports and other analytical tooling used by internal teams
  • Work with the data team to expand architecture to support and integrate additional data sources or data streams that would enrich internal analytics
  • Prototype advanced reporting and analytical features to support efforts of internal teams (and potentially evolve into customer-facing features)
  • Work with data team to develop systems/processes to support analytical tooling and reporting as our company rapidly scales
  • Become intimately familiar with HIPAA, GDPR, and other applicable regulatory frameworks and how they influence our architecture and development decisions
  • Become comfortable working across a stack which uses Clojure, Python and PostgreSQL (though you should already have extensive SQL, PL/pgSQL, PL/SQL experience)
  • Frequently communicate your efforts to Data Team Lead and other technical/non-technical stakeholders in clear written, verbal, or presentation form
  • Live our data philosophy, which focuses on ethical decision making, being aware of how biased data (and assumptions) can affect results (and people), and being laser-focused on business needs

What Will Make You Stand Out

  • Extensive experience supporting teams using/integrating with analytics ‘dashboard’ systems such as Tableau, Periscope Data, and Mode Analytics (or Jupyter-notebook based systems)
  • Deep comfort with SQL (particularly PostgreSQL, PLpg/SQL) and understanding of performance concerns
  • Comfort with statistical and analytical Python libraries such as scipy, pandas, numpy, keras, nltk, etc. (or R equivalents)
  • Interest or previous experience with Clojure or other functional languages (most of our application is written in Clojure/Clojurescript)
  • Understanding of how to detect and address correctness or accuracy issues in analytical code
  • Deep familiarity with AWS ecosystem, Kubernetes, Kafka
  • M.S./Ph.D. in computer science, applied statistics, or other relevant field
  • Understanding and experience with distributed systems
  • Relevant published work (academic, blog posts, open-source contributions)
  • Previous experience with functional programming languages/philosophy (or existing Clojure chops!)
  • Experience in a startup environment (as a remote employee using video/chat collaboration tools, if you’d like to work remotely)
  • Relevant experience in a healthcare/health-tech company

Compensation & Benefits

  • Competitive Salary and Stock Options: Compensation varies from mid-level to very senior and is commensurate with your experience.
  • Comprehensive Health and Wellness Coverage: 100% premium coverage for you (and >50% for your dependents) for: a top-tier health plan covering you in all 50 states (with option of HSA for medical expenses and as investment vehicle) dental, vision, disability (short-term and long-term), and basic term life insurance (for your entire tenure at Reify). We enable 24/7 access to doctor by phone or online via telemedicine coverage.
  • Company-provided Workstation: You have the option of getting a brand new Macbook Pro laptop if you’d like to use a separate computer for work
  • Location Flexibility & Transportation: For those working out of Boston, we provide: a free monthly public transportation pass (and are located 2-3 minutes from South Station); unlimited coffee, infused water, and more (provided by WeWork); flexibility to work from home as needed. For those working remotely: you can work from anywhere in the U.S. compatible with an EST work schedule. Additionally, we’ll fly remoters in for our quarterly “remoters’ week”, filled with fun activities, good food, and many opportunities to get to know your colleagues better.
  • Vacation and Holiday Flexibility: Generous paid-time-off policy that accrues with your tenure at Reify which includes holiday flexibility and parental leave

We value diversity and believe the unique contributions each of us brings drives our success. We do not discriminate on the basis of race, religion, color, national origin, gender, sexual orientation, age, marital status, veteran status, or disability status.

We also completely eschew the “bro culture” sometimes found at startups.

Note: We are currently only considering US citizens or Green Card holders. Thanks!

Permalink

Solve the problem at hand

Imagine being at the interview where the interviewer asks you to convert a binary string into a number.

“Please write such f() so that e.g. f("101") === 5.”

I would start writing something like this:

function char_to_int(char) {
  switch (char) {
    case "0": return 0;
    case "1": return 1;
  }
}

“Wait-wait-wait,” he would stop me. “What if string is not binary?”

“Oh, so you want me to convert decimal string? Is it not binary?”

“No, it still is. But, you know, in the future…”

So he wants me to write generic code that still solves the specific problem. It might be ok for an interview, but in real engineering work, I would always prefer concrete code to generalized one.

For one, it’s faster to write. If you only solve a problem for specific cases you usually have a very limited amount of code paths to think of. It’s simpler to write.

It’s easier to get right too. Because you already know the problem. And you know it because you have it at hand, and you can dig as deep as you need, learn all the details you need and make the right call. Whether with a generalized version you’ll be speculating about what problems you might have in the future. You might have them. You might not. You might have a different problem, or you might misjudge the details.

Concrete is also easier to read. The code is filled with clues, concrete details that tell you the specifics of the problem that is being solved here. You see helpful constants (like 0 and 1 in the example above). You see branch conditions that make sense. You see a complete list of possible states instead of abstract opaque “collections” that could contain everything. A generalized code is usually one step above that, so you’ll have to imagine concrete values before you could go through the code in your “mental debugger” (that’s how I read the code anyway).

But what if your prediction was right and you actually will need generalized version soon? Well, you could write it then. It’s the requirements that make writing good code hard, not the act of coding itself. And I doubt you’re good at predicting requirements. Nobody is. So you’ll just spend more time and create more obstacles for yourself in the future.

That’s what the Clojure ecosystem has taught me. Everything is as concrete as possible. Clojure just has no facilities to overgeneralize. It’s either a concrete thing or nothing. Get to the meat as fast as possible. Cut to the chase. If you need to print, just print, don’t invent printing facilities. If you have a record, put in fields you know about and don’t worry about the rest. Add them when you need them.

The advice is simple: don’t solve a problem you don’t have. Be rather afraid to create obstacles for the future you. You can’t see the future, but you can make the present as simple as it could be.

Permalink

Moving to deps.edn and shadow-cljs

Last summer I moved boodle from Figwheel to Figwheel Main. I was pretty satisfied with this setup, but the rise of shadow-cljs in the Clojure community has been tickling my curiosity over and over again. While configuring shadow-cljs I took the chance to set lein aside and finally explore clj and deps.edn, another thing on my to-do list that has been waiting for me for some time.

Let’s start with deps.edn:

{:deps
 {
  ;; … Clojure dependencies …
 }

 :paths ["src/clj" "resources"]

 :aliases {:run {:main-opts ["-m" "boodle.core"]}
           :test {:extra-paths ["test/clj"]
                  :extra-deps {lambdaisland/kaocha {:mvn/version "0.0-266"}}}
           :outdated {:extra-deps {olical/depot {:mvn/version "1.4.0"}}
                      :main-opts ["-m" "depot.outdated.main"]}}}

One thing worth of notice here is :paths. I included the resources directory in it because I serve it via compojure/http-kit and that’s where I put the static assets used in my main index page. This solution works when I run mount.core/start via REPL and when I use the :run alias from the command line.

Speaking of aliases, they are invaluable. With :run I can use clj -A:run instead of lein run, and with clj -A:outdated -a outdated I can easily check for outdated dependencies in my deps.edn thanks to depot. Unit tests are now handled by kaocha, and as per its instructions I have a bin/kaocha file which picks up the :test alias and runs all my tests.

I love the simplicity of deps.edn, it feels like a de-cluttered project.clj.

Configuring shadow-cljs was a bit more complicated. It took me a while to understand how to properly integrate it with CIDER. However, the manual is great and the fine people on Slack’s #shadow-cljs (thanks Ryan Haywood and Thomas Heller!) made it all nice and smooth.

This is my shadow-cljs.edn:

{:source-paths ["src/cljs"]
 :nrepl {:port 8777}
 :dependencies [
                 ;; … ClojureScript dependencies …
               ]
 :builds {:boodle {:target :browser
                   :output-dir "resources/public/js"
                   :asset-path "/js"
                   :modules {:main {:entries [boodle.core]}}}}}

Basic setup, straight from the manual pages. Notice that :output-dir refers to the same resources directory configured in deps.edn. This tells shadow-cljs to place the build output exactly where I need.

The other important change is in the CLJS libraries I use. shadow-cljs does not support CLJSJS, so I had to install pikaday and moment via yarn.

$ yarn add pikaday moment

Since boodle relies on re-frame, I had to install three other libraries to make shadow-cljs build my code happily:

$ yarn add react react-dom create-react-class

In the namespace where I use them, the :require had to be changed:

(:require ;; … other requires …
          ["react-dom" :refer [findDOMNode]]
          ["pikaday" :as pikaday])

And the code had to be adapted:

(let [default-opts {:field (findDOMNode this)
                    ;; … other options …
                   }
      instance (pikaday. opts)]
    ;; … other ClojureScript code …
)

However, a project setup is not ready unless I can work on it via Emacs. These are the steps I made to have a working CLJ/CLJS development environment. When this issue will be fixed some of it may become unnecessary.

First, I added a couple of dependencies in shadow-cljs.edn:

:dependencies [;; … other dependencies …
               [cider/cider-nrepl "0.18.0"]
               [refactor-nrepl "2.4.0"]]

Then I removed everything Figwheel Main related from the .dir-locals.el file located in the root directory of boodle:

((nil
  (cider-ns-refresh-before-fn . "mount.core/stop")
  (cider-ns-refresh-after-fn . "mount.core/start"))
 (emacs-lisp-mode
  (flycheck-disabled-checkers . "emacs-lisp-checkdoc")))

I ran cider-jack-in-clj to have a CLJ REPL and check I didn’t break something on the Clojure side of boodle.

From a terminal, I used shadow-cljs watch boodle to have shadow-cljs build boodle and offer me a nice nREPL server listening on port 8777, the one I specified in shadow-cljs.edn.

Back in CIDER, cider-connect brought up a REPL connected to shadow-cljs nREPL server, and from there I used (shadow/nrepl-select :boodle) to switch to a CLJS REPL. Connecting to boodle via the browser finalized the connection between my code and the CLJS environment.

A great journey. Once again, boodle proved fertile ground to understand the Clojure ecosystem and play with it to learn something new. If you want more details, the project is on my GitHub.

Permalink

Wash Your Hands

I was listening to an interview with Seth Goden where he discussed what it means to be a professional. The example he used was surgeons preparing for surgery. Surgeons are expected to scrub their hands every time before surgery. Part of being a professional surgeon is going through the hand washing procedure.

How could washing your hands be described as a “procedure” or as something that requires “proper technique” and a “systematic approach”? Check out this description of hand scrubbing. It has:

  • more than 10 steps
  • is timed
  • expected setup beforehand
  • expected support from teammates
  • rules to be observed
  • formal guidelines and recommended practices from professional organizations

The amount of work that goes into hand washing is way more than I would have ever thought. And this happens for every single surgery.

Part of being a professional surgeon is going through this process. Every. Single. Time.

Under a tight deadline?

Doesn’t matter, wash your hands.

Think your hands are probably clean enough?

Doesn’t matter, wash your hands.

It’s just a quick, minor surgery?

Doesn’t matter, wash your hands.

Professional Developers

For those of us that call ourselves or aspire to be professional developers, what is our equivalent of washing our hands? What do we always do, without exception, before we start typing?

Surgeons wash their hands to ensure cleaningless; what do developers need to be sure of?

I suggest that we need to be sure we’ve thought properly about a problem. Specifically, we need to discipline ourselves to think and design before and apart from typing.

And I think a solution is very simple: we need to write more.

The best way I’ve learned was really modeled well by my friend and old coworker Tim Pote. He put together a simple template that we filled out whenever we started working on a problem. If you were starting something new (feature, architecture, rewrite, etc), it was expected that you’d fill out the template.

The template asked some simple questions that encouraged us to think through problems.

Simple Template

Problem

What is the problem you’re solving? Not what is the feature, not what are you going to do. What is the problem, why is it necessary, who is it for?

Attempting to answer just this question provides so much value. You should be able to clearly articulate:

  • what the issue is
  • who is having the issue
  • who would benefit if this issue was solved
  • what the benefit would be

If you can’t answer all of those questions, you’re probably solving a non-problem. At best, you’re trying to solve an actual problem but don’t have enough understanding to provide a real solution.

In practice, this step is often very hard because the problem is ambiguous and hard to piece apart. But often, the process of working through this step helps clarify the actual problem.

Goals

What goals, if achieved, would solve the problem?

The difference between the Problem and Goals are important. Tim provided further clarification:

It’s important to note the distinction between problems and goals. Broadly speaking, a problem impacts people outside of the team. A goal impacts people inside the team. This is not strictly true, however, if we’re addressing problems for people on the team, we should explicitly state that. This distinction forces us to tie our solutions to people’s needs while still acknowledging that we can only address people’s needs to varying degrees.

For example, a business owner may want to understand how much they should spend on advertising. This may translate to a variety of goals such as, “allow users to determine the life time value of a customer,” and, “create a workflow for determining profit.”

Design Approach

How will you go about achieving the goals? What’s required? Why this approach?

Alternate Designs

It’s not a tradeoff if you don’t have more than one option (paraphrasing Rich Hickey). The first design may be the best, but there should be a discussion of tradeoffs and why one approach is better than the other.

Constraints

What constraints surround the problem and proposed solution? Besides forcing us to think through the limits around the problem and solution, it helps to have this documented for the future when we’re trying to understand why certain decisions were made.

Risks

Thinking about risk is especially useful for communicating digestable information to stakeholders, helping them to make better decisions.

Timeline

We found that, once we thought through designs, we were able to provide surprisingly accurate estimates. Even when we were significantly off in estimates, stakeholders weren’t totally caught off guard if the risks were communicated.

The REPL & Exploration

The point is not to set a hard rule of no typing until the design is complete. The REPL is a great tool for exploration, and we’d be foolish to throw it away. The key is to be clear and disciplined about what we are doing. Exploring, testing out ideas, etc. at the REPL should be treated as input to the design process. It should not be considered part of the final solution, which is the mistake often made. This requires discipline because code, once written, has the habit of hanging around and creeping into production.

Summary

We weren’t perfect when we started, and I’m still not disciplined enough in my practice. But this approach is amazingly helpful. Even when write ups were done only for individual work, with no intention of sharing, it made solving problems much easier.

Truthfully, this is hard. But the work has to be done, and in fact we do most of it already. The problem is we make implicit decisions (e.g. we accept risks without acknowledging them) and we simply “design” as we go.

Here’s the secret: By the time we sit down to type, most of the hard work should already be done.

The exact sections and breakdowns aren’t vital, but a process for systematically thinking through problems is required. Anything less, and I think we’re just pretending when we call ourselves professional developers.



Bonus: This approach is useful for thinking in general. Here’s my thought process for this blog post.

Problem

Developers waste time solving problems that don’t need to be solved and creating brittle solutions. If we were disciplined to think first, we’d provide better solutions for actual problems.

Goals

  • Effectively communicate the need for professional discipline.
  • Provide an approach to force thinking about design before and apart from typing.

Design Approach

Use the analogy of surgeons washing their hands:

  • Professionals
  • A good example of something we all do, yet not closely approaching the rigor of professionals

Alternate Designs

I could use other professions like pilots. Hand washing is probably more relatable (and I cheated: I wrote most of the post before I wrote this section).

Constraints

  • I’m not a surgeon, so I’m depending on second-hand information
  • A single blog post that is short enough for people to read

Risks

It’s just a blog so the price of failure is small.

Risk: My resources for surgeon behavior are wrong

Mitigation: Check a few different sources

Risk: Poor writing and communication skills

Mitigation: Have a few peers review my writing

Timeline

I can knock this out in one sitting. Depending on feedback, could extend for a few days.

Thanks to Tim Pote, Caleb Phillips), Jack Mocherman, and Joe Kane for reading rough drafts and providing valuable feedback.

Permalink

Introducing KotlinToday.com

When I was learning Clojure for the first time, I had two references that I found myself reading every single day. The first resource is a fantastic guide by Daniel Higginbotham called Clojure for the Brave and True. The second resource that I found myself using allowed me to immerse myself into the Clojure community and figure out where people were focusing their efforts – Planet Clojure. Planet Clojure is a metablog that pulls in blog posts from a vetted list of Clojure developers. In fact, you may have even seen some of my posts on there, too! Unfortunately, when I picked up Kotlin, I couldn’t seem to find a Planet Clojure for Kotlin, so I made KotlinToday.com.

KotlinToday is still new and consists of a very small list of bloggers that I’ve followed until it’s inception. I still follow them, but instead of checking each blog individually every day, I can just go to KotlinToday.com and find the latest posts from these individuals. Anyways, there is no major post, tutorial or announcement other than this. I merely wanted to spread the word for other Kotlin Developers so that they are aware of KotlinToday.com!

If you’re interested in having your blog listed on KotlinToday.com, please feel out the super-short form in the sidebar of the site. Thanks!

Want to find more of my personal posts on Kotlin? Click here!

Want to find more Kotlin posts by other developers? Click Here!

The post Introducing KotlinToday.com appeared first on Brad Cypert.

Permalink

Unification

In logic and computer science, unification is a process of automatically solving equations between symbolic terms. Unification has several interesting applications, notably in logic programming and type inference. In this post I want to present the basic unification algorithm with a complete implementation.

Let's start with some terminology. We'll be using terms built from constants, variables and function applications:

  • A lowercase letter represents a constant (could be any kind of constant, like an integer or a string)
  • An uppercase letter represents a variable
  • f(...) is an application of function f to some parameters, which are terms themselves

This representation is borrowed from first-order logic and is also used in the Prolog programming language. Some examples:

  • V: a single variable term
  • foo(V, k): function foo applied to variable V and constant k
  • foo(bar(k), baz(V)): a nested function application

Pattern matching

Unification can be seen as a generalization of pattern matching, so let's start with that first.

We're given a constant term and a pattern term. The pattern term has variables. Pattern matching is the problem of finding a variable assignment that will make the two terms match. For example:

  • Constant term: f(a, b, bar(t))
  • Pattern term: f(a, V, X)

Trivially, the assignment V=b and X=bar(t) works here. Another name to call such an assignment is a substitution, which maps variables to their assigned values. In a less trivial case, variables can appear multiple times in a pattern:

  • Constant term: f(top(a), a, g(top(a)), t)
  • Pattern term: f(V, a, g(V), t)

Here the right substitution is V=top(a).

Sometimes, no valid substitutions exist. If we change the constant term in the latest example to f(top(b), a, g(top(a)), t), then there is no valid substitution becase V would have to match top(b) and top(a) simultaneously, which is not possible.

Unification

Unification is just like pattern matching, except that both terms can contain variables. So we can no longer say one is the pattern term and the other the constant term. For example:

  • First term: f(a, V, bar(D))
  • Second term f(D, k, bar(a))

Given two such terms, finding a variable substitution that will make them equivalent is called unification. In this case the substitution is {D=a, V=k}.

Note that there is an infinite number of possible unifiers for some solvable unification problem. For example, given:

  • First term: f(X, Y)
  • Second term: f(Z, g(X))

We have the substitution {X=Z, Y=g(X)} but also something like {X=K, Z=K, Y=g(K)} and {X=j(K), Z=j(K), Y=g(j(K))} and so on. The first substitution is the simplest one, and also the most general. It's called the most general unifier or mgu. Intuitively, the mgu can be turned into any other unifier by performing another substitution. For example {X=Z, Y=g(X)} can be turned into {X=j(K), Z=j(K), Y=g(j(K))} by applying the substitution {Z=j(K)} to it. Note that the reverse doesn't work, as we can't turn the second into the first by using a substitution. So we say that {X=Z, Y=g(X)} is the most general unifier for the two given terms, and it's the mgu we want to find.

An algorithm for unification

Solving unification problems may seem simple, but there are a number of subtle corner cases to be aware of. In his 1991 paper Correcting a Widespread Error in Unification Algorithms, Peter Norvig noted a common error that exists in many books presenting the algorithm, including SICP.

The correct algorithm is based on J.A. Robinson's 1965 paper "A machine-oriented logic based on the resolution principle". More efficient algorithms have been developed over time since it was first published, but our focus here will be on correctness and simplicity rather than performance.

The following implementation is based on Norvig's, and the full code (with tests) is available on Github. This implementation uses Python 3, while Norvig's original is in Common Lisp. There's a slight difference in representations too, as Norvig uses the Lisp-y (f X Y) syntax to denote an application of function f. The two representations are isomorphic, and I'm picking the more classical one which is used in most papers on the subject. In any case, if you're interested in the more Lisp-y version, I have some Clojure code online that ports Norvig's implementation more directly.

We'll start by defining the data structure for terms:

class Term:
    pass

class App(Term):
    def __init__(self, fname, args=()):
       self.fname = fname
       self.args = args

    # Not shown here: __str__ and __eq__, see full code for the details...


class Var(Term):
    def __init__(self, name):
        self.name = name


class Const(Term):
    def __init__(self, value):
        self.value = value

An App represents the application of function fname to a sequence of arguments.

def unify(x, y, subst):
    """Unifies term x and y with initial subst.

    Returns a subst (map of name->term) that unifies x and y, or None if
    they can't be unified. Pass subst={} if no subst are initially
    known. Note that {} means valid (but empty) subst.
    """
    if subst is None:
        return None
    elif x == y:
        return subst
    elif isinstance(x, Var):
        return unify_variable(x, y, subst)
    elif isinstance(y, Var):
        return unify_variable(y, x, subst)
    elif isinstance(x, App) and isinstance(y, App):
        if x.fname != y.fname or len(x.args) != len(y.args):
            return None
        else:
            for i in range(len(x.args)):
                subst = unify(x.args[i], y.args[i], subst)
            return subst
    else:
        return None

unify is the main function driving the algorithm. It looks for a substitution, which is a Python dict mapping variable names to terms. When either side is a variable, it calls unify_variable which is shown next. Otherwise, if both sides are function applications, it ensures they apply the same function (otherwise there's no match) and then unifies their arguments one by one, carefully carrying the updated substitution throughout the process.

def unify_variable(v, x, subst):
    """Unifies variable v with term x, using subst.

    Returns updated subst or None on failure.
    """
    assert isinstance(v, Var)
    if v.name in subst:
        return unify(subst[v.name], x, subst)
    elif isinstance(x, Var) and x.name in subst:
        return unify(v, subst[x.name], subst)
    elif occurs_check(v, x, subst):
        return None
    else:
        # v is not yet in subst and can't simplify x. Extend subst.
        return {**subst, v.name: x}

The key idea here is recursive unification. If v is bound in the substitution, we try to unify its definition with x to guarantee consistency throughout the unification process (and vice versa when x is a variable). There's another function being used here - occurs_check; I'm retaining its classical name from early presentations of unification. Its goal is to guarantee that we don't have self-referential variable bindings like X=f(X) that would lead to potentially infinite unifiers.

def occurs_check(v, term, subst):
    """Does the variable v occur anywhere inside term?

    Variables in term are looked up in subst and the check is applied
    recursively.
    """
    assert isinstance(v, Var)
    if v == term:
        return True
    elif isinstance(term, Var) and term.name in subst:
        return occurs_check(v, subst[term.name], subst)
    elif isinstance(term, App):
        return any(occurs_check(v, arg, subst) for arg in term.args)
    else:
        return False

Let's see how this code handles some of the unification examples discussed earlier in the post. Starting with the pattern matching example, where variables are just one one side:

>>> unify(parse_term('f(a, b, bar(t))'), parse_term('f(a, V, X)'), {})
{'V': b, 'X': bar(t)}

Now the examples from the Unification section:

>>> unify(parse_term('f(a, V, bar(D))'), parse_term('f(D, k, bar(a))'), {})
{'D': a, 'V': k}
>>> unify(parse_term('f(X, Y)'), parse_term('f(Z, g(X))'), {})
{'X': Z, 'Y': g(X)}

Finally, let's try one where unification will fail due to two conflicting definitions of variable X.

>>> unify(parse_term('f(X, Y, X)'), parse_term('f(r, g(X), p)'), {})
None

Lastly, it's instructive to trace through the execution of the algorithm for a non-trivial unification to see how it works. Let's unify the terms f(X,h(X),Y,g(Y)) and f(g(Z),W,Z,X):

  • unify is called, sees the root is an App of function f and loops over the arguments.
    • unify(X, g(Z)) invokes unify_variable because X is a variable, and the result is augmenting subst with X=g(Z)
    • unify(h(X), W) invokes unify_variable because W is a variable, so the subst grows to {X=g(Z), W=h(X)}
    • unify(Y, Z) invokes unify_variable; since neither Y nor Z are in subst yet, the subst grows to {X=g(Z), W=h(X), Y=Z} (note that the binding between two variables is arbitrary; Z=Y would be equivalent)
    • unify(g(Y), X) invokes unify_variable; here things get more interesting, because X is already in the subst, so now we call unify on g(Y) and g(Z) (what X is bound to)
      • The functions match for both terms (g), so there's another loop over arguments, this time only for unifying Y and Z
      • unify_variable for Y and Z leads to lookup of Y in the subst and then unify(Z, Z), which returns the unmodified subst; the result is that nothing new is added to the subst, but the unification of g(Y) and g(Z) succeeds, because it agrees with the existing bindings in subst
  • The final result is {X=g(Z), W=h(X), Y=Z}

Efficiency

The algorithm presented here is not particularly efficient, and when dealing with large unification problems it's wise to consider more advanced options. It does too much copying around of subst, and also too much work is repeated because we don't try to cache terms that have already been unified.

For a good overview of the efficiency of unification algorithms, I recommend checking out two papers:

  • "An Efficient Unificaiton algorithm" by Martelli and Montanari
  • "Unification: A Multidisciplinary survey" by Kevin Knight

Permalink

Top 43 Programming Languages: When and How to Use Them

There are many programming languages to choose from. The TIOBE Index is a list of programming languages, ranked in order of popularity. This article lists (most of) the top 50 languages on that list (a small number of languages that don't have any jobs listed were excluded.)

Each language listing includes:

  • Commentary: A description of each language is included, with some background and history.
  • Popularity Ranking:
    • Number of Jobs: This is the number of jobs found searching for that language at Indeed.com (at the time of this writing.) The number links to the search used. Each search displays the number of jobs found for the language in New York City, New York. This "baseline" search offers a good indication of the popularity of the language.
      • High (> 1,000 Jobs)
      • Medium (500 - 1000 Jobs)
      • Low (< 500 Jobs)
      • Niche (< 100 Jobs)
  • Ease of Learning: Easy, Moderate, or Difficult
  • Use Cases:
    • General Use
      • Local (Desktop) applications
      • Web applications
      • Mobile applications
    • Specialty
      • Embedded systems
      • Mainframe
      • Industrial
      • Academia/Research

How to choose a programming language

Programming languages are tools that express intent in a structured way. Software developers choose a language the way a construction worker selects a tool; by picking the best tool for the job.

The languages on this list all have one or more of a variety of attributes:

  • Procedural
  • Object-Oriented
  • Imperative
  • Declarative
  • Functional

... and many more

These attributes, combined with the requirements of a development task, make some languages better than others for the task at hand. Understanding the differences between these attributes, combined with experience, helps developers choose the right tool for the job they're doing.

Choosing the right language combines concerns about the knowledge and experience of a development team, the maturity of the language and its supporting libraries, and more.

List of 43 programming languages

1. What is Java?

Java is a popular programming language

  • Popularity: Very high
  • Ease of Learning: Moderate to Difficult
  • Use Cases: General Use and Specialty
    • Web applications
    • Mobile
    • Embedded systems

Java is the leading general-purpose application development language and framework. It was introduced in 1991 by Sun Microsystems as a high-level, compiled, memory-managed language.

Java's syntax is similar to C/C++, with curly braces for closures and semicolons to end statements. Automatic memory management is one of the features that made Java so popular, quickly after its initial release. Before Java was introduced, languages that required manual memory management, such as C and C++, were dominant. Manual memory allocation is tedious and error-prone, so Java was hailed as a major step forward for application developers.

The promise of Java, beyond memory management, was its cross-platform capability. This was marketed as "write once, run anywhere." The Java Virtual Machine (JVM) runs Java bytecode, which is compiled from the Java language. JVMs are available for most major operating systems, including Linux, Mac, and Windows. It doesn't always work perfectly, but when it does, a program written in Java can run on any platform with a compatible JVM.

Java is used for business, web, and mobile applications. It is the native language for Google's Android OS. Java also powers millions of set-top boxes and embedded devices. Java development skills are highly sought after.

If you're considering a job in software development, you should strongly consider learning Java.

2. What is C?

C is a popular programming language

  • Popularity: Medium
  • Ease of Learning: Moderate
  • Use Cases: General Use and Specialty
    • Embedded systems
    • Hardware drivers
    • Local Applications

Until Java was introduced, C was the dominant high-level language. It was first introduced in 1972. The first versions of Unix, written in Assembly language, were ported to C. It was then used in the development of other early operating systems, including IBM System/370.

C has a long history of development on older systems with slower processors and little memory. Programs written in C had to be very efficient, so C has a reputation for high performance in cases where speed matters.

C is still very popular due to its use in systems development, including operating systems, embedded devices, and as firmware. The C standard library has been ported to many platforms, so it is viable in many use cases. However, the low-level systems programming it is typically used for is a more specialized skill than general application programming. This explains why the second-most popular language on the TIOBE index has relatively few job postings as compared to other languages in the top 10.

There is likely to be some overlap in the jobs market with C++ (see the C++ listing below.)

3. What is Python?

Python is a popular programming language

  • Popularity: Very High
  • Ease of Learning: Easy to Moderate
  • Use Cases: General Use and Specialty
    • Web Applications
    • Artificial Intelligence

Python is a relatively new interpreted programming language, first introduced in 1989. It is an interpreted language that supports automatic memory management and object-oriented programming.

Python is very popular for general-purpose programming, including web applications. It has recently become known for specialty use in artificial intelligence applications.

Python jobs are very plentiful, so it's easy to find a job using Python.

4. What is C++?

cplus is a popular programming language

  • Popularity: High
  • Ease of Learning: Difficult
  • Use Cases: General Use, Specialty
    • Local Applications
    • Web Services
    • Proprietary Services

C++ extends C with object-oriented features. The "double-plus" comes from the increment operator from C. C++ was developed to bring features from older languages to faster, more powerful platforms.

C++ occupies a similar area in the market as C, including systems programming and low-level hardware development. Over the years, the C++ standard libraries and specification have been expanded considerably, leading to criticism that it has become over-complicated and difficult to learn.

5. What is Visual Basic .NET?

Visual Basic .NET is a popular programming language

  • Popularity: Low
  • Ease of Learning: Moderate
  • Use Cases: General Use
    • Web Applications
    • Local Applications

Visual Basic.NET (VB.NET) is Microsoft's implementation of the Visual Basic language that compiles to .NET Intermediate Language. This allows developers to write .NET applications using Visual Basic.
Applications written in VB.NET are, more or less, just as capable as any other. However,

VB.NET was never very popular for business applications. Application developers preferred C, C++, and C#. Most applications written in VB.NET tend to be older, and are likely to be considered to be "legacy" applications destined for decommission or redevelopment.

6. What is C#?

C# is a popular programming language

  • Popularity: High
  • Ease of Learning: Moderate
  • Use Cases: General Use
    • Web Applications
    • Local Applications
    • Services/Microservices

C# was developed and introduced by Microsoft in 2000, along with the overall .NET Framework. Syntactically, C# is very similar to Java and C/C++. It is a compiled, object-oriented language that compiles to .NET Intermediate Language.
Originally, C# was used for Microsoft-focused development of Windows Forms and web development with ASP.NET. The .NET ecosystem has evolved recently with the introduction of the .NET Standard and .NET Core. These new frameworks and standards are cross-platform, running on Windows, Linux, and Mac.

C# is popular for local and web application programming, often (but not necessarily) in systems developed primarily based on Microsoft technology. Microsoft's Xamarin framework allows developers to write Android and iOS applications in C#. It is suitable for systems programming in some cases and has libraries available for embedded systems.

7. What is PHP?

PHP is a popular programming language

  • Popularity: High
  • Ease of Learning: Easy
  • Use Cases: General Use
    • Web Applications

PHP originally stood for "Personal Home Page" as part of its first name, PHP/FI (Forms Interpreter.) The official acronym is now PHP: Hypertext Processor. Its primary role is as a web application server-side scripting system. It was originally developed to extend a CGI program to support HTML forms and database access. The code of a PHP program is mixed in with the HTML, making it similar to Microsoft's classic (pre-.NET) Active Server Pages. The interpreter reads the HTML and code and executes the code portions of the page.

PHP is popular because it's easy to learn. It is also the basis of popular web-based applications such as WordPress and Joomla. However, PHP also has a mixed reputation relating to software quality. Early versions lacked security controls and features that made it difficult to develop highly-secure applications. Recent developments in PHP frameworks and libraries have made improvements in security.

There are plenty of PHP jobs available, for content-focused web applications like WordPress, and proprietary systems developed in PHP.

8. What is JavaScript?

JavaScript is a popular programming language

  • Popularity: Very High
  • Ease of Learning: Moderate
  • Use Cases: General Use
    • Local Applications
    • Web Applications

JavaScript is a high-level, dynamically typed, interpreted language. It uses Java-like syntax, hence the name JavaScript. JavaScript was first introduced in the early days of the public Internet, 1995.
JavaScript is used to write code that runs in web browsers, on the client side. If you've been using the Web long enough to remember the introduction of Google Maps, you witnessed some of the first magic: the "infinite scrolling" in Maps is done using JavaScript.

Since its first introduction, JavaScript support has been added to all major web browsers. JavaScript frameworks including React, Angular, and Vue offer a Model-View-Controller application development paradigm, running entirely in the browser. JavaScript now supports the visual, browser-run elements of most modern web applications, which is why most real user monitoring tools cater for JavaScript.

JavaScript can also be combined with HTML to make cross-platform mobile applications.
NodeJS is a web server that runs JavaScript on the server side. NodeJS applications are written entirely in JavaScript.

Given all these use cases and support, JavaScript is both popular and in high demand. It is not very difficult to learn, though there are advanced programming techniques that take time to master. If you are more comfortable with object-oriented languages, consider looking into TypeScript. TypeScript "overlays" object-oriented features and syntax, and transpiles to native JavaScript.

9. What is SQL?

SQL is a popular programming language

  • Popularity: Very High
  • Ease of Learning: Easy to Moderate
  • Use Cases: Specialty
    • Database Queries

SQL stands for Structured Query Language. SQL is used to query and modify data in a Relational Database Management System (RDBMS.) Vendor-specific implementations, such as PL/SQL (Oracle) and T-SQL (Microsoft) offer product-specific features.

SQL isn't a general purpose language that can be used to write applications. However, it is at least a useful, if not required, skill of most developers. The term "full-stack developer" refers to a developer with a well-rounded skill set that includes all aspects of an application. This almost always includes accessing and saving data to a database. SQL is not hard to learn initially, though there are advanced use cases in Big Data and data analysis that require significant experience.

SQL is very popular with both developers and Database Administrators, so jobs that require SQL skills are plentiful. However, it is not a complete skill unto itself. SQL experience is a big plus on a resume, but it is rarely the primary skill required for any given job.

10. What is Objective-C?

Objective-C is a popular programming language

  • Popularity: High
  • Ease of Learning: Difficult
  • Use Cases: Mobile Applications
    • Apple iOS devices: iPhone, iPad

Objective-C is a general purpose, compiled, object-oriented language. Its syntax is derived from Smalltalk. Until 2014, when Apple introduced Swift, it was the primary language used by Apple to develop applications for MacOS and iOS.

Objective-C is still relatively popular, due to a large number of applications available that were written using it. Now that modern MacOS and iOS development is done primarily in Swift, it is likely that its popularity will eventually fall off as the number of supported applications tapers over time. Objective-C is not easy to learn. It uses syntax and language conventions that are not common to other languages, so experience with other languages does not apply well to Objective-C.

If you want to focus on software development for the Apple ecosystem, it's a good idea to pick up both Objective-C and Swift. This will give you the ability to work on older applications written in Objective-C, and write new applications in Swift. Between the two, jobs are very plentiful.

11. What is Delphi/Object Pascal?

Delphi/Object Pascal is a popular programming language

  • Popularity: Ultra-Niche
  • Ease of Learning: Moderate
  • Use Cases: General
    • Local Applications

Delphi is a compiler and Integrated Developer Environment (IDE) for the Object Pascal language. Object Pascal is an object-oriented derivative of Pascal, which was developed in the late 1960s.

Delphi/Object Pascal is on this list because there is a lot of software out there written in Object Pascal, with Delphi. As we can see from the number of jobs, Object Pascal is effectively a dead language. If you want to write software as a profession, ignore Delphi and Object Pascal. Their days have passed.

12. What is Ruby?

Ruby is a popular programming language

  • Popularity: High
  • Ease of Learning: Easy to Moderate
  • Use Cases: General
    • Web Applications
    • Scripting

Ruby is an interpreted, dynamically typed, object-oriented language first introduced in the mid-1990s. It was inspired by several other languages on this list, including Lisp, Perl, and Ada.
Ruby is very popular for web application development. The Ruby on Rails framework (now known simply as "Rails") is a model-view-component server-side framework written in Ruby.

Ruby is fairly easy to learn. Its common use in web applications makes job opportunities easy to find.

13. What is MATLAB?

MATLAB is a popular programming language

  • Popularity: Medium
    • 514 Jobs
  • Ease of Learning: Moderate to Difficult
  • Use Cases: Specialty
    • Mathematical Research

MATLAB is not a programming language, per se. It is an application that is used to calculate and model complex mathematical computations. It is used primarily in research settings, at universities and labs. MATLAB can handle complex matrix manipulations and supports extensions to use complex mathematical notation. Functions written in C, C#, and FORTRAN can be called from MATLAB.

The knowledge needed to use MATLAB is more related to the mathematical concepts and skills than knowledge of programming. If you're already an advanced math student working on a Ph.D. in mathematics, MATLAB is relatively easy to learn.

14. What is Assembly language?

Assembly  is a popular programming language

  • Popularity: Low
  • Ease of Learning: Difficult
  • Use Cases: Specialty
    • Systems Programming
    • Hardware / Firmware development

"Assembly language" is a generic term for low-level code that closely represents the native machine instructions for a given microprocessor. Most of the languages on this list are "high-level" languages that are closer, syntactically, to English. High-level language code must be compiled down to an intermediate bytecode, or directly to machine instructions. Assembly code is assembled, (hence the name) not compiled.

The intent of a line of code written in C or Ruby is relatively easy to understand, just by reading it. Assembly, by contrast, is very difficult to understand without a careful reading of the entire program. Each operation, including math operations and moving data in and out of registers, is a complete statement. This means that it takes a lot more assembly code than C code to do the same amount of work.

Assembly code is most useful when performance is the most important goal. It is used for very low-level systems programming, or in some cases may be combined with application code for a performance boost. Jobs that require knowledge of assembly will include systems programming and hardware development.

15. What is Swift?

Swift  is a popular programming language

  • Popularity: Medium
  • Ease of Learning: Moderate to Difficult
  • Use Cases: Apple Mobile and Desktop applications
    • MacBook
    • iPhone
    • iPad

Apple introduced Swift in 2014 as a modern alternative to Objective-C. Its goals were to be easier to debug than Objective-C. Swift syntax is easier to read than Objective-C and requires less code to do the same amount of work. However, breaking changes introduced with new versions may have stunted its adoption.

There are a fair number of jobs available for Swift, so it is probable that Swift is here to stay. As mentioned in the Objective-C listing, if you want to develop for the Apple ecosystem, hedge your bets and learn both.

17. What is Go?

Go  is a popular programming language

  • Popularity: Low
  • Ease of Learning: Moderate
  • Use Cases: General
    • Web Applications
    • Local Applications

Go (also known as Golang) is a relatively new kid on the block. It was introduced by two Google engineers in 2009. Go syntax borrows heavily from C and Java. The design goals for Go included cross-platform compatibility, simplicity, and support for modern processors.

Go is relatively easy to learn. It has some of the complexities of C/C++ (such as pointers) but its syntax and conventions are simpler. While Go jobs are not plentiful, there is a rapidly growing following in engineering and DevOps circles.

18. What is Perl?

Perl  is a popular programming language

  • Popularity: High
  • Ease of Learning: Easy to Moderate
  • Use Cases: General
    • Local Applications
    • Web Applications

Perl was introduced in 1987 as a utilitarian scripting language, evolving from CGI scripting. Recent releases of Perl are quite different from early releases.

Perl is fairly easy to learn, but it has its detractors. The development of Perl was somewhat haphazard, leading to criticism that it is not well organized. This has left Perl with a reputation for being less than robust.

Quite a lot of software has been written in Perl, and that continues to this day. Perl jobs are not hard to find. Having said that, it would be a stretch to say that Perl is a "modern" language. Perl may be a good language to learn early in a career, as a way to get started, but it shouldn't be the only one.

19. What is R?

R  is a popular programming language

  • Popularity: Low
  • Ease of Learning: Difficult
  • Use Cases: Specialty
    • Statistical Computation and Analysis

The programming language R is primarily used by statisticians and researchers to perform statistical analysis of datasets. Demographers, insurance actuaries, and other jobs focused on statistics use R. As with MATLAB, most of the knowledge needed to work with R relates to statistics. R programmers combine statistics knowledge with the essentials of programming and software development.

R programming jobs are not hard to come by, but the number is not high due to the specialized nature of the work. If you are a data analyst doing statistical work, there's a good chance you've learned R. If that work sounds like something you want to look into, you should strongly consider adding R to your toolbox.

20. What is PL/SQL

PL/SQL is a popular programming language

  • Popularity: Low to Medium
  • Ease of Learning: Moderate
  • Use Cases: Database Queries
    • Oracle Databases

PL/SQL is the vendor-specific implementation of the SQL language listed above. The syntax and features of PL/SQL align with features of Oracle databases. All dialects of SQL are moderately difficult to learn. Simple data querying and updating is fairly easy to learn. Joins, aggregation, and advanced concepts such as cursors require more understanding of database theory.

Oracle is a dominant database vendor, so PL/SQL jobs are fairly plentiful. If you are an Oracle Database Administrator, PL/SQL is a must-learn. Full-stack developers that work at the data "layer" should consider learning PL/SQL and other dialects.

21. What is Visual Basic?

visual basic is a popular programming language

  • Popularity: Low
  • Ease of Learning: Easy
  • Use Cases: General
    • Local Applications

Visual Basic (VB) was introduced by Microsoft as a variant of the BASIC programming language. It is an event-driven language and Integrated Development Environment primarily used to develop Windows applications. VB was designed to be easy to learn and to rapidly produce usable software. Visual Basic for Applications (VBA) is embedded in older versions of Microsoft Office applications, such as Access. VBA was used to provide programmatic manipulation of Office documents. Access databases used VBA to compose mini-applications.

Microsoft deprecated Visual Basic 6.0, the last version of Visual Basic, in 2008. It is no longer supported. Jobs that require Visual Basic are dwindling. It is likely that any such job is focused on maintenance and/or porting to a modern platform.

22. What is SAS?

SAS is a popular programming language

  • Popularity: Low
  • Ease of Learning: Difficult
  • Use Cases: General
    • Local Applications

SAS originally stood for "Statistical Analysis System." SAS was first developed in 1966 on mainframe computers. It was used for statistical data analysis.

SAS is not common, though there are still some jobs available. Modern statistical analysis tools have overtaken SAS.

23. What is Dart?

Dart is a popular programming language

  • Popularity: Niche
  • Ease of Learning: Moderate
  • Use Cases: General
    • Web Applications
    • Mobile Applications

Dart was introduced in 2011 by engineers at Google. It is a statically-typed, compiled language with a syntax similar to C. Dart transpiles to JavaScript.

Dart never really took off, so it is not popular and jobs are few.

24. What is F#?

F# is a popular programming language

  • Popularity: Niche
  • Ease of Learning: Moderate to Difficult
  • Use Cases: General and Specialty
    • Web Services
    • Machine learning

F# was introduced by Microsoft as an object-oriented, functional language that compiles to .NET Intermediate Language. As a functional language, F# expresses programs as mathematical functions. Functional languages are different from object-oriented and procedural languages, in that they avoid mutable data and state.
Functional programming is quite different from other more popular forms, however, it is well-suited to particular types of applications. Certain computations can be expressed more concisely and elegantly in functional languages, than in their object-oriented counterparts.

F# is a specialty-case language with few jobs available.

25. What is COBOL?

Cobol is a popular programming language

  • Popularity: Niche
  • Ease of Learning: Moderate to Difficult
  • Use Cases:
    • Mainframe Application Development

COBOL is a very old language used primarily for mainframe development. It is somewhat difficult to learn, by comparison with more modern languages.

Programmers that have been using COBOL for decades are enjoying high employability, due to the scarcity of COBOL programmers that are working and not retired. This is not a good reason to learn it if you don't already know it, however. Much better to invest in new skills for a new generation of languages and platforms.

26. What is Scala?

Scala is a popular programming language

  • Popularity: Moderate - High
  • Ease of Learning: Moderate to Difficult
  • Use Cases: General Use
    • Software Frameworks
    • Web Applications

Scala was designed to fill gaps in the Java, language, such as functional programming features. It has gained traction in the development of software libraries and applications.

Scala has been criticized for its high learning curve. Some say that learning new languages is a good thing for software developers, especially those that force developers to think about old problems in new ways. There are plenty of Scala jobs available. It may not make a good starter language, but if you have some experience it's worth looking into.

27. What is ABAP?

ABAP is a popular programming language

  • Popularity: Niche
  • Ease of Learning: Moderate to Difficult
  • Use Cases: Specialty
    • SAP Enterprise Software

ABAP was designed for use with the SAP enterprise software system to develop reports.

ABAP is a niche language with few jobs available. The syntax is quite different from modern languages. ABAP is probably not a good long-term career focus.

28. What is Fortran?

Fortran is a popular programming language

  • Popularity: Niche
  • Ease of Learning: Difficult
  • Use Cases: Specialty
    • Mainframe Programming

FORTRAN was developed by IBM for scientific and research-focused programming. It was originally introduced in 1957, so at 61 years old it is one of the oldest languages on this list.

FORTRAN has reached the point COBOL will likely reach in about ten to fifteen years- almost completely obsolete.

29. What is Lua?

Lua is a popular programming language

  • Popularity: Niche
  • Ease of Learning: Easy
  • Use Cases: General Use
    • General Programming

Lua was developed, by necessity, in Brazil, during a time where trade policies made it prohibitively expensive to purchase licensed software from elsewhere in the world. It was designed to be portable and easy to learn for non-technical users. Lua has become popular among novice game developers as a scripting language.

Lua is a niche language with few jobs available. Learning Lua may make you a better, more well-rounded developer, but it isn't likely to land you a job.

30. What is Rust?

Rust is a popular programming language

  • Popularity: Niche
  • Ease of Learning: Difficult
  • Use Cases: Specialty and General Use
    • Application/Software frameworks and components
    • Systems programming

Rust is a relatively new and powerful language that is popular for systems programming. Its design goals include safety, reliability, and speed. These features come at the cost of complexity and a high learning curve.

Rust is not a good first language and is not a crowd-pleaser. It's great for a learning challenge, and there are jobs out there that use it, but it's probably not a good idea to stake a career on it.

31. What is Lisp?

Lisp is a popular programming language

  • Popularity: Niche
  • Ease of Learning: Difficult
  • Use Cases: Specialty
    • Systems programming

LISP is only one year younger than FORTRAN, making it one of the "great grandaddy" languages of this list. LISP has inspired the design of many younger languages on this list, including JavaScript, Scala, Python, and many others.

LISP isn't very easy to learn and has a very verbose syntax (it's great if you LOVE typing parentheses!) The jobs that list LISP as a requirement favor it less for the language itself, and more for the deep understanding it offers of languages that came after it. LISP knowledge is most useful as a means to make you a more qualified developer in more modern languages.

32. What is Groovy?

Groovy is a popular programming language

  • Popularity: Niche
  • Ease of Learning: Moderate
  • Use Cases: Specialty and General
    • Local Applications
    • Developer Tools

Groovy is closely associated with Java. Its code compiles to Java bytecode, and the syntax is very similar to Java. Groovy adds features not present in Java, such as dynamic typing and operator overloading. Groovy code can also be run uncompiled, making it a scripting platform.

Groovy has developed a niche following among developers and is used in some popular tools such as Jenkins. If you already know Java or a language like it, Groovy should be fairly easy to learn. It's listed as a plus on many job listings, but it's not likely to be a core requirement of any given job.

33. What is LabVIEW?

LabVIEW is a popular programming language

  • Popularity: Niche
  • Ease of Learning: Moderate to Difficult
  • Use Cases: Specialty
    • Industrial Automation

LabVIEW is an IDE and compiler that uses a graphical language to design programs. It is used primarily to control machines for industrial automation scenarios.

LabVIEW is to mechanical engineering and automation as MATLAB is to mathematical research. If you're an engineer building automated systems, using LabVIEW is likely to be part of your job. Otherwise, LabVIEW is not likely to be useful beyond academic pursuits.

34. What is Prolog?

Prolog is a popular programming language

  • Popularity: Niche
  • Ease of Learning: Difficult
  • Use Cases: Specialty
    • Artificial Intelligence

Prolog is a declarative language based on fairly advanced mathematical and logical theories. It is used for complex systems.

Prolog's syntax is difficult to learn and read. Unless you happen to be in a field that uses Prolog, it's probably safe to skip it.

35. What is Ada?

Ada is a popular programming language

  • Popularity: Niche
  • Ease of Learning: Moderate
  • Use Cases: General Use

Ada is one of the older languages on this list. It was first introduced in 1980. Ada is most closely related to Pascal. Ada has many typical high-level language features, including static typing and object-orientation.

Ada is a niche language with a few jobs available. Most of that work is likely to be maintenance and porting to newer platforms.

36. What is Julia?

Julia is a popular programming language

  • Popularity: Very Niche
  • Ease of Learning: Difficult
  • Use Cases: Specialty
    • Data Science

Julia is a relatively new language, introduced in 2012. Its syntax is a descendant of C. Julia was designed for data science and analytic applications.

Julia is another language on this list that is tightly associated with the work it is designed to support. If you are already in a data analytics job, there's a good chance you have learned Julia.

37. What is Haskell?

Haskell is a popular programming language

  • Popularity: Niche
  • Ease of Learning: Difficult
  • Use Cases: General/Specialty

Haskell is one of the older purely functional, high-level languages. Functional languages focus on expressing intent through mathematical statements, rather than imperative subroutines that "act" on data. Many other functional languages derive from Haskell.

Haskell is rather difficult to learn, however, it is a good opportunity to learn functional programming. If you learn Haskell, you'll learn this method of writing software. You'll also be valuable to a niche market of jobs that need this specialized method of software development.

38. What is Apex?

Apex is a popular programming language

  • Popularity: Low
  • Ease of Learning: Moderate
  • Use Cases: Specialty
    • Salesforce.com Development

Apex is a language similar to C# and Java. It was developed by Salesforce.com as a proprietary language used to develop on the Salesforce platform.

Apex is about as hard to learn as C# and Java. If you want to develop using Salesforce, learn Apex.

39. What is Kotlin?

Kotlin is a popular programming language

  • Popularity: Low
  • Ease of Learning: Moderate
  • Use Cases: Mobile Development
    • Android Applications

Kotlin runs on the Java VM and compiles to JavaScript. Its syntax is very similar to Java. Google supports Kotlin for Android OS development.

Kotlin has risen in popularity over time, but it remains a niche language overall.

40. What is Bash?

Bash is a popular programming language

  • Popularity: Low (see details)
  • Ease of Learning: Easy to Moderate
  • Use Cases:
    • Linux scripting and automation

Bash scripts are used to automate tasks on Unix and Linux platforms. The syntax is fairly simple and easy to pick up.

Bash skills are required primarily for Unix/Linux administration, and in roles that need to write scripts for those platforms. Developers can use Bash to help automate software delivery pipelines. Bash is an important skill to develop for those use cases and others, however, it is not a full programming language unto itself.

41. What is Ladder Logic?

Ladder Logic is a popular programming language

  • Popularity: Niche
  • Ease of Learning: Difficult
  • Use Cases:
    • Industrial Automation

Ladder Logic isn't so much a programming language, as it is a notation for the design of circuits. It is used primarily for the design of industrial electronics.

42. What is Clojure?

Clojure is a popular programming language

  • Popularity: Niche
  • Ease of Learning: Difficult
  • Use Cases: General

Clojure is related to LISP and uses a similar syntax. Clojure runs on the Java VM. It is used in a variety of applications and systems. Clojure jobs aren't plentiful but they are out there.

43. What is Scheme?

Scheme is a popular programming language

  • Popularity: Niche
  • Ease of Learning: Difficult
  • Use Cases: Specialty

Scheme is another LISP-related functional language. It is used in a few various applications such as OS and game development.

So, which programming language should you choose to learn?

Now we've shown you some of the most popular programming languages, it's time to choose which is the best for your project.

Permalink

PurelyFunctional.tv Newsletter 300: Rust, React,

Issue 300 – November 12, 2018 · Archives · Subscribe

Hi Clojurnators,

Well, folks. This is issue 300 of the newsletter! My goodness. Time flies. I thought I might do something big and exciting, but, no, it’s just another issue!

Please enjoy the issue!

Rock on!
Eric Normand <eric@purelyfunctional.tv>

PS Want to get this in your email? Subscribe!


Debugging Under Fire: Keep your Head when Systems have Lost their Mind YouTube

Bryan Cantrill has some good rants about dealing with disasters like cloud data centers going down.


Basic HTML with Hiccup Free lesson

We take a look at using Hiccup to represent HTML in Reagent. This lesson is part of the popular Understanding Re-frame course. Follow PurelyFunctional.tv on Twitter because you’ll be notified of free lessons.


The Summer of RUST YouTube

Bryan Cantrill rants through his history with programming languages, how he chose rust for the future, and what he likes about it.


React Today and Tomorrow and 90% Cleaner React With Hooks YouTube

I don’t use React with JavaScript much, but this future feature (called hooks) looks like it could make things much cleaner. This talk explains what they are and how to use them. The talk also shows me why I am lucky to use React from ClojureScript, where we have a much nicer time. The amount of mutation and other effects going on is cringe-inducing.


Tools, Deps, Magic Balls YouTube

Reid McKenzie talk about his new build tool project called Katamari. The demo is awesome! I’m really looking forward to using this tool.


Timothy Baldridge on defn Podcast

A great discussion about Clojure, the community, the development process, and Timothy’s projects.


Why is functional programming gaining traction? Why now? Podcast

I go over four hypotheses for why FP is gaining popularity.


A Framework Author’s Case Against Frameworks YouTube

Adrian Holovaty, co-developer of the Django web framework, talks about what he’s learned and how to avoid the deluge of framework news.


Announcement: No Clojure SYNC in 2019

I’m sorry to say it, but it won’t be happening in 2019. I’m deliberately leaving 2020 open. This post talks about why, and the future of Clojure SYNC.

The post PurelyFunctional.tv Newsletter 300: Rust, React, appeared first on PurelyFunctional.tv.

Permalink

Copyright © 2009, Planet Clojure. No rights reserved.
Planet Clojure is maintained by Baishamapayan Ghose.
Clojure and the Clojure logo are Copyright © 2008-2009, Rich Hickey.
Theme by Brajeshwar.