Wednesday, November 26, 2014

Reflections on "One Hacker Way"

"If the sovereign can realize the difficulty of being a sovereign, and the minister realize the difficulty of being a minister, then the government will be well ordered, and the common people will strive diligently after Virtue."

-- The Counsels of the Great Yu
Book of Documents as translated by James Legge

I am interested in anything which gets people's emotions on fire.  When I saw the twitter firestorm around Erik Meijer's presentation "One Hacker Way" I knew I had to see it for myself.

One Hacker Way - Erik Meijer from Reaktor on Vimeo.

It has been a few days since that viewing and overall I have to say he has some good points.

Is this the best presentation I've ever seen?  No.  Is every minute the best possible thing you could do we your time?  No.  Could it make you think about the way you do your work?  Most likely yes.

I got three points from the talk:

  1. We talk too much about code and do not write enough of it.
  2. Code which delivers value to someone is the ultimate goal, everything else is either giving context to the code or noise.
  3. We need to seek real feedback and adjust.
We talk too much about code and do not write enough of it.  Erik starts off the presentation asking his audience at a developer conference how many of them check in code last week?  He then tells those that did not that maybe they should leave.  Harsh but to the point.

There is a lot of talk about code in our industry, there are whole conferences and alliances around how best to get others to write code.  Do not take my word for it, read this blog post titled "Agile is Dead" by Dave Thomas an Agile Manifesto signer.  In his post Dave talks about how the term Agile has become meaningless, instead of following the principles leading to the Agile Manifesto a whole industry has formed to sell Agile.

"When the relationship between superiors and subordinates becomes disordered, at first the subordinates usurp the actuality (i.e. real power), but continue to preserve the name.  Once the usurpation has lasted for some time, though, the name is appropriated and usurped as well."

-- Wang Fuzhi
Commentary on Confucius Analects 13.14
Translated by Edward Slingerland

The term Agile has been usurped and has become meaningless.  At one time it did have a meaning and here is what it was:

Individuals and interactions over processes and tools
Working software over comprehensive documentation
Customer collaboration over contract negotiation
Responding to change over following a plan

-- Manifesto for Agile Software

Instead of prompting the left side of the manifesto the usurpers are selling the right side.  I believe this is is what Erik calls a disease of our industry.  Instead of talking about how to write code which delivers value and seeks feedback faster we are having people talk about tools, certifications, and overhead.

Code which delivers value to someone is the ultimate goal, everything else is either giving context to the code or noise.  This goes straight to the heart of the Agile Manifesto: "Working software over comprehensive documentation" and "Customer collaboration over contract negotiation".  If we are professionals then we need to focus on what the job is.  What is that job being asked of us?  Delivering working software with the features found through customer collaboration.  

We are not paid to move stickies across a board, we are paid to deliver working software which meets the customers needs.  This is what is meant by "Individuals and interactions over processes and tools" in the Agile Manifesto, tools and processes are to be used to reach the goal of working software which delivers value to the customer.  If you find that having stickies helps you in achieving this goal then use them.

"Our highest priority is to satisfy the customer through early and continuous delivery of valuable software."

-- Principles behind the Agile Manifesto

We need to "Responding to change over following a plan".  We need to seek real feedback and adjust.

"Business people and developers must work together daily throughout the project."

"At regular intervals, the team reflects on how to become more effective, then tunes and adjusts its behavior accordingly."

As Heraclitus would say, everything is in motions, everything is becoming, nothing is constant except change itself.  If we cannot change along with our customer's needs then we are dead.

Is "One Hacker Way" the greatest call to action ever?  No.  Do I agree with everything in it?  No.  Do I agree with most of it?  No.  Did I find the three main points valid?  Yes.

Sunday, November 9, 2014

Distinctly Preserving Order in Clojure

"Tell me, you heavens, in which part of his body
Shall I destroy him? – whether there, or there, or there? –
That I may give the local wound a name,
And make distinct the very breach whereout
Hector's great spirit flew: answer me, heavens!
"-- Shakespeare, Troilus and Cressida
Act IV, Scene V, Lines 242-246

4clojure problem number 56 states:

Write a function which removes the duplicates from a sequence. Order of the items must be maintained.

The trick is you cannot use distinct.

We are giving the following test cases:

(= (__ [1 2 1 3 1 2 4]) [1 2 3 4])

(= (__ [:a :a :b :b :c :c]) [:a :b :c])

(= (__ '([2 4] [1 2] [1 3] [1 3])) '([2 4] [1 2] [1 3]))

(= (__ (range 50)) (range 50))

Let's try using group-by and get the first value of each group.

(fn [xs] (->> (group-by identity xs) (map first)))

This works fine for all but the last test case.  It seems that group-by does not preserve order.

(type (group-by identity (range 50)))
;; clojure.lang.PersistentHashMap

The PersistentHashMap in Clojure is implemented as a wide-tree with each node having up to 32 children (for more information see this excellent post by Karl Krukow or this excellent StackOverflow answer from Daniel Janus).

At this point we need to figure out a way to reorder the output from the group-by and first to what was given in the original input.  We can use sort-by but we need to use something that will give us the original input, this is were .indexOf saves the day.

We can use Java's .indexOf wrapped in a function to give sort-by an order which corresponds with the original input.  Using .indexOf will allow us to resort the output of the group-by followed by the first mapping to order that was in the original input, giving us this final function which preserve order and gives distinct values.

  (fn [xs] 
           (group-by identity xs) 
           (map first) 
           (sort-by #(.indexOf xs %))))

This passes all of the 4clojure's test cases.

Sunday, November 2, 2014

Technetium OR Reverse Interleave in Clojure

"Yet ‘ banished ’? Hang up philosophy!
Unless philosophy can make a Juliet,
Displant a town, reverse a prince's doom,
It helps not, it prevails not. Talk no more.
-- Shakespeare, Romeo and Juliet
Act III, Scene III, Lines 58-61

4clojure problem number 43 states:

"Write a function which reverses the interleave process into x number of subsequences."
For now let's look at the interleave process before we move on to reversing it.

ClojureDocs gives us the following:

"Returns a lazy seq of the first item in each coll, then the second etc."
In other words, give interleave a bunch of collections and it will place an item from each one into a result collection in a lazy way.  Something like this (based on the wikipedia entry):

Given sequences x and y with members i = 0, 1, 2, ... the interleave sequence of x and y is:
x0, y0, x1, y1, x2, y2, ...

Let's look at using interleave with the results of the test cases given by the 4clojure problem in a Clojure REPL.

(interleave '(1 3 5) '(2 4 6))
;; (1 2 3 4 5 6)

(interleave '(0 3 6) '(1 4 7) '(2 5 8))
;; (0 1 2 3 4 5 6 7 8)

(interleave '(0 5) '(1 6) '(2 7) '(3 8) '(4 9))
;; (0 1 2 3 4 5 6 7 8 9)

If we look at one of these in more detail we can see that it is following the definition given above.

(interleave '(0 3 6) '(1 4 7) '(2 5 8))
;; (0 1 2 3 4 5 6 7 8)

Or more mathematically

(interleave '(:x0 :x1 :x2) '(:y0 :y1 :y2) '(:z0 :z1 :z2))
;; (:x0 :y0 :z0 :x1 :y1 :z1 :x2 :y2 :z2)

Interleaving does not do a sort or anything like that, so if we move around the input we get different output:

(interleave '(1 4 7) '(0 3 6) '(2 5 8))
;; (1 0 2 4 3 5 7 6 8)

Now let's look at the problem at hand.

"Write a function which reverses the interleave process into x number of subsequences."

(= (__ [1 2 3 4 5 6] 2) '((1 3 5) (2 4 6)))

(= (__ (range 9) 3) '((0 3 6) (1 4 7) (2 5 8)))

(= (__ (range 10) 5) '((0 5) (1 6) (2 7) (3 8) (4 9)))

We need to come up with a function which will reverse the interleave function.  In other words, given a collection and the number of partitions we want returned, we get a collection of collections which can be interleave to return back the original collection given.

First thing we can do is set up our partitions using the partition function.

(fn [xs n]
    (->> (partition n xs) ))

This would give us the following using the test cases from 4clojure:

((fn [xs n]
      (->> (partition n xs) ))
[1 2 3 4 5 6] 2)
;; ((1 2) (3 4) (5 6))

((fn [xs n]
      (->> (partition n xs) ))
(range 9) 3)
;; ((0 1 2) (3 4 5) (6 7 8))

((fn [xs n]
      (->> (partition n xs) ))
(range 10) 5)
;; ((0 1 2 3 4) (5 6 7 8 9))

We now have each collection containing the number of partitions we want in the final result but everything is all mixed up!

If we look at the output for the first test case right now we have:
( (1 2) (3 4) (5 6) ) but what we want is:
( (1 3 5) (2 4 6) )

Looking at this closer we have the following:
( (1 2) (3 4) (5 6) ) but we want this:
( (1 3 5) (2 4 6) )

Looks like if we just need to map out the members in order and we'll get the result we want.  Following this will give us the following function:

(fn [xs n]
      (->> (partition n xs) (apply map list)))

Running this against our test cases gives us the following:

((fn [xs n]
      (->> (partition n xs) (apply map list)))
[1 2 3 4 5 6] 2)
;; ((1 3 5) (2 4 6))

((fn [xs n]
      (->> (partition n xs) (apply map list)))
(range 9) 3)
;; ((0 3 6) (1 4 7) (2 5 8))

((fn [xs n]
      (->> (partition n xs) (apply map list)))
(range 10) 5)
;; ((0 5) (1 6) (2 7) (3 8) (4 9))

This is exactly what we want.

Now I did not see this solution at first, this is what I submitted first to 4clojure:

(fn [xs n]
      (->> (map-indexed #(hash-map (mod %1 n) %2) xs) (group-by keys) (map second) (map #(mapcat vals %))))

Yep, very complex but it does work :(

As soon as I look at ramo and amcnamara's solutions I saw what I really wanted to do in my head but was unsure of how to with my Clojure knowledge, resulting in the final function I submitted:

(fn [xs n]
      (->> (partition n xs) (apply map list)))

Saturday, October 18, 2014

Bird Watching with Clojure -- The Thrush(y)

"With heigh, with heigh, the thrush and the jay,
Are summer songs for me and my aunts
Shakespeare, The Winter's Tale
Act IV, Scene III, Lines 10-11

The Thrush

As the wind gently rustles the leaves in the combinator forest.  You sing DO RA and hear sung back to you RA DO.  You sing RA ME and hear ME RA, you sing DO DO and hear DO DO back.  You have just heard the Thrush.

The Thrush or T combinator basically flips your parameters.

T x f = f x

In the Data Aviary Bird package for Haskell it is defined in the following way:

-- | T combinator - thrush.
-- Haskell @(\#)@ in Peter Thiemann\'s Wash, reverse application.
thrush :: a -> (a -> b) -> b
thrush x f = f x

We see that when we call it we pass a value as our first parameter followed by the function we wish to apply the value against.  This is useful when you want to pipe your data through a bunch of functions.

Thrush in F#

In fact in F# this is exactly what the |> (forward pipe) does.  It is defined in the following way:

let (|>) x f = f x

Yep, the forward pipe is nothing more than a Thrush!

Thrush in Joy

In Joy the Thrush goes by the name swap, which makes perfect sense.  It is defined in the following way:

[B] [A] swap == [A] [B]

Thrush in JavaScript

Reg Braithwaite of fame has a JavaScript library which is perfect for learning combinators called  The Thrush is defined in the following way in

function Thrush (a, b) {
    return, a)
  function T (a) { return function _T (b) {
    return b(a)

This exactly what we would expect looking at the Haskell and F# code above.

This is very similar to how Reginald defined the Thrush in Ruby in his excellent book, Kestrels, Quirky Birds, and Hopelessly Egocentricity:

Thrush(y) in Clojure

Clojure has two macros which are Thrushy (Michael Fogus has an excellent post on why they are Thrushy) the -> and ->> macros.

Say we wanted to do the following:

get the sum of the even integers from 1 to 100

We can break this requirement down into a few steps.

  1. get the integers from 1 to 100
  2. get the even integers from 1
  3. sum the integers from 2

What we see happening is a linking or piping of the results of a step down to the next step.

1 to 100 => evens => sum

Here are two ways to write this in Clojure:

The first way looks like a waterfall, but in order to understand it we need to go all the way to the bottom and work are way back up.  In this example we would start with the (range 101) then move up to the (filter even?) and lastly end at the (reduce +).  We find that reading this, we are going against the left to right flow of the code.

The second, Thrush(y) way, the first thing we see is our (range 101), next we find the (filter even?), and last we find the (reduce +).  This way seems very natural as the flow of data agrees with the reading from left to right.

I personally find the pipeline style used in the second way easier to read and use it as often as I can.  Martin Folwer has written an excellent article which goes into collection pipelines in more detail.  Debasish Ghosh has also written about the Thrush in Clojure (which help me a lot when I was trying to understand the -> and ->>macros).

Saturday, October 11, 2014

Try Before You Buy OR REPLs in the Browser

"Now follow – if thou darest – to try whose right"-- Shakespeare, A Midsummer Night's Dream
Act III, Scene II, Line 336

"Can't someone else do it?"
-- Homer Simpsons, Trash of the Titans

One of the hardest things to do is setting up a new programming environment.  Things have improve a lot over the years, but it still be a very daunting task to set up a programming environment for a language you do not even know.  Often in the past I would ask myself, "can't someone else do it?"

Someone else has!  Try a REPL in your browser today!

Luckily many programming environments are waiting for you right now as REPLs in your browser.

A REPL is a Read Evaluate Print Loop, which works how it sounds:

Read input (in the form of code)
Evaluate input
Print result of input
Loop back to top

Today you can now play around with many different programming language from the comfort of your browser by using a REPL.

Here are two that I enjoy:

Clojure instaREPL
Try Haskell!

There is also which has an impressive collection of languages including: Ruby, Forth, JavaScript, Roy, and even Brainfuck, to name a few.

Clojure instaREPL and Try Haskell! are a bit more like an actual REPL running on your computer than, but does offer languages which are not normally associated with REPLs (other than Ruby).

One last thing, while TryAPL is not really a REPL, it is very awesome that you can write APL code with a simulated APL keyboard in your browser!

'Try APL today!'