Reflections on "One Hacker Way"

"If the sovereign can realize the difficulty of being a sovereign, and the minister realize the difficulty of being a minister, then the government will be well ordered, and the common people will strive diligently after Virtue."

-- The Counsels of the Great Yu
Book of Documents as translated by James Legge

I am interested in anything which gets people's emotions on fire.  When I saw the twitter firestorm around Erik Meijer's presentation "One Hacker Way" I knew I had to see it for myself.


One Hacker Way - Erik Meijer from Reaktor on Vimeo.

It has been a few days since that viewing and overall I have to say he has some good points.

Is this the best presentation I've ever seen?  No.  Is every minute the best possible thing you could do we your time?  No.  Could it make you think about the way you do your work?  Most likely yes.

I got three points from the talk:

1. We talk too much about code and do not write enough of it.
2. Code which delivers value to someone is the ultimate goal, everything else is either giving context to the code or noise.
3. We need to seek real feedback and adjust.

We talk too much about code and do not write enough of it.  Erik starts off the presentation asking his audience at a developer conference how many of them check in code last week?  He then tells those that did not that maybe they should leave.  Harsh but to the point.

There is a lot of talk about code in our industry, there are whole conferences and alliances around how best to get others to write code.  Do not take my word for it, read this blog post titled "Agile is Dead" by Dave Thomas an Agile Manifesto signer.  In his post Dave talks about how the term Agile has become meaningless, instead of following the principles leading to the Agile Manifesto a whole industry has formed to sell Agile.

"When the relationship between superiors and subordinates becomes disordered, at first the subordinates usurp the actuality (i.e. real power), but continue to preserve the name.  Once the usurpation has lasted for some time, though, the name is appropriated and usurped as well."

-- Wang Fuzhi

Commentary on Confucius Analects 13.14

Translated by Edward Slingerland

The term Agile has been usurped and has become meaningless.  At one time it did have a meaning and here is what it was:

Individuals and interactions over processes and tools
Working software over comprehensive documentation
Customer collaboration over contract negotiation
Responding to change over following a plan

-- Manifesto for Agile Software

Instead of prompting the left side of the manifesto the usurpers are selling the right side.  I believe this is is what Erik calls a disease of our industry.  Instead of talking about how to write code which delivers value and seeks feedback faster we are having people talk about tools, certifications, and overhead.

Code which delivers value to someone is the ultimate goal, everything else is either giving context to the code or noise.  This goes straight to the heart of the Agile Manifesto: "Working software over comprehensive documentation" and "Customer collaboration over contract negotiation".  If we are professionals then we need to focus on what the job is.  What is that job being asked of us?  Delivering working software with the features found through customer collaboration.  

We are not paid to move stickies across a board, we are paid to deliver working software which meets the customers needs.  This is what is meant by "Individuals and interactions over processes and tools" in the Agile Manifesto, tools and processes are to be used to reach the goal of working software which delivers value to the customer.  If you find that having stickies helps you in achieving this goal then use them.

"Our highest priority is to satisfy the customer through early and continuous delivery of valuable software."

-- Principles behind the Agile Manifesto

We need to "Responding to change over following a plan".  We need to seek real feedback and adjust.

"Business people and developers must work together daily throughout the project."

"At regular intervals, the team reflects on how to become more effective, then tunes and adjusts its behavior accordingly."

-- Principles behind the Agile Manifesto

As Heraclitus would say, everything is in motions, everything is becoming, nothing is constant except change itself.  If we cannot change along with our customer's needs then we are dead.

Is "One Hacker Way" the greatest call to action ever?  No.  Do I agree with everything in it?  No.  Do I agree with most of it?  No.  Did I find the three main points valid?  Yes.

Distinctly Preserving Order in Clojure

"Tell me, you heavens, in which part of his body
Shall I destroy him? – whether there, or there, or there? –
That I may give the local wound a name,
And make distinct the very breach whereout
Hector's great spirit flew: answer me, heavens!"
-- Shakespeare, Troilus and Cressida
Act IV, Scene V, Lines 242-246

4clojure problem number 56 states:

Write a function which removes the duplicates from a sequence. Order of the items must be maintained.

The trick is you cannot use distinct.

We are giving the following test cases:

(= (__ [1 2 1 3 1 2 4]) [1 2 3 4])

(= (__ [:a :a :b :b :c :c]) [:a :b :c])

(= (__ '([2 4] [1 2] [1 3] [1 3])) '([2 4] [1 2] [1 3]))

(= (__ (range 50)) (range 50))

Let's try using group-by and get the first value of each group.

(fn [xs] (->> (group-by identity xs) (map first)))

This works fine for all but the last test case.  It seems that group-by does not preserve order.

(type (group-by identity (range 50)))
;; clojure.lang.PersistentHashMap

The PersistentHashMap in Clojure is implemented as a wide-tree with each node having up to 32 children (for more information see this excellent post by Karl Krukow or this excellent StackOverflow answer from Daniel Janus).

At this point we need to figure out a way to reorder the output from the group-by and first to what was given in the original input.  We can use sort-by but we need to use something that will give us the original input, this is were .indexOf saves the day.

We can use Java's .indexOf wrapped in a function to give sort-by an order which corresponds with the original input.  Using .indexOf will allow us to resort the output of the group-by followed by the first mapping to order that was in the original input, giving us this final function which preserve order and gives distinct values.

  (fn [xs] 
      (->> 
           (group-by identity xs) 
           (map first) 
           (sort-by #(.indexOf xs %))))

This passes all of the 4clojure's test cases.

Technetium OR Reverse Interleave in Clojure

"Yet ‘ banished ’? Hang up philosophy!
Unless philosophy can make a Juliet,
Displant a town, reverse a prince's doom,
It helps not, it prevails not. Talk no more."

-- Shakespeare, Romeo and Juliet
Act III, Scene III, Lines 58-61

4clojure problem number 43 states:

"Write a function which reverses the interleave process into x number of subsequences."
For now let's look at the interleave process before we move on to reversing it.

ClojureDocs gives us the following:

"Returns a lazy seq of the first item in each coll, then the second etc."
In other words, give interleave a bunch of collections and it will place an item from each one into a result collection in a lazy way.  Something like this (based on the wikipedia entry):

Given sequences x and y with members i = 0, 1, 2, ... the interleave sequence of x and y is:
x0, y0, x1, y1, x2, y2, ...

Let's look at using interleave with the results of the test cases given by the 4clojure problem in a Clojure REPL.

(interleave '(1 3 5) '(2 4 6))
;; (1 2 3 4 5 6)

(interleave '(0 3 6) '(1 4 7) '(2 5 8))
;; (0 1 2 3 4 5 6 7 8)

(interleave '(0 5) '(1 6) '(2 7) '(3 8) '(4 9))
;; (0 1 2 3 4 5 6 7 8 9)

If we look at one of these in more detail we can see that it is following the definition given above.

(interleave '(0 3 6) '(1 4 7) '(2 5 8))
;; (0 1 2 3 4 5 6 7 8)

Or more mathematically

(interleave '(:x0 :x1 :x2) '(:y0 :y1 :y2) '(:z0 :z1 :z2))
;; (:x0 :y0 :z0 :x1 :y1 :z1 :x2 :y2 :z2)

Interleaving does not do a sort or anything like that, so if we move around the input we get different output:

(interleave '(1 4 7) '(0 3 6) '(2 5 8))
;; (1 0 2 4 3 5 7 6 8)

Now let's look at the problem at hand.

"Write a function which reverses the interleave process into x number of subsequences."

(= (__ [1 2 3 4 5 6] 2) '((1 3 5) (2 4 6)))


(= (__ (range 9) 3) '((0 3 6) (1 4 7) (2 5 8)))


(= (__ (range 10) 5) '((0 5) (1 6) (2 7) (3 8) (4 9)))

We need to come up with a function which will reverse the interleave function.  In other words, given a collection and the number of partitions we want returned, we get a collection of collections which can be interleave to return back the original collection given.

First thing we can do is set up our partitions using the partition function.

(fn [xs n]

    (->> (partition n xs) ))

This would give us the following using the test cases from 4clojure:

((fn [xs n]

      (->> (partition n xs) ))

[1 2 3 4 5 6] 2)

;; ((1 2) (3 4) (5 6))

((fn [xs n]

      (->> (partition n xs) ))

(range 9) 3)

;; ((0 1 2) (3 4 5) (6 7 8))

((fn [xs n]
      (->> (partition n xs) ))
(range 10) 5)
;; ((0 1 2 3 4) (5 6 7 8 9))

We now have each collection containing the number of partitions we want in the final result but everything is all mixed up!

If we look at the output for the first test case right now we have:
( (1 2) (3 4) (5 6) ) but what we want is:
( (1 3 5) (2 4 6) )

Looking at this closer we have the following:
( (1 2) (3 4) (5 6) ) but we want this:
( (1 3 5) (2 4 6) )

Looks like if we just need to map out the members in order and we'll get the result we want.  Following this will give us the following function:

(fn [xs n]
      (->> (partition n xs) (apply map list)))

Running this against our test cases gives us the following:

((fn [xs n]
      (->> (partition n xs) (apply map list)))
[1 2 3 4 5 6] 2)
;; ((1 3 5) (2 4 6))

((fn [xs n]
      (->> (partition n xs) (apply map list)))
(range 9) 3)
;; ((0 3 6) (1 4 7) (2 5 8))

((fn [xs n]
      (->> (partition n xs) (apply map list)))
(range 10) 5)
;; ((0 5) (1 6) (2 7) (3 8) (4 9))

This is exactly what we want.

Now I did not see this solution at first, this is what I submitted first to 4clojure:

(fn [xs n]

      (->> (map-indexed #(hash-map (mod %1 n) %2) xs) (group-by keys) (map second) (map #(mapcat vals %))))

Yep, very complex but it does work :(

As soon as I look at ramo and amcnamara's solutions I saw what I really wanted to do in my head but was unsure of how to with my Clojure knowledge, resulting in the final function I submitted:

(fn [xs n]
      (->> (partition n xs) (apply map list)))