Wednesday, November 26, 2014

Reflections on "One Hacker Way"

"If the sovereign can realize the difficulty of being a sovereign, and the minister realize the difficulty of being a minister, then the government will be well ordered, and the common people will strive diligently after Virtue."

-- The Counsels of the Great Yu
Book of Documents as translated by James Legge

I am interested in anything which gets people's emotions on fire.  When I saw the twitter firestorm around Erik Meijer's presentation "One Hacker Way" I knew I had to see it for myself.

One Hacker Way - Erik Meijer from Reaktor on Vimeo.

It has been a few days since that viewing and overall I have to say he has some good points.

Is this the best presentation I've ever seen?  No.  Is every minute the best possible thing you could do we your time?  No.  Could it make you think about the way you do your work?  Most likely yes.

I got three points from the talk:

  1. We talk too much about code and do not write enough of it.
  2. Code which delivers value to someone is the ultimate goal, everything else is either giving context to the code or noise.
  3. We need to seek real feedback and adjust.
We talk too much about code and do not write enough of it.  Erik starts off the presentation asking his audience at a developer conference how many of them check in code last week?  He then tells those that did not that maybe they should leave.  Harsh but to the point.

There is a lot of talk about code in our industry, there are whole conferences and alliances around how best to get others to write code.  Do not take my word for it, read this blog post titled "Agile is Dead" by Dave Thomas an Agile Manifesto signer.  In his post Dave talks about how the term Agile has become meaningless, instead of following the principles leading to the Agile Manifesto a whole industry has formed to sell Agile.

"When the relationship between superiors and subordinates becomes disordered, at first the subordinates usurp the actuality (i.e. real power), but continue to preserve the name.  Once the usurpation has lasted for some time, though, the name is appropriated and usurped as well."

-- Wang Fuzhi
Commentary on Confucius Analects 13.14
Translated by Edward Slingerland

The term Agile has been usurped and has become meaningless.  At one time it did have a meaning and here is what it was:

Individuals and interactions over processes and tools
Working software over comprehensive documentation
Customer collaboration over contract negotiation
Responding to change over following a plan

-- Manifesto for Agile Software

Instead of prompting the left side of the manifesto the usurpers are selling the right side.  I believe this is is what Erik calls a disease of our industry.  Instead of talking about how to write code which delivers value and seeks feedback faster we are having people talk about tools, certifications, and overhead.

Code which delivers value to someone is the ultimate goal, everything else is either giving context to the code or noise.  This goes straight to the heart of the Agile Manifesto: "Working software over comprehensive documentation" and "Customer collaboration over contract negotiation".  If we are professionals then we need to focus on what the job is.  What is that job being asked of us?  Delivering working software with the features found through customer collaboration.  

We are not paid to move stickies across a board, we are paid to deliver working software which meets the customers needs.  This is what is meant by "Individuals and interactions over processes and tools" in the Agile Manifesto, tools and processes are to be used to reach the goal of working software which delivers value to the customer.  If you find that having stickies helps you in achieving this goal then use them.

"Our highest priority is to satisfy the customer through early and continuous delivery of valuable software."

-- Principles behind the Agile Manifesto

We need to "Responding to change over following a plan".  We need to seek real feedback and adjust.

"Business people and developers must work together daily throughout the project."

"At regular intervals, the team reflects on how to become more effective, then tunes and adjusts its behavior accordingly."

As Heraclitus would say, everything is in motions, everything is becoming, nothing is constant except change itself.  If we cannot change along with our customer's needs then we are dead.

Is "One Hacker Way" the greatest call to action ever?  No.  Do I agree with everything in it?  No.  Do I agree with most of it?  No.  Did I find the three main points valid?  Yes.

Sunday, November 9, 2014

Distinctly Preserving Order in Clojure

"Tell me, you heavens, in which part of his body
Shall I destroy him? – whether there, or there, or there? –
That I may give the local wound a name,
And make distinct the very breach whereout
Hector's great spirit flew: answer me, heavens!
-- Shakespeare, Troilus and Cressida
Act IV, Scene V, Lines 242-246

4clojure problem number 56 states:

Write a function which removes the duplicates from a sequence. Order of the items must be maintained.

The trick is you cannot use distinct.

We are giving the following test cases:

(= (__ [1 2 1 3 1 2 4]) [1 2 3 4])

(= (__ [:a :a :b :b :c :c]) [:a :b :c])

(= (__ '([2 4] [1 2] [1 3] [1 3])) '([2 4] [1 2] [1 3]))

(= (__ (range 50)) (range 50))

Let's try using group-by and get the first value of each group.

(fn [xs] (->> (group-by identity xs) (map first)))

This works fine for all but the last test case.  It seems that group-by does not preserve order.

(type (group-by identity (range 50)))
;; clojure.lang.PersistentHashMap

The PersistentHashMap in Clojure is implemented as a wide-tree with each node having up to 32 children (for more information see this excellent post by Karl Krukow or this excellent StackOverflow answer from Daniel Janus).

At this point we need to figure out a way to reorder the output from the group-by and first to what was given in the original input.  We can use sort-by but we need to use something that will give us the original input, this is were .indexOf saves the day.

We can use Java's .indexOf wrapped in a function to give sort-by an order which corresponds with the original input.  Using .indexOf will allow us to resort the output of the group-by followed by the first mapping to order that was in the original input, giving us this final function which preserve order and gives distinct values.

  (fn [xs] 
           (group-by identity xs) 
           (map first) 
           (sort-by #(.indexOf xs %))))

This passes all of the 4clojure's test cases.

Sunday, November 2, 2014

Technetium OR Reverse Interleave in Clojure

"Yet ‘ banished ’? Hang up philosophy!
Unless philosophy can make a Juliet,
Displant a town, reverse a prince's doom,
It helps not, it prevails not. Talk no more.
-- Shakespeare, Romeo and Juliet
Act III, Scene III, Lines 58-61

4clojure problem number 43 states:

"Write a function which reverses the interleave process into x number of subsequences."
For now let's look at the interleave process before we move on to reversing it.

ClojureDocs gives us the following:

"Returns a lazy seq of the first item in each coll, then the second etc."
In other words, give interleave a bunch of collections and it will place an item from each one into a result collection in a lazy way.  Something like this (based on the wikipedia entry):

Given sequences x and y with members i = 0, 1, 2, ... the interleave sequence of x and y is:
x0, y0, x1, y1, x2, y2, ...

Let's look at using interleave with the results of the test cases given by the 4clojure problem in a Clojure REPL.

(interleave '(1 3 5) '(2 4 6))
;; (1 2 3 4 5 6)

(interleave '(0 3 6) '(1 4 7) '(2 5 8))
;; (0 1 2 3 4 5 6 7 8)

(interleave '(0 5) '(1 6) '(2 7) '(3 8) '(4 9))
;; (0 1 2 3 4 5 6 7 8 9)

If we look at one of these in more detail we can see that it is following the definition given above.

(interleave '(0 3 6) '(1 4 7) '(2 5 8))
;; (0 1 2 3 4 5 6 7 8)

Or more mathematically

(interleave '(:x0 :x1 :x2) '(:y0 :y1 :y2) '(:z0 :z1 :z2))
;; (:x0 :y0 :z0 :x1 :y1 :z1 :x2 :y2 :z2)

Interleaving does not do a sort or anything like that, so if we move around the input we get different output:

(interleave '(1 4 7) '(0 3 6) '(2 5 8))
;; (1 0 2 4 3 5 7 6 8)

Now let's look at the problem at hand.

"Write a function which reverses the interleave process into x number of subsequences."

(= (__ [1 2 3 4 5 6] 2) '((1 3 5) (2 4 6)))

(= (__ (range 9) 3) '((0 3 6) (1 4 7) (2 5 8)))

(= (__ (range 10) 5) '((0 5) (1 6) (2 7) (3 8) (4 9)))

We need to come up with a function which will reverse the interleave function.  In other words, given a collection and the number of partitions we want returned, we get a collection of collections which can be interleave to return back the original collection given.

First thing we can do is set up our partitions using the partition function.

(fn [xs n]
    (->> (partition n xs) ))

This would give us the following using the test cases from 4clojure:

((fn [xs n]
      (->> (partition n xs) ))
[1 2 3 4 5 6] 2)
;; ((1 2) (3 4) (5 6))

((fn [xs n]
      (->> (partition n xs) ))
(range 9) 3)
;; ((0 1 2) (3 4 5) (6 7 8))

((fn [xs n]
      (->> (partition n xs) ))
(range 10) 5)
;; ((0 1 2 3 4) (5 6 7 8 9))

We now have each collection containing the number of partitions we want in the final result but everything is all mixed up!

If we look at the output for the first test case right now we have:
( (1 2) (3 4) (5 6) ) but what we want is:
( (1 3 5) (2 4 6) )

Looking at this closer we have the following:
( (1 2) (3 4) (5 6) ) but we want this:
( (1 3 5) (2 4 6) )

Looks like if we just need to map out the members in order and we'll get the result we want.  Following this will give us the following function:

(fn [xs n]
      (->> (partition n xs) (apply map list)))

Running this against our test cases gives us the following:

((fn [xs n]
      (->> (partition n xs) (apply map list)))
[1 2 3 4 5 6] 2)
;; ((1 3 5) (2 4 6))

((fn [xs n]
      (->> (partition n xs) (apply map list)))
(range 9) 3)
;; ((0 3 6) (1 4 7) (2 5 8))

((fn [xs n]
      (->> (partition n xs) (apply map list)))
(range 10) 5)
;; ((0 5) (1 6) (2 7) (3 8) (4 9))

This is exactly what we want.

Now I did not see this solution at first, this is what I submitted first to 4clojure:

(fn [xs n]
      (->> (map-indexed #(hash-map (mod %1 n) %2) xs) (group-by keys) (map second) (map #(mapcat vals %))))

Yep, very complex but it does work :(

As soon as I look at ramo and amcnamara's solutions I saw what I really wanted to do in my head but was unsure of how to with my Clojure knowledge, resulting in the final function I submitted:

(fn [xs n]
      (->> (partition n xs) (apply map list)))

Saturday, October 18, 2014

Bird Watching with Clojure -- The Thrush(y)

"With heigh, with heigh, the thrush and the jay,
Are summer songs for me and my aunts
Shakespeare, The Winter's Tale
Act IV, Scene III, Lines 10-11

The Thrush

As the wind gently rustles the leaves in the combinator forest.  You sing DO RA and hear sung back to you RA DO.  You sing RA ME and hear ME RA, you sing DO DO and hear DO DO back.  You have just heard the Thrush.

The Thrush or T combinator basically flips your parameters.

T x f = f x

In the Data Aviary Bird package for Haskell it is defined in the following way:

-- | T combinator - thrush.
-- Haskell @(\#)@ in Peter Thiemann\'s Wash, reverse application.
thrush :: a -> (a -> b) -> b
thrush x f = f x

We see that when we call it we pass a value as our first parameter followed by the function we wish to apply the value against.  This is useful when you want to pipe your data through a bunch of functions.

Thrush in F#

In fact in F# this is exactly what the |> (forward pipe) does.  It is defined in the following way:

let (|>) x f = f x

Yep, the forward pipe is nothing more than a Thrush!

Thrush in Joy

In Joy the Thrush goes by the name swap, which makes perfect sense.  It is defined in the following way:

[B] [A] swap == [A] [B]

Thrush in JavaScript

Reg Braithwaite of fame has a JavaScript library which is perfect for learning combinators called  The Thrush is defined in the following way in

function Thrush (a, b) {
    return, a)
  function T (a) { return function _T (b) {
    return b(a)

This exactly what we would expect looking at the Haskell and F# code above.

This is very similar to how Reginald defined the Thrush in Ruby in his excellent book, Kestrels, Quirky Birds, and Hopelessly Egocentricity:

Thrush(y) in Clojure

Clojure has two macros which are Thrushy (Michael Fogus has an excellent post on why they are Thrushy) the -> and ->> macros.

Say we wanted to do the following:

get the sum of the even integers from 1 to 100

We can break this requirement down into a few steps.

  1. get the integers from 1 to 100
  2. get the even integers from 1
  3. sum the integers from 2

What we see happening is a linking or piping of the results of a step down to the next step.

1 to 100 => evens => sum

Here are two ways to write this in Clojure:

The first way looks like a waterfall, but in order to understand it we need to go all the way to the bottom and work are way back up.  In this example we would start with the (range 101) then move up to the (filter even?) and lastly end at the (reduce +).  We find that reading this, we are going against the left to right flow of the code.

The second, Thrush(y) way, the first thing we see is our (range 101), next we find the (filter even?), and last we find the (reduce +).  This way seems very natural as the flow of data agrees with the reading from left to right.

I personally find the pipeline style used in the second way easier to read and use it as often as I can.  Martin Folwer has written an excellent article which goes into collection pipelines in more detail.  Debasish Ghosh has also written about the Thrush in Clojure (which help me a lot when I was trying to understand the -> and ->>macros).

Saturday, October 11, 2014

Try Before You Buy OR REPLs in the Browser

"Now follow – if thou darest – to try whose right"-- Shakespeare, A Midsummer Night's Dream
Act III, Scene II, Line 336

"Can't someone else do it?"
-- Homer Simpsons, Trash of the Titans

One of the hardest things to do is setting up a new programming environment.  Things have improve a lot over the years, but it still be a very daunting task to set up a programming environment for a language you do not even know.  Often in the past I would ask myself, "can't someone else do it?"

Someone else has!  Try a REPL in your browser today!

Luckily many programming environments are waiting for you right now as REPLs in your browser.

A REPL is a Read Evaluate Print Loop, which works how it sounds:

Read input (in the form of code)
Evaluate input
Print result of input
Loop back to top

Today you can now play around with many different programming language from the comfort of your browser by using a REPL.

Here are two that I enjoy:

Clojure instaREPL
Try Haskell!

There is also which has an impressive collection of languages including: Ruby, Forth, JavaScript, Roy, and even Brainfuck, to name a few.

Clojure instaREPL and Try Haskell! are a bit more like an actual REPL running on your computer than, but does offer languages which are not normally associated with REPLs (other than Ruby).

One last thing, while TryAPL is not really a REPL, it is very awesome that you can write APL code with a simulated APL keyboard in your browser!

'Try APL today!'

Sunday, September 28, 2014

How to Get 2 and 4 OR Learning Clojure in Public

"One heart, one bed, two bosoms, and one troth"
-- Shakespeare, A Midsummer Night's Dream
Act II, Scene II, Line 48

I believe in "learning in public" and I've been inspired recently, after watching Uncle Bob's "The Last Programming Language",  going to Strange Loop 2014, and the release of The Joy of Clojure 2nd edition, I've decided to learn me a Clojure for great good.  I've been going through the 4clojure problems as my morning kata for a few weeks now and I've hit problem #52 earlier in the week.

Intro to Destructuring

Let bindings and function parameter lists support destructuring.

(= [2 4] (let [[a b c d e f g] (range)] __))

The answer is [c e] giving the following expression.

(= [2 4] (let [[a b c d e f g] (range)] [c e]))

which evaluates to true
As I understand it, the way this works is that let binding maps the values from range over a through g which gives the a the value 0, b the value 1, c the value of 2, ..., e the value of 4, and so on.  Since we are looking for a vector with the values of 2 and 4, a vector with c and e will give us the [2 4] we are looking for.

How else can we do this?

We could take range, remove the zero value and pipe it through a filter to get just even numbers and then take the 2 and 4.

range -> remove zero -> filter evens -> take first two

In Clojure this would look like this:

(->> (range) (rest) (filter even?) (take 2))

which evaluates to (2 4)

If you want it in a vector you would need to convert the list into a vector which could be done using into, which would look like this:

(->> (range) (rest) (filter even?) (take 2) (into []))

which evaluates to [2 4]

In SQL on SQL Server this would look like this:

 @min AS INT = 0
,@max AS INT = 100

;WITH [range] AS (
  SELECT @min AS num
  SELECT num + 1 FROM [range] WHERE num < @max

  FROM [range]
  WHERE num > 0
    AND (num % 2) = 0

which gives us the result with a projection containing 2 and 4. 

We are creating a Tally Table using a CTE (as seen in Vinay Pugalia linked blog post), which will gives us similar functionality as the range in Clojure (similar because Clojure's range is lazy loaded and thus can be infinite).  We then filter using the WHERE clause to remove the zero like we do with rest in Clojure, next we filter to just evens using the % operator comparing those results to fine when they equal zero (yes, this not the best way to do this but it is similar).  Last we take the first two results by using TOP with 2.

This is very similar to what I blogged about here on doing FizzBuzz in SQL Server.

There you have it, two different solutions in two different languages on two very different platforms to get the first two even numbers greater than zero.

Learning in public can be fun.

Sunday, September 21, 2014

First Look at the Future OR How Seeing the Wolfram Language Reminded Me of Watching ECW

"Arms, and the man I sing, who, forc'd by fate, 
And haughty Juno's unrelenting hate,
Expell'd and exil'd, left the Trojan shore.
Long labors, both by sea and land, he bore,
And in the doubtful war, before he won
The Latian realm, and built the destin'd town;
His banish'd gods restor'd to rites divine,
And settled sure succession in his line,
From whence the race of Alban fathers come,
And the long glories of majestic Rome.
-- The Aeneid, Virgil
Translated by John Dryden
Book I, Lines 1-10

"Wow!" that is what I have down as the main take-away in my notes for Strange Loop 2014's session, "Inside the Wolfram Language".  I walked away feeling, as I believe, Hegel felt as he famously saw Napoleon, "I have just seen the world sprit".*

In a time when designers are aiming for yet smaller and smaller frameworks and languages, Stephan Wolfram proudly shows what one can do if they think different.  The Wolfram Language combines language, algorithms, and data into one complete platform (I can assure you I have not nor expect to get anything for these words), giving the user a lot of power in one console.

What would a Hello World look like in such a language?

Yes, that is both code and execution in a tweet!

That is nice and all but how about something more in-depth?

Here is my own "Hello World" Wolfram Language tweet.

I used the build-in text of the Aeneid and computed a histogram on the length of the words used in the english translation.

Histogram[StringLength[ToLowerCase[ExampleData[{"Text","AeneidEnglish"}, "Words"]]]]

Let's break this line down.

First we get the words of the Aeneid in a data structure.

ExampleData[{"Text","AeneidEnglish"}, "Words"]

Next we convert all the words to lower case so we do not have to worry about "The" not matching "the".

ToLowerCase[ExampleData[{"Text","AeneidEnglish"}, "Words"]]

Then we calculate the length of the different words.

StringLength[ToLowerCase[ExampleData[{"Text","AeneidEnglish"}, "Words"]]]

Last, we graph the data as a histogram.

Histogram[StringLength[ToLowerCase[ExampleData[{"Text","AeneidEnglish"}, "Words"]]]]


CloudDeploy[Histogram[StringLength[ToLowerCase[ExampleData[{"Text","AeneidEnglish"}, "Words"]]]]]


Not bad for a one-liner.

Just one more thing, if we tweet to the program to @wolframtap it will tweet back the results.

Very nice.

* Actual text from Hegel's correspondence to Niethammer, "I saw the Emperor -this soul of the world- go out from the city to survey his reign; it is a truly wonderful sensation to see such an individual, who, concentrating on one point while seated on a horse, stretches over the world and dominates it." (as quoted here)

Sunday, September 7, 2014

JavaScript Scopes to Functions Not Files

"The cloud-capped towers, the gorgeous palaces,
The solemn temples, the great globe itself,
Yea, all which it inherit, shall dissolve,
And, like this insubstantial pageant faded,
"Shakespeare, The Tempest
Act IV, Scene I, Lines 152-155

While taking this Pluralsight class by Joe Eames I saw a short demo of how in JavaScript variables are scoped to functions and not to files.  I kind-of-all-ready knew this but have not really looked into it.

Here is a quick demo to show what it means to the browser.

See this Plunker if you want to play around with the code.

The browser loads the index.html file which cause it to load and execute a.js, b.js, and c.js.

  • index.html
    • a.js
    • b.js
    • c.js
The browser executes the JavaScript in the following order.
  1. a.js executes adding the variable a to the global name space (which in the browser is window)
  2. b.js executes calling console.log with the variable a
  3. c.js executes adding the function f to the global name space
  4. c.js then executes the function f (this is called an iffe
    1. which adds the variable x to function f's scope
    2. then calls console.log with the variable x
  5. c.js then tries to call console.log with a variable x but there is not an x in the global name space
With the scope looking like the following.
  • window
    • variable a
      • "this is from a.js"
    • function f
      • variable x
        • "scoped to function f"
Looking at the scope it is easy to see why the call to console.log with a in the file b.js works but the call to console.log with x in the c.js outside of the function f fails.

This is nothing new, but with JavaScript taking off in popularity this is something to keep in mind since it is very different than how most programming languages work.

Sunday, August 24, 2014

Functional Programming All the Things OR How to Use Curry to Make Your Test More Readable

"I read in's looks"
-- Shakespeare, Henry VIII
Act I, Scene I, Line 125.2

We, the software craftsmen, understand that tests are a first class citizen.  If the system code was deleted it would be a great opportunity to rewrite the system and know once all the tests pass that the system has all the same functionality.  The question is should tests have the same style as the system code?

Say you have the Coin Changer kata and you want to use a function which takes both the coin value and the amount of change to be given.

If we wrote this in a C-style interface it look something like the following:

int[] changeFor(int[] coins, int amount)

While testing this we have a choice: we can either pass in the coins each time in the test thus violating the DRY principle or we could some how fill in the coins we want to use on each test in a testing series.  The second choice is very interesting since it is screaming for currying the coins per testing series.  Following this idea we have the following for the Coin Changer kata using AngularJS.

We have the following for the system code of the changeFor function.

    $scope.changeFor = function(coins, amount) {
      if (_.isEmpty(coins)) return [];

      return _.reduce(coins, function(m, coin) {
        m.change.push(Math.floor(m.amount / coin));
        m.amount %= coin;
        return m;
      }, {
        amount: amount,
        change: []

We see that lodash's isEmpty makes sure that we have an array of coins before we go to the aggregation using the reduce function.  In the reduce we create an object with the current amount and change calculated so that we do not have to mutate the values passed in to the changeFor function.

We test the cases when we have no coins being passed in with the following code from appSpec.js.

describe('given no coins', function() {
var noCoinsFor;
beforeEach(function() {
noCoinsFor = _.curry($scope.changeFor)([]);
it('should return nothing for 0 cents', function() {
it('should return nothing for 99 cents', function() {

We set up changeFor function by currying with an empty array for the coins in the beforeEach in the top level describe.  Using curry in the beforeEach we do not have to pass in the empty array on each call in the its.  This makes the test a bit more readable.

Similarly when we test the penny series we can curry the penny.

describe('given pennies', function(){
var penniesFor;
penniesFor = _.curry($scope.changeFor)([1]);
it('should return nothing for 0 cents', function(){
it('should return 1 penny for 1 cents', function(){
it('should return 2 pennies for 2 cents', function(){

This makes the grouping in the describe seem more logical and the test more readable.

The same with the nickel and penny.

describe('given nickels and pennies', function(){
var nickelsPenniesFor;
nickelsPenniesFor = _.curry($scope.changeFor)([5, 1]);
it('should return nothing for 0 cents', function(){
expect(nickelsPenniesFor(0)).toEqual([0, 0]);
it('should return 1 penny for 1 cents', function(){
expect(nickelsPenniesFor(1)).toEqual([0, 1]);
it('should return 1 nickel and 1 penny for 6 cents', function(){
expect(nickelsPenniesFor(6)).toEqual([1, 1]);

Finally our integration tests for US coins.

describe('given quarters, dimes, nickels, and pennies', function(){
var changeFor;
changeFor = _.curry($scope.changeFor)([25, 10, 5, 1]);
it('should return nothing for 0 cents', function(){
expect(changeFor(0)).toEqual([0, 0, 0, 0]);
it('should return 1 penny for 1 cents', function(){
expect(changeFor(1)).toEqual([0, 0, 0, 1]);
it('should return 3 quarters, 2 dimes, 0 nickels and 4 pennies for 99 cents', function(){
expect(changeFor(99)).toEqual([3, 2, 0, 4]);

By using the curry function we have added to the readability of the Jasmine tests.  This style is a bit different than the system code in the AngularJS application.

Check out the plunker which goes with this post.