Function Composition, Function Chaining, Currying, and Partial Functions / Application in F# and Ruby

Introduction

A while ago I wrote an article comparing / contrasting language elements in Ruby with C# with regards to classes: constructors, destructors, fields, properties, initializers, events, methods, etc.  Ruby however also has a foot in the functional language paradigm, though it requires a certain discipline to ensure that you're adhering to qualities of a functional programming language such as statelessness and immutability.  However, for purposes of this article, what I am more interested in is exploring how Ruby handles certain concepts that seem to be core to most functional programming languages:

• function composition
• function pipelining (chaining)
• continuation functions
• currying
• partial functions
• partial application

These concepts are all intertwined with each other, and as we'll see, you can certainly implement these behaviors in Ruby, though not as elegantly as with F# (or other FP languages.)  Incidentally, one of the reasons Ruby gets away with being called an FP language is its support for lambda expressions and the ease in which code blocks yielded to for iteration, application specific handling (like IoC), and so forth.  Again, those are not areas that I'm interested in pursuing in this article because at this point they are part of most imperative languages as well, such as C#, further blurring the lines between imperative, declarative, and functional programming.

Background

The impetus for this investigation was a little helper function that I ended up writing a couple days ago.  This is a bit of a diversion, so sit back and enjoy the brief detour.  One of the interesting things about Ruby on Rails development is the ability to write "features" that define "scenarios" comprised of "steps" that test the behavior of the website.  This is all part of the Behavior Driven Development paradigm many Ruby developers adhere to, and this capability is provided by a package called Cucumber, which also works with .NET.  Other BDD testing paradigms in the .NET world are SpecFlow and NSpec.

Like anything that is intended to be helpful, it results in its own set of problems.  For BDD, this becomes the large number of steps that an application defines, often across numerous files.  Steps look like this:

When /^(?:|I )go to (.+)$/ do |page_name|
  visit path_to(page_name)
end

or:

When /^(?:|I )fill in "([^"]*)" for "([^"]*)"(?: within "([^"]*)")?$/ do |value, field, selector|
  with_scope(selector) do
    fill_in(field, :with => value)
  end
end

What's a developer to do to keep track of it all, and how can we document what all the different parameters are?  Well, there's a nifty way for Ruby to inspect itself and so, borrowing from the work of others, I wrote a small set of functions to sort all the BDD testing steps and display them as an HTML document.  Here's a snippet of what it looks like in its output:

You can see all the Ruby code on my blog, but the salient point is this code:

def step_definitions_to_html
  # Where are your step definitions defined?
  step_definition_dir = "./features/step_definitions"
  # Where do you want output to go:
  outfile = "output.htm"

  steps = get_step_definitions(step_definition_dir)   # get the step definitions
  steps = sort_steps_by_name(step)	              # sort them
  output_step_definitions_as_html(outfile, steps)     # output as HTML
  show_html(outfile)                                  # show the file in the browser
end

which in my opinion is horribly imperative.  I thought it would be interesting to write this as a function composition or function pipeline, looking, abstractly, like this:

get_steps(input_path) >> sort_steps >> display_steps(output_file)

What I didn't realize was the number of rabbit holes I would have to go down with Ruby for something that ultimately is very simple in F#.

Function Pipelining

Let's make the problem a little simpler.  Let's say:

I have a function "a" that converts an integer to a string.

F#:

let a n = Convert.ToString(n:int32);;
val a : n:int32 -> string

Read this as: a is a function that takes an int32 and returns a string.

Ruby:

def a(n) n.to_s; end

I have a function "b" that appends a fixed string with two parameters

F#:

let b n intstr = Convert.ToString(n:float) + ", " + intstr;;
val b : n:float -> intstr:string -> string

Read this as: b is a function that takes a float and a string and returns a string.

Ruby:

def b(n, instr) n.to_s + ", " + instr; end

The reason we're doing this with different types is because it's easier to see what the intermediate functions are when they have different types - we'll be looking at function pipelining and composition with the types int, string, and float.

One of those two parameters is the output of function "a", which converts an integer to a string, such that I can write the function chain:

F#:

3 |> a |> b 1.2;;
val it : string = "1.2, 3"

Given the function b that we defined above:

let b n intstr = Convert.ToString(n:float) + ", " + intstr

We can see that the function a evaluates to "3".  This becomes the input to the last parameter of function b, resulting in a curried function in which b's first parameter (n) still needs to be provided:

let b n intstr = Convert.ToString(n:float) + ", " + "3"

However, because this is function pipelining, we can't create an intermediate function (which would be function composition).  So, for example, this results in an error:

 let r = 3 |> a |> b;;
------------------^

stdin(146,19): error FS0001: Type mismatch. Expecting a
string -> 'a 
but given a
float -> string -> string 
The type 'string' does not match the type 'float'

So, with function pipelining, we need to provide all the parameters so that the entire pipeline can be evaluated.

How do we do this in Ruby?  The problem with the word "pipelined" is that some people in Ruby-land interpret it as running in a separate thread (or fiber), not waiting for the function to continue before executing the next function in the pipe.  This isn't what we want.  Then, the problem with the word "chaining" is that almost everyone in Ruby-land interprets it as function that return "self" (the equivalent to "this" in C#) so that another function of the object can then be called, resulting in notation like:

a(3).b("def")

which of course isn't what we want either because a(3) would returns an object, not "3"!  What we want to do is preserve the original meaning of the function rather than having to coerce a particular coding style.

Now, going back to the F# code, the pipeline operator is defined to simply swap the function and the argument:

let inline (|>) x f = f x

To do something like this in Ruby, we have to go down the rabbit holes of lambda expressions, the Proc class, and for syntax-sugar purposes, operator overrides.

Extending Ruby With Functions

Function pipelining (and as we'll see later, function composition) are not built-in behaviors of the Ruby language.  However, we can create functions that behave this way by converting our Ruby functions and values into lambda expressions.  This allows us to define operators on the expression as well as evaluate it.  So, first, let's define a lambda that simply returns the value 3:

l3 = lambda {3}
=> #<Proc:0x2a12f60@(irb):33 (lambda)>

As you can see, this return a Proc object.  We can evaluate the expression (note the syntax, there's three different ways of evaluating lambda expressions!):

l3.()
=> 3

And we see that it returns 3.

Now we need a lambda that encapsulates our function a, which converts numbers to the strings:

la = lambda{|x| a(x)}
=> #<Proc:0x2a1e480@(irb):43 (lambda)>

And if we evaluate this:

la.(3)
=> "3"

We get the expected string conversion.

Now we just need to do the equivalent of:

la.(l3.())
=> "3"

using an operator to get the syntactical sugar we want.  This involves overriding the operator:

class Proc
  def self.chain(f, g)
    lambda { g[f.()] }
  end
  def >=(g)
    Proc.chain(self, g)
  end
end

Here, the >= operator is overloaded (Ruby only allows overloading of operators that are already defined), such that the chain function is called with two parameters, the left operand and the right operand.  The function chain returns a lambda expression that evaluates the right operand, passing in as a parameter the evaluated lambda expression of the left operand.  Thus we can see the equivalent of the F# implementation:

let inline (|>) x f = f x

Now, when we enter into our Interactive Ruby Console (IRB):

(lambda{3} >= lambda{|n| n.to_s}).()

this evaluates to the string "3".

Now, this is a lot of typing and looks ugly as sin, and remember, we want this to work with existing Ruby functions, such as those we had defined earlier:

def a(n) n.to_s; end
def b(n, instr) n.to_s + ", " + instr; end

So we'll create a couple helper functions:

def val(n) lambda {n}; end
def fnc(f) lambda(&method(f)); end

The function val takes a value and convert it into a lambda expression.  The function fnc, given a function, converts the function into a lambda expression.  We can now write:

(val(3) >= fnc(:a)).()
=> "3"

Note how we "symbolize" function a.

But where do we put the parameter value "1.2" if we want to chain in function b?  The answer lies in our exploration of the F# code above and going down another Ruby rabbit hole, currying.

Currying

The next rabbit hole on our journey into Ruby-in-Functional-land is to realize that, because function b has two parameters, what we need to do is curry the function, passing in the value from the left side of the >= operator and leaving the rest of the parameters to be specified by the caller.  We can do this explicitly:

(val(3) >= fnc(:a) >= fnc(:b).curry.("1.2")).()
=> "1.2, 3"

Thus, the string "3" will be passed in as the parameter intstr, leaving the curried function simply as:

def b(n, instr) n.to_s + ", " + "3"; end

We can make this a little cleaner with a currying "make function into a lambda expression" function:

def fncc(f, *args) lambda(&method(f)).curry.(*args); end

And express the function chain thus:

(val(3) >= fnc(:a) >= fncc(:b, 1.2)).()
=> "1.2, 3"

Behind the scenes, when fnc(:a) >= fncc(:b) evaluates, the value to the left of the operator is passed in to the function b as the last parameter, and the fncc returns a partial function (as a lambda) that takes the remaining (first) parameter.

So.  We have accomplished function chaining a la F#, though it's rather awkward because we have to be explicit about the currying (as well as converting functions to lambda expressions), whereas the equivalent in F#:

3 |> a |> b 1.2;;
val it : string = "1.2, 3"

is much cleaner. 

If Only...

Note that if we didn't have to deal with those pesky additional parameters in the function b, we could write:

(val(3) >= :a).()

if we were to change the >= operator definition to perform the "lambda-ization" for us:

class Proc
  def self.chain(f, g)
    lambda { g.(f.()) }
  end
  def >=(g)
    Proc.chain(self, fnc(g))
  end
end

Unfortunately, doing this functions that require additional paramerters just doesn't work.

Function Composition

Truth be told, what we've really done is actually more like function composition because we've been turning even values, like 3, into lambda's.  However, function composition behaves a bit differently in F#, and, leveraging the work we've already done, we can implement function composition in Ruby similarly.

First off, in F#, we can't do this:

 3 >> a >> b 1.2;;
 ^

stdin(92,1): error FS0001: This expression was expected to have type
'a -> 'b 
but here has type
int 

Function composition expects a function, not a int.

But we can do this:

(a >> b 1.2) 3;;
val it : string = "1.2, 3"

Here we have created a composed partial function:

a >> b 1.2;;
val it : (int32 -> string) = <fun:it@152-28>

The composite function is expecting an integer and returns a string.  It's a partial application because we've already applied the last parameter to the function b.

Note again how F# has done the currying implicitly, except now we can assign the composite function:

let r = a >> b 1.2;;
val r : (int32 -> string)

> r 3;;
val it : string = "1.2, 3"

Remember how in F#, the pipeline operator simply switches the parameters:

let inline (|>) x f = f x

Well, with function composition, the >> composition operator does the same thing:

let inline (>>) f g x = g(f x)

Here, reversing f and g, such that g is called with the result of the computation of "f x".

We can define the same operator and behavior in Ruby:

class Proc
  def self.compose(f, g)
    lambda { |*args| g[f[*args]] }
  end
  def >>(g)
    Proc.compose(self, g)
  end
end

and, having created a few helper functions earlier, we can now compose functions (notice the explicit currying in the fncc function):

(fnc(:a) >> fncc(:b, 1.2)).(3)
=> "1.2, 3"

Which compare this to the F# syntax above:

(a >> b 1.2) 3;;

Summarizing What We've Done So Far

F# Function Pipelining

let a n = Convert.ToString(n:int32);;
let b n intstr = Convert.ToString(n:float) + ", " + intstr;;
3 |> a |> b 1.2;;

Ruby Function Pipelining

class Proc
  def self.chain(f, g)
    lambda { g[f.()] }
  end
  def >=(g)
    Proc.chain(self, g)
  end
end

def val(n) lambda {n}; end
def fnc(f) lambda(&method(f)); end
def fncc(f, *args) lambda(&method(f)).curry.(*args); end

def a(n) n.to_s; end
def b(n, instr) n.to_s + ", " + instr; end

(val(3) >= fnc(:a) >= fncc(:b, 1.2)).()

F# Function Composition

let a n = Convert.ToString(n:int32);;
let b n intstr = Convert.ToString(n:float) + ", " + intstr;;
(a >> b 1.2) 3;;
let r = a >> b 1.2;;
r 3;;

Ruby Function Composition

class Proc
  def self.compose(f, g)
    lambda { |*args| g[f[*args]] }
  end
  def >>(g)
    Proc.compose(self, g)
  end
end

def val(n) lambda {n}; end
def fnc(f) lambda(&method(f)); end
def fncc(f, *args) lambda(&method(f)).curry.(*args); end

def a(n) n.to_s; end
def b(n, instr) n.to_s + ", " + instr; end
(fnc(:a) >> fncc(:b, 1.2)).(3)
r = (fnc(:a) >> fncc(:b, 1.2))
r.(3)

An Alternative Implementation for Function Pipelining

Jörg W Mittag answer my stackoverflow question regarding an equivalent to F#'s function chaining with a different implementation not involving operator overloading:

class Object
  def pipe(callable)
    callable.(self)
  end
end

Notice that the pipe method is being defined in the Object class, and also notice the idiomatic "->" syntax to define a lambda.

We can use this code like this (leveraging our other function helpers):

3.pipe(fnc(:a)).pipe(fncc(:b, 1.2))
=> "1.2, 3"

Notice how we don't need to convert the literal value 3 into a lambda because we're extending the Object class, and everything in Ruby is an object, thus, "pipe" becomes a function that can be applied to the number 3.  The lambda function is passed in and then evaluated, passing the object that called pipe.  This effectively implements "x.pipe(f) = f(x)".  A very snazzy piece of code!

Where Are We Now?

Remember that Ruby code I wanted to change into a more functional programming style implementation?

def step_definitions_to_html
  # Where are your step definitions defined?
  step_definition_dir = "./features/step_definitions"
  # Where do you want output to go:
  outfile = "output.htm"
  steps = get_step_definitions(step_definition_dir)   # get the step definitions
  steps = steps.sort_by {|step| step.regex}           # sort them
  output_step_definitions_as_html(outfile, steps)     # output as HTML
  show_html(outfile)                                  # show the file in the browser
end

It would now look like this:

def step_definitions_to_html
  step_definition_dir = "./features/step_definitions"
  # Where do you want output to go:
  outfile = "output.htm"

  steps = val((step_definition_dir) >=
          fnc(:get_step_definitions) >=                        # get the step definitions
          fnc(:sort_steps_by_name) >=	                       # sort them
          fncc(:output_step_definitions_as_html, outfile) >=   # output as HTML
          fnc(:show_html)                                      # show the file in the browser
end

Conclusion

At the end of the day, I probably wouldn't even write this code using pipelining in F#, but rather resort to the more "imperative-ish" style of using local variables to store the intermediate results.  None-the-less, this has been a interesting journey into some of the more esoteric aspects of the Ruby language.

Acknowledgements and References

Jörg W Mittag on stackoverflow for his excellent answer suggesting an alternative implementation for function pipelining
Ruby Functional Programming
Composing Functions in Ruby
Functions (F#)
The Ruby Language, Operator Expressions
Understanding Ruby Blocks, Procs, and Lambdas 
Ruby Currying