Pipeline operators in JavaScript

javascript8 min

byLucas Santos

This page was machine translated. Read original / Suggest a fix

JavaScript is always evolving and, as usual, I’m going to write about yet another proposal that’s been gaining traction in the community. The pipeline operators. This proposal is still at stage 1, meaning very early in the process, but it’s already been dragging on for about 6 years. You can still test it online though, using Babel.

If you don’t already know how JavaScript works and how it evolves, I invite you to watch my video explaining a bit about this topic:

Play

This isn’t the first time that pipeline operators have been suggested for the language (actually, it’s the third), but this time it might be a bit different because we have another set of information we can use to complete this puzzle.

What’s the proposal#

Pipeline operators translate to flow operators, and the idea is basically the same as the .pipe function we have in streams (which I already explained here, here and here): essentially, they work by calling functions and passing the output of one function to the input of another, pretty similar to what bash’s |, for example, does.

The biggest difference is that, unlike |, which only accepts unary functions, meaning functions with a single input parameter (like (x) => {}), pipe operators are supposed to accept any type of operation.

To understand a bit better how these operators work and why they were suggested for the language, we first need to understand two programming styles, two ways of writing code: deep nesting and fluent interfaces. And then know a bit about the history behind functional languages.

Deep Nesting#

When we talk about pipelines, we’re basically talking about sequential function execution, meaning the result of a function or expression gets passed to the next one, like a cake recipe, where after each step we take what we already have and pass it to the next stage of the process until we get a final result.

A great example of this is the array’s reduce function, which basically applies the same function consecutively over a set of values that gets modified, passing the result of the previous execution’s set to the next one:

const numeros = [1,2,3,4,5]
numeros.reduce((atual, acumulador) => acumulador + atual, 0)
// 1 => { atual: 1, acumulador: 0 }
// 2 => { atual: 2, acumulador: 1 }
// 3 => { atual: 3, acumulador: 3 }
// 4 => { atual: 4, acumulador: 6 }
// 5 => { atual: 5, acumulador: 10 }
// 6 => { atual: undefined, acumulador: 15 }
// 7 => resultado 15

This can also be done with what’s called nesting, which is when we pass one function execution into another consecutively, so imagining we had the sum function we used in the reduce above, we could represent that same function like this:

function soma (a, b) { return a + b }
soma(5,
     soma(4,
          soma(3,
               soma(2,
                    soma(1, 0)
                   )
              )
         )
    )

I think it’s easy to see the problem here… Deep nesting, along with currying, are techniques that, despite being fairly common in object-oriented languages too, are much more common in languages with more functional approaches like Hack, Clojure and F#. That’s because these languages, as the name suggests, are based on functions for working with data in a way that’s a bit closer to the system known in mathematics as Lambda Calculus.

The point is that deep nesting is really hard to read, because we don’t know where the initial data is coming from, and also because reading has to start from the inside out (or from right to left), since we need to know the result of the first function passed in order to infer the result of the last call.

On the other hand, deep nesting applies to pretty much every type of expression: we can have arithmetic operations, arrays, await, yield, and all sorts of things. For example, the function above could (and probably will, inside the compiler) be written like this:

const resultado = (5 +
 (4 +
  (3 +
   (2 +
    (1 + 0)
   )
  )
 )
)

Currying is when we have functions that are unary by nature, so when we want to compose something, we return a function that’s going to call another function. That way we can compose the two functions as if they were two calls, for example, a function that multiplies two numbers:

const multiplicaDois = x => y => x * y
const resultado = multiplicaDois(5)(2) // -> 10

Currying, despite being elegant, is a bit costly because we have to type quite a bit more, and on top of that, longer and more complex functions end up getting harder for anyone to read. Still, currying is widely used, especially by libraries like Ramda, which are built around currying from the ground up.

But there’s another way of writing code that most of us are already a bit used to: fluent interfaces.

Fluent Interfaces#

You’ve probably run into fluent interfaces at some point in your life, even if you don’t know what we’re talking about. If you’ve ever used jQuery, or even the most common JavaScript array functions, you’ve already used a fluent interface.

This type of design is also called method chaining.

The big idea behind fluent interfaces is that you don’t need to call the object again in order to run a different, but subsequent, function on the same data from your original object, for example:

const somaDosImpares = [1, 2, 3]
  .map(x => x * 2)
  .filter(x => x % 2 !== 0)
  .reduce((prev, acc) => prev+acc, 0)

The biggest example so far of this architecture model is jQuery, which consists of a single main mega-object called jQuery (or $) that gets dozens and dozens of child methods, all of which return the same main object, so you can chain all of them together. This also looks a lot like a design pattern called builder.

Notice that I’m not calling my array again, I’m simply chaining (that’s where the term “chaining” comes from) the methods of that array one after another, and I get the closest thing we have today to an interface that’s both highly readable and also mimics the flow behavior we want to achieve with pipeline operators.

The problem is that this method’s applicability is limited, because it’s only possible if you’re working within a paradigm that has functions designated as methods on a class, meaning when we’re working directly with object orientation.

But on the other hand, when it’s applied, reading and usability become so easy that a lot of libraries hack their code together just to be able to use method chaining. Think about it, when we have this type of design:

  • Our code flows from left to right, like we’re used to
  • Every expression that could otherwise get nested stays at the same level
  • All the arguments are grouped under the same main element (the object in question)
  • Editing the code becomes trivial: if we need to add more steps, we just drop in a new function in the middle; if we need to remove one, we just delete the line

The biggest problem is that we can’t fit every interface and function type into this same design, because we can’t return arithmetic expressions (like 1+2), or await, or yield, nor object literals or arrays. We’re always going to be limited to what a function or method can do.

Enter the pipe operators#

Flow operators combine both worlds and improve the applicability of both models into a more unified and more readable interface. So instead of having a bunch of nested methods, or a bunch of functions, we can simply do this:

const resultado = [1,2,3].map(x => x*2) |> %[0] // => 2

The syntax is simple: on the left side of the |> operator we have any expression that produces a value, and that produced value gets thrown into a placeholder (or temporary object) that, for now, is written as %, meaning % is the result of whatever is on the left of the |>. Then, on the right side of the operator, we have the transformation applied to the obtained result. The final result of these two expressions is the output, and that’s what gets assigned to resultado.

If you check it out using Babel, for the code below:

const toBase64 = (d) => Buffer.from(d).toString('base64')


const baseText = 'https://lsantos.dev'
|> %.toUpperCase()
|> toBase64(%)

We’re going to get the following output:

"use strict";


const toBase64 = d => Buffer.from(d).toString('base64');


const baseText = toBase64('https://lsantos.dev'.toUpperCase());

Likewise, if we use functions with currying, babel will be able to decipher that information and produce a valid representation.

Right now there are two most well-known implementations of pipe. The first one is F#‘s, a functional programming language created by Microsoft based on OCaml. The second is Hack’s, a language created by Facebook a good while back that is, essentially, PHP with static types.

The biggest difference between the operators is that, in Hack’s version, it accepts any type of expression as a valid operand on both the left and right side of the expression, through the special variable %.

So we can do literally anything:

value |> someFunction(1, %, 3) // function calls
value |> %.someMethod() // method call
value |> % + 1 // operator
value |> [%, 'b', 'c'] // Array literal
value |> {someProp: %} // object literal
value |> await % // awaiting a Promise
value |> (yield %) // yielding a generator value

In F#‘s case, though, we’re a bit more limited to unary functions, so the % variable doesn’t exist. That means we always need some kind of function on the right side of the operator:

const f = soma(1,2) |> x => soma(x, 3)

Among other reasons explained here, the proposal is focusing mainly on being able to apply Hack’s model to JavaScript, not F#‘s.

Conclusion#

For now this operator is still trying to get off the ground, but there are already plans described in this section showing that a few other options for extending the operator are already under consideration, like conditional and optional operators using if or ?, and loop operators with for of, plus using this operator with catch.

There’s still no date or timeline for this proposal to become reality, but plenty of eyes are on what’s happening!