"What are generators for in JS?"

Two of the most interesting, and also most complex, structures in JavaScript are iterators and generators. These two structures are not new. In fact, I wrote about both generators and iterators back in 2017. But even though these two structures (especially iterators) are heavily used in many frameworks and even in core language constructs like Promises and our beloved async/await, generators are still not widely known or used.

This article is exactly about that! You’ll understand how generators can be used in your applications in a more practical way.

What are generators?#

First, let’s recap what generators are. They are a low-level construct, typically used to build other tools. A generator is a function that returns a generator object, which is an object that implements the iterable protocol. In other words, it has a Symbol.iterator that is used to perform loops.

The difference is that a generator function returns a special type of iterator that can pause its own execution while maintaining its internal state and context. A generator is declared with a * in front of a function, like this:

1
function *generator() {
2
    yield 1
3
    yield 2
4
    yield 3
5
}

Or:

1
function* generator() {
2
    yield 1
3
    yield 2
4
    yield 3
5
}

Both are the same and can be used interchangeably. These generators return an iterator, meaning you can call .next() on each of these iterators, and when executed, the value after yield will be returned on each call. For example:

1
function* generator () {
2
    yield 1
3
    yield 2
4
    yield 3
5
}
6

7
const g = generator()
8
console.log(g.next()) // 1
9
console.log(g.next()) // 2
10
console.log(g.next()) // 3

We can also use generators with the spread operator, for example:

1
function* generator () {
2
    yield 1
3
    yield 2
4
    yield 3
5
}
6

7
const g = generator()
8
console.log([...g]) // [1, 2, 3]

But I won’t go too deep here. If you want to understand what generators are, I recommend reading the article I mentioned as it will give you a solid foundation for understanding everything.

Uses of Generators#

Now let’s stop discussing what generators are and talk about why they exist.

Lazy iterators#

This is probably the most common use of a generator. When I said that generators are special cases of iterators, I also said they have a very unusual characteristic. They can pause their execution.

What does that mean? Basically, calls to a generator, whether through a loop like for .. of or .next(), are only executed at that specific moment. For example:

1
function* jsFacts() {
2
    yield 'The most present language on the Web'
3
    yield 'Created in 1995 by Brendan Eich'
4
    yield 'Can be used on the backend with Node, Deno and others'
5
}
6

7
for (let fact of jsFacts()) {
8
    console.log(`JS: ${fact}`)
9
}
10

11
// JS: The most present language on the Web
12
// JS: Created in 1995 by Brendan Eich
13
// JS: Can be used on the backend with Node, Deno and others

What’s happening here is that the strings won’t exist until the next item is called. While for strings this might seem pointless, generators as lazy iterators can be widely used to build recordSets, which are structures that fetch data from, for example, a database, and instead of returning all values at once, return them one by one so as not to overload memory.

For example, imagine we have a database with terabytes of information, we want to get all the data, but we don’t want it all at once. We can build a lazy iterator that calls the database to return only one value at a time:

1
const linha = [{
2
  nome: 'Alan Turing',
3
  id: 1,
4
  idade: 42,
5
  titulo: 'Pai da computação'
6
}, {
7
  nome: 'Ada Lovelace',
8
  id: 2,
9
  idade: 36,
10
  titulo: 'Primeira programadora'
11
}, {
12
  nome: 'Grace Hopper',
13
  id: 3,
14
  idade: 85,
15
  titulo: 'Inventora do compilador'
16
}]
17

18
const findInDatabase = (skip, limit) => {
19
  return linha.slice(skip, skip + limit)
20
}
21

22
function* recordSet() {
23
  let skip = 0
24
  const limit = 1
25
  let currentRecord = findInDatabase(skip, limit)
26

27
  while (currentRecord.length > 0) {
28
    skip += limit
29
    yield currentRecord[0]
30
    currentRecord = findInDatabase(skip, limit)
31
  }
32
}

If we pay attention to what we’re doing, we’ll see that we’re defining an internal state with a skip variable, which is the number of records we want to skip from the array (which is our database). We do the first iteration and save the current record in currentRecord, which is another part of the internal state. Then we can create a loop validating whether the result received is valid, meaning whether it still has records in the database. If yes, we add the skip to the limit of data we want (in this case just 1 at a time), return the current result, and then pause execution.

Whenever we initialize our generator with:

1
const records = recordSet()

We execute all the code up to yield currentRecord[0]. Then, when we do:

1
const records = recordSet()
2
console.log(records.next()) // {value: { nome: 'Alan Turing', id: 1, idade: 42, titulo: 'Pai da computação' }, done: false }

We execute our generator and get the value coming from yield. Then we execute the loop again until the next yield.

We can also use another variation of the generator so we don’t have internal state and define the stopping condition inside the loop like this:

1
function* recordSet() {
2
  let skip = 0
3
  const limit = 1
4

5
  while (record.length > 0) {
6
    const record = findInDatabase(skip, limit)
7
    if (record.length === 0) return
8
    skip += limit
9
    yield record[0]
10
  }
11
}

The values we get from next are objects of type {value: any, done: boolean} that come from the iterator we’re accessing.

Infinite ranges#

Another thing we can do is create an infinite counter:

1
function* infiniteSequence() {
2
  var i = 0;
3
  while (true) {
4
    yield i++;
5
  }
6
}

What might not seem like much, but it’s a powerful tool for global monitoring. For example, if we want to count how many promises an application created during its lifetime, we can do something like this:

1
function* sequence() {
2
  var i = 1
3
  while (true) {
4
    yield i++
5
  }
6
}
7

8
const promisesCreatedCounter = sequence()
9
let promisesCreatedTotal = 0
10

11
const proxyPromise = new Proxy(Promise, {
12
  get(target, prop) {
13
    if (prop === 'prototype') {
14
      promisesCreatedTotal = promisesCreatedCounter.next().value
15
    }
16
    return target[prop]
17
  }
18
})
19

20
const p = new proxyPromise((resolve) => {
21
  resolve('done')
22
})
23

24
p.then(() => {
25
  console.log('Promise resolved')
26
})
27

28
console.log('Promises created: ' + promisesCreatedTotal) // Promises Created: 1

Furthermore, each instance of such a sequence is unique, which means you can reuse the same counter in different places and it will always start from 1.

Utility functions#

As I mentioned before, it’s possible to build many tools from generators because they are a low-level tool. Some of the functions we can build with them are, for example:

take#

A function that, given an iterable, will take n number of elements from that iterable and return them:

1
const take = (n) => function*(iteravel) {
2
    let i = 0
3
    for (let elemento of iteravel) {
4
        if (i >= n) return
5
        yield elemento
6
        i++
7
    }
8
}

If we have an array like [1,2,3,4] and do take(4)([1,2,3,4,5,6]) we get [1,2,3,4]

repeat#

A variation of the sequence function where we have the same value repeated infinitely:

1
function* repeat(valor) {
2
    while (true) {
3
        yield valor
4
    }
5
}

scan#

We can define a function similar to Array.prototype.reduce, but instead of having a single final value, we can have each step of the intermediate values as an array:

1
function* scan(reducer, valorInicial, iteravel) {
2
  let resultado = valorInicial;
3
  yield resultado;
4
  for (const atual of iteravel) {
5
    resultado = reducer(resultado, atual);
6
    yield resultado;
7
  }
8
}

And many others. The point is that these functions individually might not seem like much, but they allow us to build more complex functions and sequences when combined. And that’s what generators are all about: creating small functions that can serve a greater purpose.

In fact, many of these functions are described in a proposal I’ll still write about here called Iterator Helpers.

Conclusion#

Generators will probably always be rarely used because they have very specific use cases, but it’s always interesting to understand that these tools exist and that you can take advantage of them.

Keep an eye out for upcoming articles about iterators, and if you don’t want to miss anything about the content I release, subscribe to the newsletter here!