I am going to start a series of introductory articles about functional programming. We will address this subject as a kind of notes for newcomers, but also as notes for my future self. The goal won't be to cover exhaustively everything related to functional programming, but to provide some concepts and basic operations on which this paradigm is based.

Functional programming is built on some common operations, as composition and partial application, leading to some quite specialized techniques, as the transducers. Its goal is to minimize the shared states living within our applications as well as producing more declarative code —and therefore more legible— and, finally, allowing modules easier to test.

For this series we are going to use a common language: JavaScript. This is a language that works well with functional programming, although it was not purely conceinved for it as Erlang or Haskell. Because of this we will have to implement some structures and operations that other languages may provide natively,

As we will be covering functional programming in a concise way here, for those looking for a more exhaustive approach I recommend two resources:

• Professor Fisby's Mostly Adequate Guide to Functional Programming, Brian Lonsdorf, 2021: https://mostly-adequate.gitbook.io/mostly-adequate-guide.
• Funcional Light Javascript, Kyle Simpson, 2017: https://github.com/getify/Functional-Light-JS/tree/master.

In this first article we are going to discuss the most basic concepts and operations that we can find in functional programming. We will talk about:

• Immutability
• Purity
• Arity
• Currying
• Partial application
• Point-free style
• Compose and pipe
• Eager and lazy evaluation

We'll go over each of these concepts, presenting easy examples in JavaScript to keep things as straightforward as possible.

Immutability

One of the main challenges we may encounter in programming is shared state.

Let’s say we are working with JavaScript and have an object with a property a, stored in memory in the variable x with a certain value. If we then copy x into a new variable y what we are actually copying is the reference to x in y. From this point forward, any changes made to y will also affect x.

As you can imagine, this can be problematic, as it may lead to unexpected and imperceptible modifications to x.

language-javascript
⁠
const sherlock = { name: "Sherlock" };
⁠const bilbo = sherlock;
⁠bilbo.name = "Bilbo";
⁠⁠console.log(sherlock); // { name: "Bilbo" }

To address this issue we may use the spread operator ... to extract the properties from john when creating a new object bob:

language-javascript

const sherlock = { name: "Sherlock" };
⁠const bilbo = { ...sherlock };
⁠bilbo.name = "Bilbo";
⁠sherlock.name = "Sherlock";
⁠console.log(sherlock); // { name: "Sherlock" }

Keep in mind that the spread operator copies the first-level properties of an object by value, but nested properties are copied by reference with shallow copy. To avoid this issue, we can use the methods provided by the JSON object.

Another example of the problem with shared states could arise when dealing with concurrent operations. In this case, we have an increment operation whose result is stored in a sharedState object. But we also have a decrement operation that calculates a new value, which is also saved in sharedState. The result is a shared state: sharedState refers to both the value produced by increment and the one from decrement. Considering that both operations are impure, we end up with code that is hard to follow and difficult to test.

language-javascript

const sharedState = { type: "shared state", value: 0 };
⁠
⁠const increment = () => {
⁠  sharedState.value += 1;
⁠};
⁠
⁠const decrement = () => {
⁠  sharedState.value -= 1;
⁠};
⁠
⁠increment();
⁠decrement();
⁠decrement();
⁠increment();
⁠decrement();
⁠increment();
⁠
⁠⁠console.log(sharedState); // { type: "shared state", value: 0 }

This example is not exactly concurrent, but it illustrates the challenge of tracking the value of our shared state throughout the application's flow. If we also had different modules consuming and modifying sharedState we might run into issues when trying to determine the state of our application at any given moment during its execution.

But all these issues can be easily solved by removing the shared state and generating a new state with each operation performed:

language-javascript

const initialState = { type: "non-shared state", value: 0 };
⁠
⁠const increment = (state) => ({
⁠  ...state,
⁠  value: state.value + 1,
⁠});
⁠
⁠const decrement = (state) => ({
⁠  ...state,
⁠  value: state.value - 1,
⁠});
⁠
⁠const stateIncremented = increment(initialState);
⁠const initialState2 = decrement(stateIncremented);
⁠const initialDecremented = decrement(initialState2);
⁠const initialState3 = increment(initialDecremented);
⁠const stateDecremented2 = decrement(initialState3);
⁠const initialState4 = increment(stateDecremented2);
⁠
⁠console.log(initialState4); // { type: "non-shared state", value: 0 }

The variable names chosen aren't particularly elegant or expressive, but they were selected to highlight the advantage of this code over the previous one. We have moved the program's state from a mutable variable, whose content is only known at runtime, to an immutable variable, whose content is predictable at compile time. This makes the new version of our program easier to understand, debug, and test. In this sense immutability improves the predictability of our application.

We could even keep a history of all the states of our application, which would make it easier for us to debug and perform post-mortem analysis if any issues arise.

Purity

Functional programming aims for simplicity and predictability in our programs, which is why it relies heavily on functions. And for functions to be predictable they must return the same output for the same input. In this sense, a function represents an immutable relationship between two sets: input and output, and this relationship must remain constant.

language-javascript

⁠const pureFunction = (a, b) => a < b;
⁠console.log(pureFunction(1, 2)); // true
⁠
⁠const impureFunction = (a, b) => (Math.random() > 0.5 ? a < b : a > b);
⁠console.log(impureFunction(1, 2)); // true or false

Purity makes predictability easier, but it also allows us to memoize function results, avoiding unnecessary computations.

Arity

Arity refers to the number of parameters a function takes.

• A function with an arity of 0 doesn’t take any parameters: it’s called nullary.
• A function with an arity of 1 takes just one parameter: it’s unary.
• A function with an arity of 2 takes two parameters: it’s binary.
• A function with an arity of n can take any number of parameters: it’s called n-ary, also known as variadic.

language-javascript

⁠const nullaryFunction = () => "Hello world";
⁠console.log(nullaryFunction()); // hello world
⁠
⁠const unaryFunction = (greeting) => `${greeting} world`;
⁠console.log(unaryFunction("hello")); // hello world
⁠
⁠const binaryFunction = (greeting, subject) => `${greeting} ${subject}`;
⁠console.log(binaryFunction("hello", "world")); // hello world
⁠
⁠const nAryFunction = (...strings) => 
⁠  strings.reduce((acc, curr) => `${acc}${curr} `, "");
⁠console.log(nAryFunction("hello", "world", "and", "pluton")); 
⁠// hello world and pluton

Nothing special here, just some terminology. But it's important, as unary functions are at the core of some operations in functional programming.

Currying

Currying is the process of transforming a function that takes multiple parameters —with an arity greater than 1— into a series of nested unary functions, that is, functions that take only one parameter.

language-javascript

⁠// Non curried function
⁠const sum = (a, b, c) => a + b + c;
⁠const result = sum(1, 2, 3);
⁠console.log(result); // 6
⁠
⁠// Curried function
⁠const sum = (a) => (b) => (c) => a + b + c;
⁠⁠const result = sum(1)(2)(3);
⁠console.log(result); // 6

As we'll see later, this is a key technique in functional programming: there are many operations where we need functions to have a similar structure —one input for one output— allowing them to be combined in different ways.

Partial application

Partial application refers to the process of transforming a function that takes various parameters into another function that only takes some of those parameters. It "fixes" certain arguments within its context and returns a new function that will take the remaining arguments, combining them with the prefixed ones.

This is really useful as it allows us to create functions with parameters that are pre-applied, which means we can specialize functions.

For example, let's say we have a function called createUrl that takes three string parameters: protocol, domain, and path, combining them and returning another string:

language-javascript

⁠const createUrl = (scheme, domain, path) => `${scheme}://${domain}/${path}`;
⁠
⁠const httpUrl = createUrl("http", "example.com", "hello");
⁠console.log(httpUrl); // "http://example.com/hello"
⁠
⁠const httpsUrl = createUrl("https", "example.com", "hello");
⁠console.log(httpsUrl); // "https://example.com/hello"

We may probably use this function in multiple places in our code with "http" or "https". To write more declarative code and avoid passing the scheme parameter every time we need a URL we can create two specialized versions of createUrl: createHttpUrl and createHttpsUrl, applying the scheme parameter in the context of these functions.

language-javascript

⁠const createUrl = (scheme, domain, path) => `${scheme}://${domain}/${path}`;
⁠
⁠// Partial application: fixing `scheme` parameter
⁠const createUrlWithProtocol = (scheme) => 
⁠  (domain, path) => createUrl(scheme, domain, path);
⁠⁠
⁠⁠const createHttpUrl = createUrlWithProtocol("http");
⁠const createHttpsUrl = createUrlWithProtocol("https");
⁠
⁠const httpUrl = createHttpUrl("http", "example.com", "hello");
⁠console.log(httpUrl); // "http://example.com/hello"
⁠
⁠⁠const httpsUrl = createHttpsUrl("https", "example.com", "hello");
⁠console.log(httpsUrl); // "https://example.com/hello"

These new functions are semantically more expressive, and they reduce the arity of the functions we use to create the URLs.

Point-free style

Point-free style is a way of writing functions where they are processed without referencing their parameters —points—.

Let's take a look at an example: when we pass a function as an argument to another function, it’s common practice to write it directly inside that function:

language-javascript

⁠const userName = users.map(
⁠  ({ first, lastName }) => `${first} ${lastName}`
⁠);

In a point-free style we would extract the function declaration and store it in a variable, which is then passed to the new function.

language-javascript

⁠const composeName = ({ first, lastName }) => `${first} ${lastName}`;
⁠const userName = users.map(composeName⁠);

In general point-free style is more declarative and readable, and also makes function composition easier, as we’ll see later.

Compose and pipe

In algebra, function composition refers to the process of transforming a sequential set of unary functions into nested ones, so that the result of one function is passed as a parameter to the next, wrapping it.

Let's say we have two functions

f(x) = 2x\\ ⁠⁠g(x) = x^2
⁠
We express that we compose them with the following syntax:

f \circ g =(g(x))

Which means that we are executing g(x) first, and then f(x) with its result.

It’s important to emphasize that composition is not commutative:

f \circ g \neq g \circ f

although it is associative.:

f(g(h(x))) = (f \circ g)(h(x)) = f(g(h(x))).

This can easily be translated into JavaScript, as functions are first-class citizens in this language: we can treat them like variables, so any function can accept another as a parameter.

Given two functions, we can execute one inside the other:

language-javascript

⁠⁠const f = (x) => x * 2;
⁠const g = (x) => x * x;
⁠
⁠const result = f(g(3));
⁠console.log(result); // 18

And let's say that this type of operation is something we do often: it would be interesting if we had a utility that, given two functions, would return them composed:

⁠language-javascript

⁠const f = (x) => x * 2;
⁠const g = (x) => x * x;
⁠
⁠const compose = (f, g) => (x) => f(g(x));
⁠
⁠const composed = compose(f, g);
⁠const resultComposed = composed(3); // 18

In this case, we have a function that takes two parameters —arity 2—, performs partial application to fix them in the closure context, and returns a new function. This new function, when given a new parameter, will use the fixed ones to execute our nested functions, using the new parameter as the initial argument.

This way, the returned function is stored in the variable composed, which has f and g in its context, and will use them when called with a new argument (3).

Now let's say we want to compose three functions:

f \circ g \circ h = f(g(x))

Using the same logic, we can translate this into JavaScript with a function compose of arity 3:

language-javascript

⁠const f = (x) => x * 2;
⁠⁠const g = (x) => x * x;
⁠const h = (x) => x - 1;
⁠
⁠const compose = (f, g, h) => (x) => f(g(h(x)));
⁠
⁠const composed = compose(f, g, h);
⁠console.log(composed(3)); // 8

It works perfectly, but it's not very scalable. The next step would be to create an n-ary function that, given any number of functions passed as parameters, returns them composed.

language-javascript

⁠const f = (x) => x * 2;
⁠⁠const g = (x) => x * x;
⁠const h = (x) => x - 1;
⁠
const compose = (...fns) => (x) =>
⁠  fns.reduceRight((acc, curr) => curr(acc), x);
⁠
⁠const composed = compose(f, g, h);
⁠console.log(composed(3)); // 8

This function takes any number of functions as individual parameters. It distributes them into an array using the spread operator ..., and sets them within the closure context, returning a new function. This new function will be unary, receiving the initial data as a parameter to process our functions. It will then loop through the array of functions from right-to-left, executing them, which means it will execute the last function first with the initial data provided in the parameter.

What we've just done is create a compose function: a very handy n-ary utility for composing any number of functions as long as they are unary.

Now, let's say we want to compose our functions, but starting from the first one to the last: instead of using Array.reduceRight, we can use Array.reduce.

language-javascript

⁠const f = (x) => x * 2;
⁠⁠const g = (x) => x * x;
⁠const h = (x) => x - 1;
⁠
const pipe = (...fns) => (x) =>
  fns.reduce((acc, curr) => curr(acc), x);
⁠
⁠const piped = pipe(f, g, h);
⁠console.log(piped(3)); // 35

This type of utility is known as a pipe. As we mentioned earlier, composition is not commutative, so it returns a different result compared to the compose function.

Eager and lazy evaluation

By "eager evaluation," we refer to the computation model where a variable is evaluated immediately after the expression appears in the code.

For example, in JavaScript:

language-javascript

⁠⁠const sum = (a, b) => a + b;
const result = sum(1, 2); // Computed here
⁠console.log(result); // 3

As we can see, the expression is evaluated when we call the function, not when the result of the function is consumed.

With lazy evaluation, on the other hand, computation is delayed until the result is actually used.

Except for the logical operators &&, || and ternary operator, JavaScript does not support lazy evaluation. This is a problem, since our code will always consume resources for the total number of operations reflected in the code.

However, lazy evaluation can be implemented in various ways. For example, using the mentioned logical operators —though in a very limited way— or by using generators:

language-javascript

⁠function* lazySum(a, b) {
⁠  yield a + b;
⁠}
⁠
⁠let result = lazySum(1, 3); // computation delayed
⁠console.log(result); // not evaluated yet
⁠console.log(result.next().value); // 4

Here, the sum won't be executed when we call lazySum, but when we use the result and call next(). Later on, we'll explore operations in functional programming that are executed lazily.

Conclusion

That's all for today. All of these concepts and operations are quite common, but they are the foundation for the operations we'll cover soon. The code can be run in the TypeScript Playground, and is available at this repository.

If you have any question or if you found a mistake please ping me to the address at the footer.

In our previous article we covered some of the basics of functional programming. We discussed concepts such as immutability, purity, and arity, as well as operations like partial application, currying, and composition, all of them fundamental components of this paradigm.

In this article we will build on these basic operations to create some functions that will help us to process large datasets. But first, let’s briefly review composition.

Function composition

When we described function composition, we started with a common definition provided in algebra:

... process of transforming a sequential set of unary functions into nested ones, so that the result of one function is passed as a parameter to the next, wrapping it.

We achieved this by using a function called compose, which executes right-to-left:

language-javascript
⁠
⁠⁠const f = (x) => x * 2;
⁠const g = (x) => x * x;
⁠const h = (x) => x + 2;
⁠
const compose = (...fns) => (x) =>
⁠  fns.reduceRight((acc, curr) => curr(acc), x);
⁠
⁠const composed = compose(f, g, h);
⁠const result = composed(3);
⁠console.log(result); // 50

Or a pipe, executed left-to-right:

language-javascript
⁠
⁠⁠const f = (x) => x * 2;
⁠const g = (x) => x * x;
⁠const h = (x) => x + 2;
⁠
const pipe = (...fns) => (x) =>
⁠  fns.reduce((acc, curr) => curr(acc), x);
⁠
⁠const piped = pipe(f, g, h);
⁠const result = piped(3);
⁠console.log(result); // 38

We also mentioned that composition is not commutative, f \circ g \neq g \circ f:

⁠language-javascript
⁠
⁠const composedA = compose(f, g);
⁠const composedB = compose(g, f);
⁠
⁠const isCommutative = composedA(3) === composedB(3);
⁠
⁠console.log(isCommutative); // false

Although it is associative, f \circ (g \circ h) = (f \circ g) \circ h:

language-javascript
⁠
⁠const composedA = compose(f, compose(g, h));
⁠const composedB = compose(compose(f, g), h);
⁠
⁠const isAssociative = composedA(3) === composedB(3);
⁠
⁠console.log(isAssociative); // true

Nothing new, this was all covered in our previous article. However it’s worth reviewing, as it is essential for what we are about to discuss next.

Working with iterables

So far we've seen how we can compose functions to work with primitives or objects; but we have skipped the case of iterables, such as arrays.

To handle them effectively we'll need to rely on some utilities, which we will implement next.

Reducers

The first function that will help us work with iterables is known as a "reducer": binary functions — with an arity of two — that combine two values into a single result.

A simple reducer has the following form:

⁠language-javascript
⁠
⁠const accumulate = (a, b) => a + b;
⁠console.log(accumulate(1, 2)); // 3
⁠
⁠const concat = (array, val) => [...array, val];
⁠console.log(concat([1], 2)); // [1, 2]
⁠
⁠const power = (base, exponent) => base ** exponent;
⁠console.log(power(2, 4)); // 16

Although it may also merge two objects:

language-javascript
⁠
⁠const merge = (a, b) => ({ ...a, ...b });
⁠console.log(merge({ a: 1 }, { b: 2 })); // { a: 1, b: 2 }

Or it may contain slightly more elaborate logic: for example, a reducer that takes a list and a value, which is added to each element of that list:

language-javascript
⁠
⁠const sumEach = (iterable, val) => {
⁠  const result = [];
⁠
⁠  for (const item of iterable) {
⁠    result.push(item + val);
⁠  }
⁠
⁠  return result;
⁠};
⁠
⁠const result = sumEach([1, 2], 3);
⁠console.log(result); // [4, 5]

This last function is interesting, but it has a problem: it does two intertwined things. On one hand, it "reduces" two values into one — two integers into a single value, an array and an integer into another array, two objects into one, etc—. On the other hand, it contains some logic that is executed on the values to be reduced —adding a value to each item in the array—.

It would be perfect if we could decouple both tasks.

We can start by extracting the summing part, which we would pass as a callback, and that in turn would be a reducer. This way we can execute it with the iterable to be transformed, initialized with an initial value initialValue. And since our sumEach function will no longer be responsible for the summing, we can give it a name that aligns with what it does: reduce.

language-javascript
⁠
⁠const reduce = (reducer, iterable, initialValue) => {
⁠  let accumulation = initialValue;
⁠
⁠  for (const item of iterable) {
⁠    accumulation = reducer(accumulation, item);
⁠  }
⁠
⁠  return accumulation;
⁠};
⁠
⁠const data = [1, 2, 3];
⁠const increaseOne = (acc, val) => acc.concat([val + 1]);
⁠const result = reduce(increaseOne, data, []);
⁠console.log(result); // [2, 3, 4]

Here we have already taken a step forward: we have managed to separate the summing operation from the reduction operation, and store it in increaseOne. And if this reduce function sounds familiar, it's because it is practically identical to JavaScript's Array.reduce method.

language-javascript
⁠
⁠const data = [1, 2, 3];
⁠const increaseOne = (acc, val) => acc.concat([val + 1]);
⁠const result = data.reduce(increaseOne, []);
⁠console.log(result); // [2, 3, 4]

If we now recall our compose function, we’ll see that it is using Array.reduceRight().

language-javascript
⁠
⁠const compose = (...fns) => (initialData) => 
⁠  fns.reduceRight((acc, curr) => curr(acc), initialData);

Lets see what happens by replacing Array.reduceRight() with our reduce:

language-javascript
⁠
⁠const compose = (...fns) => (initialData) => 
⁠  reduce((acc, curr) => curr(acc), fns.reverse(), initialData);
⁠
⁠const functionA = (value: number) => value + 1;
⁠const functionB = (value: number) => value * 2;
⁠const functionC = (value: number) => value + 3;
⁠
⁠const composed = compose(functionA, functionB, functionC);
⁠const result = composed(1);
⁠
⁠console.log(result); // 9

It works perfectly. This makes sense, as the input and output types are almost the same. It’s worth noting that we’ve reversed the array holding the functions to be composed with Array.reverse(), since compose must execute them right-to-left.

To implement pipe, we would proceed in the same way, but without reversing the array, as it should execute the functions left-to-right.

language-javascript
⁠
⁠const pipe = (...fns) => (initialData) => 
⁠  reduce((acc, curr) => curr(acc), fns, initialData);
⁠
⁠const functionA = (value: number) => value + 1;
⁠const functionB = (value: number) => value * 2;
⁠const functionC = (value: number) => value + 3;
⁠
⁠const piped = pipe(functionA, functionB, functionC);
⁠const result = piped(data);
⁠
⁠console.log(result); // 7

Fantastic, we have now implemented our own Array.reduce(), which is agnostic to the reducer passed to it, and we have used it in our own compose and pipe functions.

Transformers

Now that we have our reduce function, we can take another step forward and explore what "transformers" are and what they are for.

One of the most common operations that may be performed on iterables is converting each of its elements from one type A to another type B.

To achieve this we can create a map function that will receive a transformation function, which we will call mapper, and whici will be responsible for performing the operation to transform each element of the iterable. This map will create another function that will accept a reducer, which in turn will handle the operation of accumulating the elements of the iterable into a given data structure—another array, an object, an integer, etc.—independently of the transformation performed on each element of the iterable. This second function will return another reducer, which will apply both operations —the transformation (the mapper) and the accumulation (the reducer)— to the iterable passed to it:

language-javascript
⁠
const map = (mapper) => (reducer) => (acc, value) =>
⁠  reducer(acc, mapper(value));

It sounds complicated, but let’s stick with the idea that our map, when called with a mapper, creates a function with a particular trait: it receives a reducer and returns another reducer.

Let’s look at the following example: given an array of integers, we can use the map function by passing it a function that will convert the integer it receives into a string. Then we call this function again, passing the accumulator to be used —returning a new array with concat— and finally pass this last function to reduce for it to be utilized.

language-javascript
⁠
⁠const map = (mapper) => (reducer) => (acc, value) =>
⁠  reducer(acc, mapper(value));
⁠
⁠const data = [1, 2, 3];
const integerToString = map((x) => x.toString());
⁠const integerToStringWithReducer = integerToString(concat);
⁠const result = reduce(integerToStringWithReducer, data, []);
⁠
⁠console.log(result); // ["1", "2", "3"]

And in the same way we created a map function that transforms values, we could create a similar function to select elements from an array.

For example, instead of our map, we can create a filter function that takes another function as an argument, which we’ll call predicate, and evaluates whether a given element of the array should remain in the array or not. And just like in map, this second function will receive an accumulator responsible for the reduce operation to be performed and return another reducer. The difference is that in the case of filter, this last reducer will apply the predicate to the value in question, and if the result is true, it will add it to the array; otherwise it will discard it.

language-javascript
⁠
const filter = (predicate) => (reducer) => (acc, val) => 
⁠  predicate(val) ? reducer(acc, val) : acc;

We could use this filter function in a very similar way to how we used map.

language-javascript
⁠
const filter = (predicate) => (reducer) => (acc, val) =>
⁠  predicate(val) ? reducer(acc, val) : acc;
⁠
⁠const data = [1, 2, 3];
⁠const isOdd = filter((x) => x % 2 !== 0);
⁠const isOddWithReducer = isOdd(concat);
⁠const result = reduce(isOddWithReducer, data, []);
⁠
⁠console.log(result); // [1, 3]

Just like with reduce, we can see that filter and map have their native counterparts in Array.map and Array.filter, producing similar results. However, in this case, our functions have a special trait: they can be composed.

For example, we may create two functions that first select the odd values and then add one to each of the resulting ones using pipe. ⁠

⁠language-javascript
⁠
const data = [1, 2, 3, 4, 5];
⁠
⁠const isOdd = filter((x: number) => x % 2 !== 0);
⁠const incrementValue = map((x: number) => x + 1);
⁠
const incrementOddValues = pipe(incrementValue, isOdd);
⁠const incrementValueWithReducer = incrementOddValues(concat);
⁠const result = reduce(incrementValueWithReducer, data, []);
⁠
⁠console.log(result); // [2, 4, 6]

And just like before, composition is not commutative, so the order of the functions matters: if instead of using pipe — left-to-right — we use compose — right-to-left — we will see a different result.

language-javascript
⁠
const data = [1, 2, 3, 4, 5];
⁠
⁠const isOdd = filter((x: number) => x % 2 !== 0);
⁠const incrementValue = map((x: number) => x + 1);
⁠
const incrementOddValues = compose(incrementValue, isOdd);
⁠const incrementValueWithReducer = incrementOddValues(concat);
⁠const result = reduce(incrementValueWithReducer, data, []);
⁠
⁠console.log(result); // [3, 5]

Conclusion

Let’s recap: we have seen what reducers are, what transformers are, and two functions related to them: mappers and filters.

• Reducer: a pure function that accepts two values and reduces them into a single value. They cannot be composed, as they return a single value while accepting two.
• Transformer: function responsible for transforming data in iterables. We use them to create other types of functions that can be easily composed.A function responsible for transforming data in iterables. We use them to create other types of functions that can be easily composed.
• Mapper: function that accepts another function as a parameter, which converts data from type A to type B using the provided function.
• Filter: function that accepts a predicate function —which returns a boolean —and creates another function that will use that predicate to determine if an element from the array should be kept in the returned value or not.

It’s important to have a clear understanding of these concepts, as they form the foundation for what we will cover next: transducers.

This is all for today. As always, the code can be run in the TypeScript Playground, and it’s also available for download at git.antoniodiaz.me.

If you have any questions or have found an error, feel free to reach out to me at the address shown in the footer.

This is our third article about functional programming in JavaScript. We have already covered its basics and digged into the transformers.  Now its the time to address transducers.

To do this, we will need to rely on the composition of transformers we saw earlier.

Transformers composition

The map and filter functions we saw in the previous article have a difference compared to those provided by Array.map and Array.filter: they create functions that receive a reducer and return another reducer.

language-javascript
⁠
⁠⁠⁠// map = (fn) => (reducer) => reducer
⁠
⁠const map = (mapper) => (reducer) => (acc, val) => 
⁠  reducer(acc, mapper(val));
⁠
const filter = (predicate) => (reducer) => (acc, val) =>
⁠  predicate(val) ? reducer(acc, val) : acc;

Which means that our transformers are composable.

language-javascript
⁠
⁠(reducer) => reducer
            ⁠(reducer) => reducer
            ⁠            ⁠(reducer) => reducer ⁠...

Let's try with an example: we might need to select those elements of an array that are less than five, and then calculate their squares. We can create a filter function for the selection and a mapper function for the power calculation, and use them with our reduce function.

language-javascript
⁠⁠
⁠const data = [1, 2, 3, 4, 5];
⁠const lessThanFive = filter((x) => x < 5);
⁠const squares = map((x: number) => power(x, 2));
⁠
⁠const filteredAndSquared = reduce(
⁠  lessThanFive(squares(arrayConcat)),
⁠  data, []
⁠⁠);
⁠console.log(filteredAndSquared); // [1, 4, 9, 16]

And again, let's remember we said that although composition is associative, it is not commutative: the order in which the functions are composed matters.

f \circ g \neq g \circ f

We could take advantage of this and, if necessary, reverse the positions of lessThanFive and squares to first calculate the squares, and then get those that are less than five.

language-javascript
⁠
⁠⁠⁠const data = [1, 2, 3, 4, 5];
⁠⁠const squaredAndFiltered = reduce(
  squares(lessThanFive(arrayConcat)),
⁠  data, []
⁠);
⁠console.log(squaredAndFiltered); // [1, 4]

Or we might need to filter those elements whose squares are less than five:

language-javascript
⁠
⁠⁠const data = [1, 2, 3, 4, 5];
⁠const isSquareLessThanFive = filter((x) => power(x, 2) < 5);
⁠const filtered = reduce(
⁠  isSquareLessThanFive(arrayConcat), 
⁠  data, ⁠[]
⁠);
⁠console.log(filtered); // [1, 2]

Or we could even square each element and reduce it to a single value.

language-javascript
⁠
⁠const accumulate = (a: number, b: number): number => a + b;
⁠
⁠⁠⁠const data = [1, 2, 3, 4, 5];
const filteredSquaredAndAccumulated = reduce(
⁠  lessThanFive(squares(accumulate)),
⁠  data,
⁠  0
⁠);
⁠console.log(filteredSquaredAndAccumulated); // 30

And since our transformers can be composed, we can use our compose function for all these operations:

⁠⁠language-javascript
⁠
⁠const compose = (...fns) => (items) =>
  fns.reduceRight((acc, curr) => curr(acc), items);
⁠
⁠⁠⁠const data = [1, 2, 3, 4, 5];
⁠const filterAndSquare = compose(lessThanFive, squares);
⁠
⁠const filteredSquaredAndAccumulated = reduce(
⁠  filterAndSquare(accumulate), data, 0
⁠);
⁠
⁠console.log(filteredSquaredAndAccumulated); // 30

With this, we have all the pieces needed for the operation we are focusing on in this article: transduction.

Transducers

As we mentioned, our filter and map functions have their analogs in Array.filter and Array.map. One might then ask, what is the point of implementing our own functions when the Array object already has these methods.

Well, let’s consider the following scenario: suppose we have a large array —ten million elements— on which we need to perform various operations.

language-javascript
⁠
⁠const data = Array.from({ length: 10000000 }, (_, index) => index + 1);

To process it with Array.map and Array.filter, we need to first store the entire array in memory. And since both methods return a new array, each time we call one of them we are creating a new one.

For an array with just a few hundred elements this is not much of an issue. But it is not a scalable pattern because when we deal with a large list we could be consuming a significant amount of resources.

Additionally we could face another problem: since we're storing the processed data in memory, if the application encounters a failure and throws during the processing we may lose the data already processed.

This all happens because Array methods execute sequentially over the entire array: that is, if we want to apply Array.map and Array.filter over an array, map will first traverse the entire array, transforming all its elements and returning a new array, and then filter will do the same, selecting the elements we are interested in.

language-javascript
⁠
const data = ["1", "2", "3"];
⁠
⁠const result = data
⁠  .map((x) => (console.log("map"), Number(x)))
⁠  .filter((x) => (console.log("filter"), x % 2 !== 0));
⁠console.log(result); // [1, 3]
⁠
⁠// map
⁠⁠// map
⁠⁠⁠// map
⁠⁠⁠// filter
⁠⁠⁠⁠// filter
⁠⁠⁠⁠// filter
// [1, 3]

It would be fantastic if we could decouple the processing of each individual element from the processing of the entire array.

To achieve this we could start with a change: instead of making successive passes over the entire array, we could try performing all the necessary actions on the first element, then the second, and so on, until we reach the last element.

Well, we do not need to implement anything to achieve this, because this is precisely what our map and filter functions are already doing with reduce.

language-javascript
⁠
⁠const data = ["1", "2", "3"];
⁠
⁠const toInt = map((x: string) => (console.log("map"), Number(x)));
⁠const isOdd = filter((x: number) => (console.log("filter"), x % 2 !== 0));
⁠const toIntAndFilterOdds = compose(toInt, isOdd);
⁠const result = reduce(toIntAndFilterOdds(concat), data, []);
⁠
⁠console.log(result); 
⁠
⁠⁠⁠// map
⁠⁠⁠// filter
⁠⁠⁠⁠// map
⁠⁠⁠// filter
⁠⁠// map
⁠⁠⁠// filter
⁠// [1, 3]

Although it seems like a small difference, it is not, as it opens the door to transform reduce into a function that, instead of receiving a transformer with a reducer already applied, receives them separately and applies them within itself.

In this way, instead of reducing with reduce:

language-javascript
⁠
⁠⁠const reduce = (reducer, iterable, initialValue) => {
⁠  let acc = initialValue;
⁠
⁠  for (const item of iterable) {
⁠    acc = reducer(acc, item);
⁠  }
⁠
⁠  return acc;
⁠};

We would use transduce, which would look like this:

language-javascript

⁠const transduce = (
⁠  transformer, 
⁠  reducer, 
  ⁠iterable,
⁠  initialValue
⁠) => {
⁠⁠  let acc = initialValue;
⁠  const transformedReducer = transformer(reducer);
⁠
⁠  for (const item of iterable) {
⁠    acc = transformedReducer(acc, item);
⁠  }
⁠
⁠  return acc;
⁠};
⁠
⁠const data = ["1", "2", "3"];
⁠
⁠const toInt = map((x: string) => (console.log("map"), Number(x)));
⁠const isOdd = filter((x: number) => (console.log("filter"), x % 2 !== 0));
⁠const toIntAndFilterOdds = compose(toInt, isOdd);
const result = transduce(toIntAndFilterOdds, concat, data, []);
⁠
⁠console.log(result);
⁠⁠
⁠⁠⁠// map
⁠⁠⁠// filter
⁠⁠⁠⁠// map
⁠⁠⁠// filter
⁠⁠// map
⁠⁠⁠// filter
⁠// [1, 3]

This is a transducer: a function that takes a transformer —which can be a series of composed transformers— and a reducer —responsible for the reduction operation— along with an iterable and an initial value, and will reduce the iterable over the initial value using the transformer and the reducer.

Transducers have a significant efficiency advantage over method chaining, as seen with the Array methods. This becomes clear if we revisit the large array scenario we discussed earlier.

Let’s say we have a list of ten million products with different prices that we sell remotely, with a shipping cost of ten dollars. Buyers can finance the payment in twelve monthly installments, but only if each resulting monthly installment is an integer —without decimals—. With this, we have been asked to find the highest spending a customer can make when buying a financeable product.

We can do this with Array methods:

language-javascript
⁠
⁠// Array with 10.000.000 elements
⁠const data = [
⁠  { name: "Product 1", price: 73 },
⁠  ...
⁠];
⁠
⁠const addShippingCosts = (product: Product) => product.price + 10;
⁠const isEligibleForFinancing = (price: number) => price % 12 === 0;
⁠const getBiggerExpense = (a: number, b: number) => (a > b ? a : b);
⁠
⁠console.time("array_methods");
⁠const result = data
⁠  .map(addShippingCosts)
⁠  .filter(isEligibleForFinancing)
⁠  .reduce(getBiggerExpense);
⁠
console.log(result); // 120
⁠
⁠console.timeEnd("array_methods"); // 141

Great, we get the correct result. Let’s take note of how long it takes to execute this code: at the time of writing, the TypeScript Playground shows 141 ms, though it might vary depending on the machine it’s executed on.

Now, let’s try using a transducer:

language-javascript
⁠
⁠// Array with 10.000.000 elements
⁠const data = [
⁠  { name: "Product 1", price: 73 },
⁠  ...
⁠];
⁠
⁠const eligibleForFinancing = filter(isEligibleForFinancing);
⁠const priceWithShippingCosts = map(addShippingCosts);
⁠const piped = pipe(eligibleForFinancing, priceWithShippingCosts);
⁠
⁠⁠console.time("transduce");
⁠const result = transduce(piped, getBiggerExpense, data, 0);
console.log(result); // 120
⁠console.timeEnd("transduce"); // 62

As we can see the same result is obtained, but with a notable efficiency gain. This, of course, will depend on the time and space complexity of the algorithms used: but it’s clear that, with the same algorithms, transducers are more efficient than method chaining.

However, once we call transduce, it will attempt to process the entire array by storing it in memory, which could potentially overload the system. In other words, the current transducer is eagerly evaluated.

What we actually want is a function that allows us to process the data with lazy evaluation, so that the data is processed only on demand. And if we recall our first article, we mentioned that lazy evaluation in JavaScript can be implemented with generators. Let’s look at an example of a generator:

language-javascript

⁠function* lazySum(a, b) {
⁠  yield a + b;
⁠}
⁠
⁠let result = lazySum(1, 3);
⁠console.log(result); //  {} (not evaluated yet)
⁠console.log(result.next().value); // 4

So, let's try to implement our transduce function in a way that lazily evaluates the data:

language-javascript
⁠
function* lazyTransduce(
⁠  transformer,
⁠  reducer,
⁠  iterable,
⁠  initialValue
⁠) {
⁠  const transformedReducer = transformer(reducer);
⁠  let acc = initialValue;
⁠
⁠  for (const item of iterable) {
⁠    acc = transformedReducer(acc, item);
⁠
⁠    yield acc;
⁠  }
⁠}

Instead of returning, lazyTransduce will return a generator that will yield its state in acc each time its next() method is executed.

language-javascript
⁠
⁠// Array with 10.000.000 elements
⁠const data = [
⁠  { name: "Product 1", price: 73 },
⁠  ...
⁠];
⁠
⁠const eligibleForFinancing = filter(isEligibleForFinancing);
⁠const priceWithShippingCosts = map(addShippingCosts);
⁠const piped = pipe(eligibleForFinancing, priceWithShippingCosts);
⁠
⁠const result = lazyTransduce(piped, getBiggerExpense, data, 0);
⁠
⁠console.log("result.next().value: ", result.next().value); // 0
⁠console.log("result.next().value: ", result.next().value); // 0
⁠console.log("result.next().value: ", result.next().value); // 0
⁠⁠console.log("result.next().value: ", result.next().value); // 120
⁠⁠⁠console.log("result.next().value: ", result.next().value); // 120
⁠⁠⁠console.log("result.next().value: ", result.next().value); // 120
⁠...

Finally, this is what is properly referred to as a transducer. Here, lazy evaluation allows us to process the data on demand and perform actions on it while maintaining full control over the pace and frequency at which we do so.

For example, we might want to persist the data every certain number of elements, or send notifications to external services about the data found. And all of this can be done without causing system overload, since the spatial complexity is negligible and only the memory required for each iteration step will be used.

And just like when we used our reduce function, we can pass different reducers to our transducers to accumulate data in various ways. For example, by concatenating the data into a new array using concat:

language-javascript
⁠
⁠const data = [
⁠  { name: "Product 1", price: 74 },
⁠  { name: "Product 2", price: 93 },
⁠  { name: "Product 3", price: 110 },
⁠  { name: "Product 5", price: 82 },
⁠];
⁠
⁠const eligibleForFinancing = filter(isEligibleForFinancing);
⁠const priceWithShippingCosts = map(addShippingCosts);
⁠const piped = pipe(eligibleForFinancing, priceWithShippingCosts);
⁠
const result = lazyTransduce(piped, concat, data, []);
⁠
⁠console.log(result.next().value); // [84]
⁠console.log(result.next().value); // [84]
⁠console.log(result.next().value); // [84, 20]
⁠console.log(result.next().value); // [84, 120]
⁠console.log(result.next().done); // true

There is much to say about transducers, especially because there are numerous implementations: they are behind methods offered by various libraries, like pipe in RxJS, or are directly supported, as in the case of Ramda. The same goes for them as with other utilities: it is not always the best decission to implement our own ones due to the work and risk involved in developing the most efficient implementation possible. However, it’s important to be aware of their existence and their fundamentals, as they clearly show their potential.

As usual, the code can be run in the TypeScript Playground and is available for download at git.antoniodiaz.me.

If you have any questions or have found an error, feel free to contact me at the address listed in the footer.

Background

Rust has been gaining popularity among developers, being the most loved language in stackoverflow survey for at least the past five years; but other indexes, such as Tiobe, rank it below the 20 most used languages. It is also difficult to find developers currently working with this language, or companies using it for active development.

What does all this mean? While Rust is recognized as a valuable tool by the developer community, its application is taking time due to its specialization and particularities. Why then invest in such a language? How does the future of Rust look? To answer these questions we will have a look at the language, its syntax, its basic features and how Rust code can be written for web applications. 

Characteristics

Rust originated within the Mozilla team in 2010 as a side project. Since then it has evolved into a community-driven language, with the foundation.rust-lang.org as its main governance institution.

It has some characteristics that, altogether, make it different from many other languages:

• Is a compiled language.
• Is strongly typed with both nominative and structural typing.
• Lacks a garbage collection system, using a «borrow checker» instead.
• Provides pointers/references for stack and advanced pointers for heap memory management.
• Allows both object oriented and functional programming.
• It is conceived specially for concurrent code development.
• Is agnostic to the asynchronous runtime.
• It has a test suite integrated in the language.

Let's go through them to find out what they offer:

Being a compiled language, Rust will provide a lot of information about the memory usage of our program before runtime. Making use of strongly typed structures Rust guarantees that, once the program compiles, it is memory safe. Of course we as developers can make mistakes in many other ways, especially regarding business logic; but the compiler ensures that structurally our program will be as performant and coherent as possible. It is important to mention that it uses nominative typing for structs and structural for tuples: that means that two instances of different structs will have different types to the compiler even when these structs have indentical shapes. With tuples happens the opposite: two tuples with similar shapes have the same type when they have similar shape.

This happens mainly because of the lack of any garbage collection system. Languages such as Java, Python, or Javascript rely on a garbage collector to handle the memory usage and clean up unused variables. In other languages, as in C or C++, it is required to manually handle memory resources; but in Rust we have a «borrow checker» system at compilation time that ensures that, when a variable goes out of the scope of its usage, it is cleaned.

This is done automatically for values with known sizes, which are stored in the stack. For these values we can make use of C-like pointers (*&) to reference and dereference them, although within some constraints imposed by Rust to avoid unsafe memory handling.

For those values whose size is not known beforehand by the compiler and have to be stored in the heap, such an array —«Vector»—, Rust provides specific structures to wrap them (Box, Rc, Arc, etc.), ensuring that they will be cleaned as soon as possible.

One of the main features of Rust is a comeback to a simplified version of C structs instead of C++/Java-like classes. There are three main keywords related to structs in Rust: struct, impl and trait. We can define a new struct using the struct keyword with the values that it is holding; after defining it, we can add methods with the impl keyword. In case we want to describe a common behaviour for several implementations, we can do it with the trait keyword, similar to what in other languages is called an interface:

language-rust
⁠
⁠// We can define a struct for Person
⁠// with a name that will be a String
⁠struct Person {
  name: String,
⁠}

⁠// We can also define a similar struct, but for a Cat
⁠// also with a name that will be a string
⁠struct Cat {
  name: String,
⁠}
⁠
⁠// Each living being can perform certain actions
⁠trait LivingBeingActions {
  fn introduce(&self);
⁠}
⁠
⁠// The person can introduce himself verbally
⁠impl LivingBeingActions for Person {
  fn introduce(&self) {
    println!("- {}: Hi! It's a pleasure :)", &self.name);
  }
⁠}
⁠
⁠// The can can also introduce himself, except… in cat's style
⁠impl LivingBeingActions for Cat {
  fn introduce(&self) {
    println!("- {}: Mrrrrr… Miaaou", &self.name);
  }
⁠}
⁠
⁠// We can instantiate the Person and the Cat,
⁠// and then they can introduce themselves
⁠fn main() {
  let john_doe = Person {
    name: String::from("John Doe"),
  };
  let grumpy = Cat {
    name: String::from("Grumpy"),
  };
  john_doe.introduce(); // - John Doe: Hi!, a pleasure :)
  grumpy.introduce(); // - Grumpy: Mrrrrr... Miaaou
⁠}

This use of structs instead of classes will prevent nested structures, favouring composition instead. Rust will allow us to write code in an object-oriented manner as well as in a functional style, but within a limited frame in both cases. It is perfectly possible to use object-oriented patterns as dependency injection or features as polymorphism, as well as write functional stateless piped code with closures. Both approaches fit naturally, although with some constraints that force the developer to write atomic and readable structures. The resulting code has a somewhat complex syntax, but flat and simple structures which, at the end, make the code easier to read and understand.

Rust comes with a standard library that provides several functionalities. Among them are the threading tools, which will allow us to write concurrent programs. This, with the help of the compiler, static typing, borrow checker and the advanced pointers will ensure that our multithreaded program is memory safe even in multi-threaded programs.

An interesting feature of Rust is that it is not coupled with an asynchronous runtime. For this, we will need to bring an external library: a very common option is the Tokio library, that can be found in the Rust official packages repository, although there are other options available.

The tests are written with the embedded tests suite. It is possible to write both unit tests for each module as well as integration tests for the whole application. Its features are limited but clear following the philosophy of the language.

Working with Rust coming from Java-like languages is somewhat confusing due to the considerable paradigm shift; of course, once this difficulty is overcome the language writes naturally. It has similarities to common used languages such as C++ and C, but with a cleaner footprint. Given the tools it provides for memory management and generics, it could easily be described as «declarative C».

A basic Rust example

To demonstrate how a program can be structured in Rust we can create a naive and simple application, just to visualize the main language features.

Following our previous example, we can develop a program that will ask for a users list from, lets say, a PostgreSQL database. The catch is that we want the list of users in reverse order. 

So we have some different concerns here: 

• The User itself, which will be a plain model. 
• The business logic, which is reversing the order of the retrieved list: this will be a use case. This use case will make use of any retrieval system that complies with a trait.
• Finally we will need the system to retrieve data from the PostgreSQL database, which will be a struct. To avoid coupling the use case with the PostgreSQL repository we can create a trait that all repositories have to comply with. The use case will be able to receive any repository instance that complies with this trait.

Now that we have an idea of our application, let's implement it. We will omit a few lines of code needed for the program to compile, but the whole example can be seen in the Rust Playground.

First we can create our entity, an User with just an id field:

language-rust
⁠
⁠pub struct User {
  pub id: i8,
⁠}

Now that we have our main model, the next question is what we want to do with it. In other words: the use case to retrieve a reverted list of users. We know that it will need to hold the repository instance, but we don’t know the type: this is a task for a generic type.

language-rust
⁠
⁠struct GetUsersRevertedUseCase<T> {
  repository: T,
⁠}

Our use case is gathering the type of the repository instance when it is constructed. Let's create the new method to construct it, for which we can use the impl keyword:

language-rust
⁠
⁠impl<T: Repository> GetUsersRevertedUseCase<T> {
  fn new(repository: T) -> GetUsersRevertedUseCase<T> {
    GetUsersRevertedUseCase { repository }
  }
⁠}

⁠One thing to note here is the <T: Repository> syntax. We are telling the compiler that T will be any struct implementing the Repository trait, returning error if any other type of instance is passed. Also our new method returns an instance of itself.

We can think now about the different repositories that this use case may depend on. We know that it will be an asynchronous action, and that it will return a list of users. Lets implement this in a trait:

language-rust
⁠
⁠trait Repository {
  async fn get_users(&self) -> Result<Vec<User>, String>;
⁠}

We have new things here. First, we have the async keyword, which makes our function return a Result type. This Result type is one of the particularities of Rust: it represents the result of an action that can be successful —Ok(T)— or error —Err(K)—. In our case, if the method is successful, it will return Ok<Vec<User>>, which is an array —a vector in Rust language— of User items; otherwise, if it errors, it will return Err(String). For the sake of brevity we won’t return any error, but the Result type forces us to take that possibility into account, as we are writing asynchronous code.

Now we can think about how we will trigger our use case, so let's implement an execute method for it.

language-rust
⁠
⁠impl<T: Repository> GetUsersRevertedUseCase<T> {
  […]
  async fn execute(&self) -> Result<Vec<User>, String> {
    let users = self.repository.get_users().await.unwrap();
    let reverted_users: Vec<User> = users.into_iter().rev().collect();
    Ok(reverted_users)
  }
⁠}

We see the async syntax, this time awaiting the function of the repo we are calling. We also see the Result type again. Everything is familiar. 

As the repository is returning a Result —Ok<T> or Err<K>— type, we need to extract the value —the array of users— to operate with it. We do this with the unwrap() function, extracting the value if the operation is successful, or panicking with the error if it's not.

We also have some methods operating over the users vector to reverse it: into_iter, which turns a vector into an iterable list of items; rev, which reverses this iterable list, and collect, which returns back the iterable list into a vector. These utility methods belong to Vec struct, and are embedded into the Rust to manipulate vectors in a functional manner.

Ok, now we have a use case and an interface for any repository we may pass to it, so we can proceed to create an actual repository for PostgreSQL:

language-rust
⁠
⁠impl Repository for PostgreSQLRepository {
  async fn get_users(&self) -> Result<Vec<User>, String> {
⁠    let duration_secs = Duration::from_secs(2);
⁠    sleep(duration_secs);
⁠    let user_1 = User { id: 1 };
⁠    let user_2 = User { id: 2 };
⁠    Ok(vec![user_1, user_2])
⁠  }
⁠}

We are going to mock the asynchronous behaviour, so we are using two methods of the standard library: sleep and Duration. Duration will return a special time type that can be used by other methods, such as sleep; and sleep will simply halt execution for the given time. We are just halting our function for two seconds, then instantiating two users, and finally returning a Vector holding them.

We now have all needed components for our program: an User entity, the GetUsersRevertedUseCase, the interface Repository and the actual PostgreSQLRepository that implements it. Lets create an entry point and make use of all these elements:

language-rust
⁠
⁠#[tokio::main]
⁠async fn main() {
⁠  let repository = PostgreSQLRepository;
⁠  let use_case = GetUsersRevertedUseCase::new(repository);
⁠  let users = use_case.execute().await.unwrap();
⁠  println!("{:#?}", users);
⁠}

As we are running asynchronous code we will need an asynchronous runtime: thus, we can call the macro #[tokio:main]. Then we just create an instance of the repository, we construct our use case injecting the repository, and execute it awaiting. The result will be a vector of users in reverse order:

language-rust
⁠
⁠[
⁠  User {
⁠    id: 2,
⁠  },
⁠  User {
⁠    id: 1,
⁠  },
⁠]

As mentioned above this code is available in the Rust Playground, where we can compile and run it.

Conclusion

Rust is not a straightforward language. Study and some patience is required to understand both the basic syntax and the underlying features that it provides; it is also young, and its ecosystem has to evolve. But it is promising.

It is in performance demanding tasks where the language fits perfectly. For general web development, where rapid prototyping with agile workflows is a requirement, interpreted languages such as Python or TypeScript may be more suitable; but in cases where memory leaks may cause spikes and performance issues in server usage, Rust is a much better option.

The current acceptance of Rust within the industry is very limited, but the Rust foundation is supported by Google, Microsoft, AWS, Huawei and Mozilla: there is a need for such a performant language, and we will probably see how it is introduced in the following years. There are difficulties though: the funding companies will try to influence Rust to make it evolve fitting their particular needs, and there are strong disagreements among the internal Rust development teams. But the language fits so well into the aforementioned tasks that it will be worthwhile to follow its evolution and integration into the web industry.

In conclusion: it is possible and desirable to use Rust in specific contexts where performance and assurance is required.

Some basic notes about Merge Sort algorithm.

Description

Recursive algorithm based on the divide and conquer strategy. It has two main components:

• A function that splits the a list recursively until it receives a single item.
• A «merge» function that merges two already sorted lists into a combined sorted one.

⁠Time complexity⁠

Pseudocode

Implementation in Rust

language-rust
⁠
pub fn merge_sorted_arrays(left: Vec<i32>, right: Vec<i32>) -> Vec<i32> {
⁠  let left_length = left.len();
⁠  let right_length = right.len();
⁠
⁠  // Final array will be the size of both provided arrays
⁠  let mut sorted_items = vec![0; left_length + right_length];
⁠
⁠  let mut i = 0;
⁠  let mut j = 0;
⁠
⁠  // Iterate over the array that we will fill with items either 
⁠  // in left or right arrays
⁠  for k in 0..sorted_items.len() {
⁠    // If neither "i" and "j" pointers did reach the limit 
⁠    // of their respective arrays
⁠    if i + 1 <= left.len() && j + 1 <= right.len() {
⁠      // Decide between current item in left array and current item 
⁠      // in right array which one will be next in final array
⁠      if left[i] <= right[j] {
⁠        sorted_items[k] = left[i];
⁠        i = i + 1;
⁠      } else if left[i] > right[j] {
⁠        sorted_items[k] = right[j];
⁠        j = j + 1;
⁠      }
⁠    } else if i + 1 <= left.len() {
⁠      // If only "i" pointer didnt reach the limit —but "j" did—
⁠      sorted_items[k] = left[i];
⁠      i = i + 1;
⁠    } else if j + 1 <= right.len() {
⁠      // If only "j" pointer didnt reach the limit —but "i" did—
⁠      sorted_items[k] = right[j];
⁠      j = j + 1;
⁠    }
⁠  }
⁠
⁠  sorted_items
⁠}
⁠
⁠pub fn sort_array(unsorted_array: &Vec<i32>) -> Vec<i32> {
⁠  let n = unsorted_array.len();
⁠  if n <= 1 {
⁠    // Base case: if input has length of one it is already sorted, 
⁠    // return it
⁠    let sorted_array = unsorted_array.to_vec();
⁠    sorted_array
⁠  } else {
⁠    // If input has length bigger than two, then:println!
⁠    // 1. Split it
⁠    let (left, right) = unsorted_array.split_at(n / 2);
⁠    let left_vec = left.to_vec();
⁠    let right_vec = right.to_vec();
⁠
⁠    // 2. Sort it recursively
⁠    let left_sorted = sort_array(&left_vec);
⁠    let right_sorted = sort_array(&right_vec);
⁠    let sorted_array_1 = merge_sorted_arrays(left_sorted, right_sorted);
⁠    sorted_array_1
⁠  }
⁠}
⁠
⁠fn main() {
⁠  let unsorted_array: Vec<i32> = vec![2, 1, 4, 3];
⁠  let sorted_array = sort_array(&unsorted_array);
⁠  println!("unsorted_array: {:?}", unsorted_array);
⁠  println!("sorted_array: {:?}", sorted_array);
⁠}
⁠
⁠#[cfg(test)]
⁠mod tests {
⁠  use super::*;
⁠
⁠  #[test]
⁠  fn works_with_sorted_arrays() {
⁠    let unsorted_array = vec![1, 2];
⁠    let sorted_array = sort_array(&unsorted_array);
⁠    let expected_result = vec![1, 2];
⁠    assert_eq!(expected_result, sorted_array);
⁠  }
⁠
⁠  #[test]
⁠  fn works_with_simple_unsorted_array() {
⁠    let unsorted_array = vec![2, 1];
⁠    let sorted_array = sort_array(&unsorted_array);
⁠    let expected_result = vec![1, 2];
⁠    assert_eq!(expected_result, sorted_array);
⁠  }
⁠
⁠  #[test]
⁠  fn works_with_simple_uneven_lenght_arrays() {
⁠    let unsorted_array = vec![2, 1, 3];
⁠    let sorted_array = sort_array(&unsorted_array);
⁠    let expected_result = vec![1, 2, 3];
⁠    assert_eq!(expected_result, sorted_array);
⁠  }
⁠
⁠  #[test]
⁠  fn it_works_with_sorted_arrays() {
⁠    let left = vec![1, 2];
⁠    let right = vec![3, 4];
⁠    let merged = merge_sorted_arrays(left, right);
⁠    let expected_result = vec![1, 2, 3, 4];
⁠    assert_eq!(expected_result, merged);
⁠  }
⁠
⁠  #[test]
⁠  fn it_works_with_splits() {
⁠    let left = vec![3, 4];
⁠    let right = vec![1, 2];
⁠    let merged = merge_sorted_arrays(left, right);
⁠    let expected_result = vec![1, 2, 3, 4];
⁠    assert_eq!(expected_result, merged);
⁠  }
⁠
⁠  #[test]
⁠  fn it_works_with_uneven_length_lists() {
⁠    let left = vec![3, 4, 5];
⁠    let right = vec![1, 2];
⁠    let merged = merge_sorted_arrays(left, right);
⁠    let expected_result = vec![1, 2, 3, 4, 5];
⁠    assert_eq!(expected_result, merged);
⁠  }
⁠
⁠  #[test]
⁠  fn it_works_with_single_item_lists() {
⁠    let left = vec![3];
⁠    let right = vec![1];
⁠    let merged = merge_sorted_arrays(left, right);
⁠    let expected_result = vec![1, 3];
⁠    assert_eq!(expected_result, merged);
⁠  }
⁠}

When I went through the exercises on The Rust Programming Language book I remember finding the example about closures interesting. The provided code was just about implementing a naive cache system to avoid repeated expensive calculations for previously passed inputs. It looked like this:

language-rust
⁠
// Struct accepting a Generic, which has 
⁠// to be a function with an u32 as parameter
⁠// and returning an u32 value
⁠pub struct Cacher<T: Fn(u32) -> u32> {
⁠  pub calculation: T,
⁠  pub value: Option<u32>,
⁠}
⁠
⁠// Implementation for Cacher, where we add our behaviour
⁠impl<T: Fn(u32) -> u32> Cacher<T> {
⁠  // Instantiate the struct with a function, that will be
⁠  // stored within the struct scope as a closure
⁠  pub fn new(calculation: T) -> Cacher<T> {
⁠    Cacher {
⁠      calculation,
⁠      value: None,
⁠    }
⁠  }
⁠
⁠  // Getter to retrieve the current value
⁠  pub fn value(&mut self, input: u32) -> u32 {
⁠    match self.value {
⁠      Some(value) => value,
⁠      None => {
⁠        let value = (self.calculation)(input);
⁠        self.value = Some(value);

⁠⁠        value
⁠      }
⁠    }
⁠  }
⁠}
⁠

Easy enough. If value is empty the cacher will calculate the result and return it; otherwise, it will return the existing value. But there is a catch: if we run the Cacher::value() method twice with different values, it will return the first one that was stored, which we can demonstrate with this test:

language-rust
⁠
#[cfg(test)]
⁠mod tests {
⁠  use super::*;
⁠  use ::std::thread;
⁠  use ::std::time::Duration;
⁠
⁠  #[test]
⁠  fn call_with_different_values() {
⁠    fn my_function(num: u32) -> u32 {
⁠   ⁠   num
⁠    }
⁠
⁠    let mut cacher = Cacher::new(my_function);
⁠
⁠    let value_1 = cacher.value(1);
⁠    let value_2 = cacher.value(2);
⁠
⁠    assert_eq!(value_1, 1);
⁠    assert_eq!(value_2, 2);
⁠  }
⁠}

The issue is that we are storing the value within Cacher.value directly, so once it is calculated the same value will be returned the rest of the times. What we need is a structure that can store the resulting values in relation to the inputs, and one with fast search and insert times. We need a Hashmap.

So instead of using value as an Option<u32> to store a single value, we can use values with an std::collections::HashMap, as follows:

language-rust
⁠
use std::collections::HashMap; 
⁠
⁠struct Cacher<T: Fn(u32) -> u32> {
⁠  calculation: T,
⁠  values: HashMap<u32, u32>,
⁠}

Our hashmap will have u32 keys with u32 values. We can now proceed to instantiate it on our constructor:

language-rust
⁠
fn new(calculation: T) -> Cacher<T> {
⁠  Cacher {
⁠    calculation,
⁠    values: HashMap::new(),
⁠  }
⁠}

And finally create or getter/setter method with the match pattern that will decide what to do:

language-rust
⁠
fn value(&mut self, input: u32) -> u32 {
⁠  match self.values.get(&input) {
⁠    Some(value) => *value,
⁠    None => {
⁠      let value = (self.calculation)(input);
⁠      self.values.insert(input, value);
⁠
⁠      value
⁠    }
⁠  }
⁠}

So if the input already exists in self.values —self.values.get(&input)—, retrieve it; otherwise, set into self.values both the input and value as key/value pair. Notice that when a given value exists —Some(value)—, we retrieve it as *value: that's because what we store is the reference &input, so we need to dereference it to retrieve its actual value. If we remove the dereference we will get the corresponding error notifying that Rust expected `u32`, found `&u32`.

language-none
⁠
⣿ Standard Error
⁠
⁠Compiling playground v0.0.1 (/playground)
⁠error[E0308]: mismatched types
⁠  --> src/lib.rs:18:28
⁠   |
⁠16 |     pub fn value(&mut self, input: u32) -> u32 {
⁠   |                    --- expected `u32` because of return type
⁠17 |         match self.values.get(&input) {
⁠18 |             Some(value) => value,
⁠   |                            ^^^^^ expected `u32`, found `&u32`
⁠   |
⁠help: consider dereferencing the borrow
⁠   |
⁠18 |             Some(value) => *value,
⁠   |                            +

Finally note the parenthesis on (self.calculation)(input). They seem unnecesary, but actually they are. Lets break things first and remove them:

language-none
⁠
⣿ Standard Error
⁠
Compiling playground v0.0.1 (/playground)
⁠error[E0599]: no method named `calculation` found for mutable reference 
⁠`&mut Cacher<T>` in the current scope
⁠  --> src/lib.rs:20:34
⁠   |
⁠20 |                 let value = self.calculation(input);
⁠   |                                  ^^^^^^^^^^^ field, not a method
⁠   |
⁠help: to call the function stored in `calculation`, surround the field access 
⁠with parentheses
⁠   |
⁠20 |                 let value = (self.calculation)(input);
⁠   |                             +                +

As previously, Rust informs us both about the error and possible fix. It is intelligent enough to identify that we are trying to call the function we stored as a closure, thus the need of the parenthesis. If we restore them and try to run our test again, this what we get:

language-none
⁠
running 1 test
⁠test tests::call_with_different_values ... ok

Fantastic, it is passing \(^O^)/.
⁠
⁠Of course this is something really simple, but I like the example because it has a reference and a dereference, a structure from std , a closure, a generic with a bound type and a match pattern. You can't ask for more!
⁠

Given a struct MyStruct living within module my_struct, and a trait Feature living in the module my_feature, it is required that MyStruct holds different implementations of the trait Feature without depending on these implementations.

Our goal will be to tell MyStruct that it may accept any struct that implements Feature via new() method as parameter.

⁠In other languages —mostly with structural typing— the footprint of this new() method would look like this:

language-rust
⁠
⁠fn new(feature: Feature) -> MyStruct;

This won't work in Rust, as it can't infere the size of Feature at compile time.

But this may be achieved via two different strategies:

• Using generics
• Using trait objects

Dependency inversion with generics

It is possible to let the compiler pass the type of Feature as a generic.

A first step would be to define Feature trait within feature module:

language-rust
⁠
⁠⁠mod feature {
⁠  pub trait Feature {
⁠    fn action(&self) -> String;
⁠  }
⁠}

Within my_struct module would live MyStruct. It would receive any generic of type Feature, which can be seen in impl <T: Feature>....

MyStruct will then receive T as generic; now it will be possible to type feature parameter as T.

language-rust
⁠
⁠⁠mod my_struct {
⁠  use super::feature::Feature;
⁠
⁠  pub struct MyStruct<T> {
⁠    feature: T,
⁠  }
⁠
⁠  impl<T: Feature> MyStruct<T> {
⁠    pub fn new(feature: T) -> MyStruct<T> {
⁠      MyStruct { feature }
⁠    }
⁠
⁠    pub fn action(&self) -> String {
⁠      self.feature.action()
⁠    }
⁠  }
⁠}

The different implementations of Feature would live in different modules:

language-rust
⁠
⁠mod feature_implementation01 {
⁠  use super::feature::Feature;
⁠
⁠  pub struct FeatureImplementation01;
⁠
⁠  impl Feature for FeatureImplementation01 {
⁠    fn action(&self) -> String {
⁠      String::from("Feature 01 - with generics")
⁠    }
⁠  }
⁠}
⁠

language-rust
⁠
⁠⁠mod feature_implementation02 {
⁠  use super::feature::Feature;
⁠
⁠  pub struct FeatureImplementation02;
⁠
⁠  impl Feature for FeatureImplementation02 {
⁠    fn action(&self) -> String {
⁠      String::from("Feature 02 - with generics")
⁠    }
⁠  }
⁠}

Finally the client code would consume this logic:

language-rust
⁠
⁠⁠pub fn main() {
⁠  use feature_implementation01::FeatureImplementation01;
⁠  use feature_implementation02::FeatureImplementation02;
⁠  use my_struct::MyStruct;
⁠
⁠  let feature_implementation_01_instance = FeatureImplementation01;
⁠  // If required, it is possible to pass the type of 
⁠  // the implementation into the generic with turbofish syntax
⁠  let my_struct = 
⁠    MyStruct::<FeatureImplementation01>::new(
⁠      feature_implementation_01_instance
⁠    );
⁠  let result = my_struct.action();
⁠  println!("{:#?}", result);
⁠
⁠  ⁠// If required, it is possible to pass the type of 
⁠  // the implementation into the generic with turbofish syntax
⁠  let feature_implementation_02_instance = FeatureImplementation02;
⁠  let my_struct = 
⁠    MyStruct::<FeatureImplementation01>::new(
⁠      feature_implementation_02_instance
⁠    );
⁠  let result = my_struct.action();
⁠  println!("{:#?}", result);
⁠}

It is important to note that when MyStruct::new is called, it is receiving the type of the struct implementing Feature with the turbofish syntax:

language-rust
⁠
⁠let my_struct = MyStruct::<FeatureImplementation01>::new(
⁠  feature_implementation_01_instance
⁠);

⁠Rust compiler may be able to infere it. This is the case with this code: if we remove ::<FeatureImplementation01>, the compiler will be able to find out which type is passed; but it is not always the case, and sometimes the developer may be required to pass it manually.

⁠With Smart pointers

Rust provides several smart pointers for dinamically sized structures. The obvious choice here is Box.
The required changes to achieve this will hapen only on my_struct module and within the client code, as seen below.
It is required to remove the generics on MyStruct; the issue then will be that feature is typed as the trait Feature: but the size of structs implementing this trait may vary.
⁠This can be solved by wrapping Feature within Box<dyn Feature>⁠, telling the compiler to send this struct into the heap, which is dinamically sized. To be explicit about traits being dinamically dispatched Rust forces us to use the dyn keyword.

language-rust
⁠
⁠mod my_struct {
⁠  use super::feature::Feature;
⁠
⁠  pub struct MyStruct {
⁠    feature: Box<dyn Feature>,
⁠  }
⁠
⁠  impl MyStruct {
⁠    pub fn new(feature: Box<dyn Feature>) -> MyStruct {
⁠      MyStruct { feature }
⁠    }
⁠
⁠    pub fn action(&self) -> String {
⁠      self.feature.action()
⁠    }
⁠  }
⁠}

The client code will change as well, as it requires to wrap the feature instance within a Box instance:

language-rust
⁠
⁠fn main() {
⁠  use feature_implementation01::FeatureImplementation01;
⁠  use feature_implementation02::FeatureImplementation02;
⁠  use my_struct::MyStruct;
⁠
⁠  let feature_implementation_01_instance = FeatureImplementation01;
⁠  let my_struct = MyStruct::new(Box::new(feature_implementation_01_instance));
⁠  let result = my_struct.action();
⁠  println!("{:#?}", result);
⁠
⁠  let feature_implementation_02_instance = FeatureImplementation02;
⁠  let my_struct = MyStruct::new(Box::new(feature_implementation_02_instance));
⁠  let result = my_struct.action();
⁠  println!("{:#?}", result);
⁠}

Now, lets say MyStruct makes use of MyService, which at the same time requires the feature instance hold within its struct.

language-rust
⁠
⁠mod my_service {
⁠  use super::feature::Feature;
⁠
⁠  pub struct MyService {
⁠    feature: Box<dyn Feature>,
⁠  }
⁠
⁠  impl MyService {
⁠    pub fn new(feature: Box<dyn Feature>) -> MyService {
⁠      MyService { feature }
⁠    }
⁠  }
⁠}

language-rust
⁠
⁠mod my_struct {
⁠  use super::feature::Feature;
⁠  use super::my_service::MyService;
⁠
⁠  pub struct MyStruct {
⁠    feature: Box<dyn Feature>,
⁠  }
⁠
⁠  impl MyStruct {
⁠    pub fn new(feature: Box<dyn Feature>) -> MyStruct {
⁠      MyStruct { feature }
⁠    }
⁠
⁠    pub fn action(self) -> String {
⁠      let my_service = MyService::new(self.feature);
⁠      println!("{}", my_service.action());
⁠
⁠      self.feature.action().to_owned()
⁠    }
⁠  }
⁠}

Now the compiler returns an error, as MyService is borrowing self.feature:

language-none
⁠
⁠⁠error[E0382]: borrow of moved value: `self.feature`
⁠  --> src/with_box.rs:25:7
⁠   |
⁠21 |       let my_service = MyService::new(self.feature);
⁠   |                                       ------------ value moved here
⁠...
⁠25 |       self.feature.action().to_owned()
⁠   |       ^^^^^^^^^^^^^^^^^^^^^ value borrowed here after move

When passing self.feature to MyService it is consumed, so it wont be available after this line. A solution would be to clone() it; but the compiler will return a new error:

language-none
⁠
⁠error[E0599]: the method `clone` exists for struct 
⁠   `Box<(dyn with_box::feature::Feature + 'static)>`, 
⁠   but its trait bounds were not satisfied
⁠   --> src/with_box.rs:21:52
⁠    |
⁠2   |     pub trait Feature {
⁠    |     -----------------
⁠    |     |
⁠    |     doesn't satisfy `dyn with_box::feature::Feature: Clone`
⁠    |     doesn't satisfy `dyn with_box::feature::Feature: Sized`
⁠...
⁠21  |         let my_service = MyService::new(self.feature.clone());
⁠    |                                                      ^^^^^ method 
⁠                cannot be called on 
⁠                `Box<(dyn with_box::feature::Feature + 'static)>` 
⁠                due to unsatisfied trait bounds

The trait Box doesnt implement Clone. It would be possible to implement Clone for Box<dyn Feature>, but there is a simpler solution: use the Rc pointer instead of Box.

language-rust
⁠
⁠⁠mod my_struct {
⁠  use super::feature::Feature;
⁠  use super::my_service::MyService;
⁠  use std::rc::Rc;
⁠
⁠  pub struct MyStruct {
⁠    feature: Rc<dyn Feature>,
⁠  }
⁠
⁠  impl MyStruct {
⁠    pub fn new(feature: Rc<dyn Feature>) -> MyStruct {
⁠      MyStruct { feature }
⁠    }
⁠
⁠    pub fn action(self) -> String {
⁠      let my_service = MyService::new(self.feature.clone());
⁠      println!("{}", my_service.action());
⁠
⁠      self.feature.action().to_owned()
⁠    }
⁠  }
⁠}

Rc implements Clone, so we can call it over self.feature.

A last case should be taken into account: the code may be asynchronous; but Rc is not thread safe.

The following examples will make use of Tokio as async runtime as well as async_trait to be able to create traits with asynchronous methods.
⁠
For example, if we require MyStruct to implement a trait MyStructTrait, the compiler will warn that both MyStructTrait and Feature should implement Send + Sync :

language-rust
⁠
⁠mod my_struct {
⁠  use super::feature::Feature;
⁠  use async_trait::async_trait;
⁠  use std::rc::Rc;
⁠
⁠  pub struct MyStruct {
⁠    feature: Rc<dyn Feature>,
⁠  }
⁠
⁠  #[async_trait]
⁠  pub trait MyStructTrait: Send {
⁠    fn new(feature: Rc<dyn Feature>) -> Self;
⁠    async fn action(&self) -> Result<String, String>;
⁠  }
⁠
⁠  #[async_trait]
⁠  impl MyStructTrait for MyStruct {
⁠    fn new(feature: Rc<dyn Feature>) -> Self {
⁠      MyStruct { feature }
⁠    }
⁠
⁠    async fn action(&self) -> Result<String, String> {
⁠      self.feature.action().await
⁠    }
⁠  }
⁠}

language-none
⁠
⁠error: future cannot be sent between threads safely
⁠  --> src/with_arc.rs:31:54
⁠   |
⁠31 |       async fn action(&self) -> Result<String, String> {
⁠   |  ______________________________________________________^
⁠32 | |       self.feature.action().await
⁠33 | |     }
⁠   | |_____^ future created by async block is not `Send`

It is possible to tell the compiler to extend both Feature and MyStructTrait with Send + Sync; there will be another obstacule though: x cannot be sent between threads safely.

language-none
⁠
⁠error[E0277]: `Rc<(dyn with_arc::feature::Feature + 'static)>` 
⁠  cannot be sent between threads safely
⁠  --> src/with_arc.rs:26:8
⁠   |
⁠26 |   impl MyStructTrait for MyStruct {
⁠   |        ^^^^^^^^^^^^^ `Rc<(dyn with_arc::feature::Feature + 
⁠                          'static)>
⁠       ` cannot be sent between threads safely

The pointer Rc is useful on sync code, but is not thread safe: luckily for this Rust provides Arc, which, as per docs: «... provides shared ownership of a value of type T, allocated in the heap. […] Unlike Rc<T>, Arc<T> uses atomic operations for its reference counting. This means that it is thread-safe.»

Using Arc has more memory consumption than Rc, but for this case is negligible. And as implements Clone it will be possible to perform self.feature.clone() in order to pass it to Service.

⁠language-rust
⁠
⁠mod my_struct {
⁠  use super::feature::Feature;
⁠  use async_trait::async_trait;
⁠  use std::sync::Arc;
⁠
⁠  pub struct MyStruct {
⁠    feature: Arc<dyn Feature>,
⁠  }
⁠
⁠  #[async_trait]
⁠  pub trait MyStructTrait: Send + Sync {
⁠    fn new(feature: Arc<dyn Feature>) -> Self;
⁠    async fn action(&self) -> Result<String, String>;
⁠  }
⁠
⁠  #[async_trait]
⁠  impl MyStructTrait for MyStruct {
⁠    fn new(feature: Arc<dyn Feature>) -> Self {
⁠      MyStruct { feature }
⁠    }
⁠
⁠    async fn action(&self) -> Result<String, String> {
⁠      self.feature.action().await
⁠    }
⁠  }
⁠}

Conclusion

The convenience of generics over trait objects will depend on the context. Generics will have better performance, as its size is known at compile time and will use static dispatch; trait objects, on the other hand, will be send into the heap and will use dynamic dispatch, wich will have some memory overhead. But the flexibility it provides may worth it for many cases.

In short: each pointer —Box, Rc or Arc— will allow different actions:

• Box: synchronous code where the trait object instance will be consumed once.
• Rc: synchronous code where the trait object instance will have to be cloned in order to feed different factories.
• Arc: asynchronous code that requires thread safety.

With these four options —generics, Box, Rc and Arc— it is possible to perform dependency inversion and apply the patterns that may be required in our codebase.
⁠The full source code for the four examples can be found at  git.antoniodiaz.me/antoniodcorrea/rust-dependency-inversion .

Some notes about type conversions between Rust and Postgres with tokio-postgres library, async PostgreSQL client for Rust.

There are two possibilities when mapping from Rust types into types related to some infrastructure:

• The type mapping from Postgres types into my Rust types.
• The type mapping from Rust types into Postgres types.

The library used, tokio-postgres provides traits with some basic implementations that can be used to convert the application types into SQL types and the other way round. For example, there is FromSQL, which will automatically transform Rust primitives into PosgreSQL types: a Rust bool will be transformed into a Postgres bool; i64 into bigint, &str or String into text, and so on. It has to be done carefully, as some unintended results may arise if the developer is not sure about how types are converted; but in general it is consistent with the expectations.

The problem comes when it is required to perform type conversion between objects or some SQL types, as a postgres ENUM into a Rust enum. This is a common feature in libraries providing some kind of Object Relational Mapping (ORMs) functionality; thankfully tokio_postgres does not have this ability.

It is important to remark «thankfully»: ORMs are marketed as a way to speed up development for simple tasks; but they bring some issues into the projects, reason why they were called the Vietnam of Computer Science.

The debate about ORMs is long, and is filled with nuances; among others, there are two main points:

• ORMs force the developer to couple entity and infrastructure layers.
• ORMs are unable to fill the gap between objects and relational structures except for basic cases. For any non trivial case, optimization performed by SQL is lost and queries are duplicated in order to build the required objects.

Therefore, lacking an ORM and being required to manually map between Rust and Postgres types is not a drawback, but good practice.

Mapping from SQL into Rust

As previously mentioned, tokio-postgres exposes a FromSQL trait with a basic implementation for primitives and some specific types. With this implementation we don't have to worry about text conversion into String, or bigint into i64.

The issue arises when it is required a type conversion between a Postgres enum myenum into some Rust enum MyEnum.

language-postgres
⁠
⁠create type myenum as enum('variant_a', 'variant_b');

language-rust
⁠
⁠pub enum MyEnum {
⁠  VariantA,
⁠  VariantB,
⁠}

Enum conversions are not supported by tokio-postgres, as this library doesn't provide any Object <-> Relational mapping feature: the system won't know how to map one to the other. This is where the FromSQL trait exposed by the library can be handy to create an own implementation for the required Rust enum.

FromSQL is a trait that exposes four methods, from which two will be useful for this case: from_sql, which will be in charge of actual type conversion, and accepts, which will check if the type conversion should be performed for the current type or not.

language-rust
⁠
pub trait FromSql<'a>: Sized {
⁠  fn from_sql(
⁠    ty: &Type, 
⁠    raw: &'a [u8]
⁠  ) -> Result<Self, Box<dyn Error + Sync + Send>>;
⁠  […]
⁠  fn accepts(ty: &Type) -> bool;
⁠}

The task is actually easy: from_sql will have to return the correct enum variants, which can be achieved matching against the raw binary string; in the accepts method is possible to perform a check against the name of the enum.

language-rust
⁠
impl FromSql<'_> for MyEnum {
⁠  fn from_sql(
⁠      _sql_type: &Type, 
⁠      value: &[u8]
⁠    ) -> Result<Self, Box<dyn Error + Sync + Send>> {
⁠    match value {
⁠      b"variant_a" => Ok(MyEnum::VariantA),
⁠      b"variant_b" => Ok(MyEnum::VariantB),
⁠      _ => Ok(MyEnum::VariantA),
⁠    }
⁠  }
⁠
⁠  fn accepts(sql_type: FromSqlType) -> bool {
⁠    sql_type.name() == "myenum"
⁠  }
⁠}

As this implementation is related to Postgres, it can live within the Postgres repository or the database related code, and outside of the entities and domain layer. This way it is avoided coupling domain and infrastructure, being infrastructure who is referencing the domain —when mentioning MyEnum—, and not the other way round.

This implementation will tell tokio-postgres how to perform type conversion between SQL and Rust for this specific type, so everytime a query returns a myenum Postgres type, it will appear in the application as MyEnum.

Mapping from Rust into SQL

The opposite case is when in the database there is a myenum Postgres type that should be exposed to our application. tokio-postgres can perform this type conversion, but this will require a custom implementation of ToSql trait using to_sql and accepts methods:

language-rust
⁠
pub trait ToSql: fmt::Debug {
⁠    fn to_sql(
      &self, 
⁠      ty: &Type, 
⁠      out: &mut BytesMut
⁠    ) -> Result<IsNull, Box<dyn Error + Sync + Send>> where Self: Sized;
⁠
⁠    fn accepts(ty: &Type) -> bool where Self: Sized;
⁠
⁠    fn to_sql_checked(
⁠      &self,
⁠      ty: &Type,
⁠      out: &mut BytesMut,
⁠    ) -> Result<IsNull, Box<dyn Error + Sync + Send>>;
⁠}

It is required to implement to_sql_checked as well, but the types::to_sql_checked! macro can generate this method implementation automatically.

When implementing to_sql the Rust enum variant should be transformed into a &str first, which can be achieved implementing Display for MyEnum in order to map between MyEnum variants and a string.

language-rust
⁠
use std::fmt::{Display, Formatter, Result };
⁠
⁠impl Display for MyEnum {
⁠  fn fmt(&self, f: &mut Formatter) -> Result {
⁠    match self {
⁠      MyEnum::VariantA => write!(f, "variant_a"),
⁠      MyEnum::VariantB => write!(f, "variant_b"),
    }
⁠  }
⁠}

After implementing Display for MyEnum it is possible to implement ToSql for MyEnum:

language-rust
⁠
use tokio_postgres::{
⁠  types::{to_sql_checked, ToSql, IsNull, Type},
⁠};
⁠use bytes::BytesMut;
⁠use std::{error::Error, result::Result};
⁠
⁠impl ToSql for Role {
⁠  fn to_sql(
⁠    &self, 
⁠    ty: &Type, 
⁠    out: &mut BytesMut
⁠  ) -> Result<IsNull, Box<dyn Error + Sync + Send>>; {
⁠    format!("{}", self).to_sql(ty, out)
⁠  }
⁠
⁠  fn accepts(sql_type: &Type) -> bool {
⁠    sql_type.name() == "myenum"
⁠  }
⁠
⁠  to_sql_checked!();
⁠}

⁠With this, every time Postgres receives a Rust MyEnum, it will be converted into Postgres myenum.

Conclusion

There are other cases where we will need to perform some type conversion, as with the Row type that can be used with Rust's trait From to convert between Postgres rows into Rust entity objects. For example, for a given struct Article, the From implementation on the repository may look like this:

language-rust
⁠
// Domain
⁠pub struct Article {
⁠  pub id: String,
⁠  pub title: String,
⁠  pub created_at: DateTime,
⁠}
⁠
⁠// Repository
⁠impl From<Row> for Article {
⁠  fn from(row: Row) -> Self {
⁠    Self {
⁠      id: row.get("id"),
⁠      title: row.get("title"),
⁠      created_at: row.get("created_at"),
⁠    }
⁠  }
⁠}

⁠let result = client.query("select * from article").await?;
⁠let articles: Vec<Article> = result.into_iter()
⁠    .map(Article::from)
    ⁠.collect();

In general this library it is quite straightforward to use, and limits its own features allowing us to manually perform our type conversion.