The New Web: Massive Concurrency with Elixir, Phoenix Channels, and Redux

Share Button

TL;DR: Elixir, Phoenix, React, and Redux are mature and productive technologies for building incredibly fast, massively concurrent, reliable, real-time web applications. Nothing else comes close.

In Engineering Terms, Where We Are and Where We Want To Go

HelloSign been able to build a successful business based on a simple set of technologies: PHP, Symfony, and jQuery. To this day, these technologies serve as core parts of our stack and work quite well. We do not see us ever completely removing these parts of our codebase.

Yet we’ve had some issues in terms of maintainability, performance, and productivity, especially with jQuery. Maintaining complicated imperative jQuery code can be something of a nightmare. Plus there were parts of our architecture that were not giving us the performance our customers needed. This led us to a search for some long-term solutions to these tricky problems.

Ultimately, we embraced a few approaches to our problems as we started to scale.

For greenfield projects, we settled on a new tech stack consisting of Elixir, Phoenix, React, and Redux. But the tech stack is really an outcome of a shift in philosophy at HelloSign toward functional programming and immutability, and event driven architectures. This new approach allowed us to build a massively concurrent, fault-tolerant, real-time application at scale.

Why Elixir?

From Wikipedia: “Elixir is a functional, concurrent, general-purpose programming language that runs on the Erlang virtual machine. Elixir builds on top of Erlang and shares the same abstractions for building distributed, fault-tolerant applications.”

So why did we choose Elixir?

First of all, because it’s built on top of the Erlang VM, we get a lot of the benefits of Erlang. Originally created by Ericsson, Erlang was specifically created for telephony systems. That means it needed to have certain features: high availability, massive concurrency, and fault tolerance. Think about it – when was the last time a landline phone didn’t work?

Elixir gives you a few added benefits Erlang does not.

For one, the syntax is a lot nicer. Originally created by core contributor to Rails, Elixir’s syntax was intentionally designed to resemble Ruby. Elixir took with it the friendliness and inclusivity of the Ruby community. The tooling is also a lot better.

But Elixir has another benefit: a really great web framework similar to Ruby on Rails but written in Elixir called Phoenix.

The Phoenix Web Framework

In many ways, Phoenix was inspired by Ruby on Rails, yet avoids a lot of mistakes and magic that Rails is known for. Best of all, it’s fast. Very fast. Performance in Phoenix is not measured in milliseconds – it’s measured in microseconds.

Unlike Rails, Phoenix has a powerful real-time story (Action Cable really falls short here) through what it calls Channels. It’s just what it sounds like: a way to send and receive messages. Processes can send messages about topics and receivers can subscribe to topics so they can get those messages and handle them accordingly. While Elixir is based on message passing, Channels allow senders and receivers of messages to be anything, not just Elixir processes. Phoenix even provides an excellent library for JavaScript clients to easily connect to a channel and send/receive messages on that channel.

Valid Phoenix Channel messages require the following keys:

  • topic – The string topic or topic:subtopic pair namespace, for example “messages”, “messages:123”
  • event – The string event name, for example “phx_join”
  • payload – The message payload
  • ref – The unique string ref

If you’re familiar with Flux, this might seem awfully similar to Flux Standard Actions.

The Front-End Architecture

1. React for Views

React is a popular JavaScript view library for building UIs. It’s used at a number of companies, including Facebook, Instagram, Netflix, Airbnb, and many others. With just Facebook and Netflix alone, that’s 30% of the internet, so it’s battle-tested. Plus because it’s so unopinionated, it can be added to existing codebases very well. That means we can add it to our existing jQuery code, only re-writing and/or refactoring the parts that need to.

2. Redux for State Management

One issue we were finding with pure React applications is a great way to separate concerns: How do we separate view logic from application logic? How do we manage application state? We settled on Redux, which is a simple state container designed for React-like applications.

3. Everything Happens Through Flux Standard Actions / Messages

One benefit of Redux is that state is updated via Flux Standard Actions and a unidirectional data flow, which as mentioned earlier look a lot like Phoenix Channel messages. Since Redux state is immutable and is updated based on actions, given some initial state you can rewind all the actions to get to the exact state at any given point. This means if we have all the user’s actions and they leave the app at any point, when they return we can replay all the actions and get to the exact same state as before.

4. The Saga Pattern and ES6 Generators for “Concurrency”

It is true that because JavaScript is single-threaded, you don’t have true concurrency. But with ES6 Generators, you can essentially emulate concurrency. Generator functions are a lot like normal JavaScript functions except that they can halt execution part-way through the function call.

In normal functions, execution of the function never stops. But in a generator function, execution of the function will halt on `yield` statements.

This means we can yield a time consuming / asynchronous call and continue execution once that call is finished. However, how do we manage all these concurrent operations in a way that fits our architecture?

We decided to adopt the Saga pattern for our JavaScript client code. Originally proposed in a paper by Hector Garcia-Molina and Kenneth Salem, the Saga pattern proposes an alternative mechanism (called sagas) for managing long running distributed transactions. In essence, it’s a failure management pattern. There are strategies around how to handle failures, whether to roll back to a prior state or fail forward, depending on the situation. In many ways, this fits with Elixir’s “Let it Fail” approach to concurrency and fault-tolerance.

We use Redux Saga to tap into the Redux “channel” or “stream” of actions, and with ES6 generators we can create de-facto daemon processes that respond to particular actions. We can also spin up sagas that open up a Phoenix channel and dispatch messages to that channel or buffer messages from the channel and dispatch them to Redux store.

Conclusion

These approaches have been a huge success at HelloSign. Phoenix Channels, Websockets, and Redux work so well together because they’re based around the same ideas: functional programming and event-driven architectures. This makes the whole greater than the sum of its parts.

By now, we have an architecture that we feel can grow and scale with HelloSign for years to come. Elixir, Phoenix, React, and Redux are mature enough to productively build the future of web applications.

Like This Post? You Might Enjoy These Too

Or poke around HelloSign’s documentation.