console.blog( ,

The Web Application Architecture

Thomas Randolph

Thomas Randolph

 

Part 1: The fundamentals

Because this is a topic that is both broad and deep, it's not efficient to try to pack everything into one entry. Instead, for this first post, I'll just cover the three most foundational topics for a front end application.
That is:

  1. De-complexity
  2. Robust foundations
  3. Inter- and intra-app communication

De-complexity

The first principle of any web application must be a strict adherence to simplicity. I beg of you that you watch this video of Rich Hickey giving a talk titled "Simple Made Easy" at Strangeloop 2011.

One of the many take-aways from that talk is this: Simple is the opposite of complex, and complex means "interconnected" or "intertwined." Simple code is not monolithic, because monoliths touch many parts of your application, inherently increasing the complexity of it. Moreover, simplicity is not how easy something is. They are not opposed ideas, but to avoid complexity, you must sometimes choose simplicity over how easy something is.

Robust foundations

In tandem with the rejection of simplicity in favor of temporary easiness, the lost decade of development also built a culture of platform rejection. Despite building for the web, the companies and thought-leaders driving the industry pushed for less and less dependence on it. Throughout the years, this gave rise to many attempts to bridge the gap back to the platform like prop drilling, global variables (Flux, Redux, Vuex), "polymorphic" code, and on and on. Each aimed to solve a problem introduced by the fevered dogma that the way things were done before was bad, actually, and we've figured out the right new way. Never you mind all these glaring issues, we'll patch them over with the hottest new thing you need to add to your stack. Developers constantly longed for a return to the platform, but they were sold a pack of lies that the platform was broken and they needed these bloated tooling solutions (and of course their glorious leaders) to make good software.

Features that should have been implemented on top of browser-native tools were instead implemented - shoddily, slowly, and in user-hostile ways - in authored code, running through a slow compiler (sometimes multiple slow compilers!).

The creator of JavaScript often says: "Always bet on JS". As he notes, this is not because JS is necessarily great, but because it is established and evolvable.

Indeed, JS - and the entire web platform - has been evolving (quite rapidly) while the anti-web web developers have been churning through libraries and patterns to try to re-implement the web on top of the web, while trying to avoid using the web.

There are far too many deeply technical topics to go into thoroughly here, so here is a short list of facts:

  1. Your user interface - if it requires components - should be fundamentally based on the family of Web Components technologies: custom elements, Shadow DOM, and their associated tooling like constructible stylesheets. This is core to the platform and a non-negotiable skill nearly a decade after their wide adoption and support.
  2. Data is separate from application behavior, component structure, or network access. This means that data must be managed as a separate entity from these other parts, just like it is when at rest (in your back end database). In practice, you will likely use robust tooling like IndexedDB to store long-lived data, localStorage to store short-lived data, and sessionStorage to store even shorter-lived data. Components should have no idea this exists.
  3. Front end routing is not a user interface component concern.
  4. Network (or any other side effect like disk access, peripheral control, data storage & retrieval, etc.) is not a user interface component concern.
  5. The user interface has a particular state which is wholly and completely separate from your application's data. They should never mix or even be stored in similar ways. A component is a machine that can move through pre-determined states in pre-determined ways, represent it as such, specifically for that component. You could use a tool like a finite state machine.

Inter- and Intra-app Communication

Other than a general focus on simplicity and relying on the robust platform itself, one of the most fundamental parts of building a web application is addressing how things communicate. Some of those "things" will be within the same app, and some of those "things" will be other apps, but they all need to communicate.

While "app communication" may not be the most important concept (that's undoubtedly a fervent adherence to simplicity), it is without a doubt the most important section to grasp and implement correctly, which is why most of this post is dedicated to it.

The previous sections have been - largely - conceptual. Application communication moves from concepts like "simplicity" and "using the platform" to critical direction. Failing to properly solve the problem of application communication in a versatile and flexible way that still maintains application simplicity is a matter of survival; applications that fail in this manner always have deep performance and user satisfaction problems and it is always sooner than expected and they are always complex, tangled messes that are intractable. The "dreaded full rewrite" is a trope not because it's fun, but because developers fail to architect the communication fundamental and reach for the same broken solutions.

Many supposed solutions to this problem have arisen from the ashes of what modern development torched as "the past." There were solutions that relied heavily on the concept of data changing (Flux, Redux, Vuex). There were solutions that depend on API responses - cached or not - only to eventually be essentially the same concept (GraphQL, Apollo). There were solutions that demanded extremely tight coupling, which resulted in code patterns like property drilling and logical mixins.

In nearly all of these so-called solutions, imperative commands are integrated directly into where they are needed in the moment. This is easy and convenient for developers, but leads to out-of-control bloat and - perhaps more importantly - disastrously fractured code that attempts to interface with the application in just one way, but tries to solve many problems that become increasingly specialized over time. The costs of this bloat and complexity are never borne by the developers, they are always passed onto the consumer.

The actual solution - which is foundational to the entire internet and web browsers - is to use events.

If an application strictly adheres to an event system, it cleanly separates interactions from actions, and triggers from behaviors. Safe in that separation, the application is free to add more specialized actions and behaviors, bifurcate interactions and triggers, and generally grow linearly and - critically - simply.

It may be helpful to think of this architecture as similar to one of the most successful architectures of all time: the human body.

Take a simple action that's often used as an example: touching a hot stove. There is no embedded command in the hands or fingers for "when you touch a hot stove." What happens is that the nerves in the fingers identify a trigger: "this temperature is extremely high." That trigger is passed along to the brain just like that: "the fingers have sensed an extremely high temperature." The brain then decides how to handle that. In the vast majority of people, the brain converts that message into "pain!" Consider that "pain!" is not handled by the fingers or the hands. It is the central nervous system's generic response to many events. It just so happens that the brain responds to "the fingers have sensed an extremely high temperature" with the generalized reaction of "pain!" Finally, the brain (when functioning normally) sends a message back down along the nerves: "the hand should be retracted!" This is managed by the particular appendage, which knows how to receive messages like "retract hand!"

A flowchart with a large 'Brain' section at the top. Below the brain on the left side is a section titled 'Knowledge and Abilities'. Below the brain on the right side is a section titled 'Body'. The two sections are separated by a red barrier. They do not directly communicate with each other. Each section has bi-directional communication with the Brain. Knowledge and Abilities communicates with the brain by receiving triggers and sending reactions. Body communicates with the brain by sending triggers and receiving reactions. Knowledge and Abilities has two main items: 'Pain!' and 'Retract Hand'. It also has two other items, which define how it handles triggers. These latter two items are structured with up to three sections: when, send, and act. The first of these items has all three sections and it reads: 'When Pain! Send Retract Hand. Act by creating a negative association.' The other item creates a more contextual trigger. It reads: 'When extreme temps, send Pain!' The body section has its own abilities, but is mostly handlers. The body has the 'contract muscle' ability. The body has sub-sections titled Right Hand and Right Arm. The right hand has two items in the same 'when/send/act' structure as the brain. The first of these items reads: 'When extreme temperature, send extreme temps.' The latter reads: 'When Retract Hand, contract muscle x, contract muscle y.' The Right Arm section also has one of these items. It reads: 'When Retract Hand, contract muscle z.'
An overly-simplistic outline of how the brain and body interact with messages.

This brain/body structure works well because each part of the body doesn't need to know what everything means. The brain knows that extreme temps mean pain and knows how to react to that, but the hand simply sends along the thing that's happening and knows how to contract muscles.

This separation is important, because it makes humans flexible. We can learn new reactions and associations without growing a new appendage. If we lose a hand, we can adjust how we live our lives because what we need to do is handled by the brain, which can adapt and send the same common reactions in unique combinations to get the same job done.

This flexibility is possible in web applications, too. Consider a similar flowchart, but for reloading some data in a display.

A flowchart with a large 'Events' section at the top. Below the events on the left side is a section titled 'Business Logic'. Below the events on the right side is a section titled 'Components'. The two sections are separated by a red barrier. They do not directly communicate with each other. Each section has bi-directional communication with the events area by sending and recieving events. The business logic section has one main ability and one handler. The ability is to 'Update our-data.' When that ability succeeds, it sends the event 'our-data:updated.' The handler item has three parts: when, send, and act. This handler reads: 'When we recieve the event our-data:request-update, send our-data:updating and act by triggering Update our-data.' The components section has a component called 'Data Table'. The data table has three of the items that have the when/send/act structure. The first item reads: 'When the user clicks the reload button, send the our-data:request-update event.' The second item reads: 'When we receive the our-data:loading event, disable the reload button and show a spinner.' And the third item reads: 'When we receive the our-data:updated event, enable the reload button, hide the spinner, and update our data.'
A simple example of the "brain/body" structure of an application event system, showing a "data reload" cycle of triggers and behaviors.

Note that this application does not have to connect "reloading Our Data" to the data table button, or even to any UI element at all. It's just an event that can happen as a result of anything, including user interaction. Likewise, note that the data table doesn't update when the data for it changes, it updates when the application tells it that an update occurred. That doesn't necessarily mean the data changed, just that the application considers the data updated. That's a powerful nuance that disconnects triggers from behaviors. The data table could refresh its data display at any time the our-data:updated event is observed, regardless of the trigger.

Like a body and brain, this strict separation allows flexibility, adjustment, and remixing. New combinations of reactions are possible without affecting what triggers cause those reactions or where they originate.

.finally

This is not a comprehensive entry on this subject. The web application architecture is an extremely broad topic, and it has deeply technical discussions embedded in it.

But, for a beginning, this will have to do. Focus on these three things to eliminate the technical debt of bad early decisions that compound for years.

  1. Simplicity above all
  2. Use the robust platform features
  3. Communicate only with events