The Web Application Architecture - Intermediate Stage

Part 2: Solving Part 1

In order to discuss more intermediate topics of the proper architecture for a web application, we have to operate on the assumption that your application has already accomplished the fundamentals.

The problem is that the current frog-in-a-pot front end ecosystem has convinced most developers and - by extension - their companies that they have accomplished the fundamentals, and they're ready to move on to more advanced topics. I don't believe this is remotely true. However, I'm not your dad, so I can't stop you from reading this.
Consider, though, that if your development environment has a mandatory build system, or if any of your source code has specific signaling just for your bundler, or if you use React, Vue, Apollo, or any Flux implementation, you cannot possibly have accomplished even one of the three items necessary for the fundamentals of proper web application architecture. These are by no means the full list of tools and practices, but they are certainly some of the most popular.

That statement probably causes a visceral reaction in you, perhaps even disgust and anger. I am genuinely sorry to make you feel this way. Unfortunately, the level to which web developers have incorporated the tooling that has built up in their toolbelt over time into the core of their identities is alarming, and we can't keep dancing around the fact that the tooling is a huge part of the problem.
If you've incorporated these tools into your identity, you are not the problem! You are valuable and useful! 🙂 But we do have to look at the cold truth about our tooling, processes, and architectures.

So: read on - nobody can stop you. Just know that if you don't truly nail the fundamentals, anything else you do will be built on a foundation of sand.

It's the fun part

The silver lining here is that once you've nailed the fundamentals of de-complexity, robust foundations, and inter- & intra-app communication, everything else becomes much easier.
Removing complexity and establishing a standard way everything "talks" removes much of the difficulty of removing, adding, and upgrading things. You're then left to work on the actual problems that further your software's goals.
But wait! you exclaim, my tooling also allows me to focus on my core business problems instead of all the other junk!
But does it? In every "bog-standard" ecosystem I've worked in, developers almost never solved novel problems or dramatically improved overall application performance or created uniquely elegant solutions to dramatically improve customer experience.
Almost without fail, most developers spend the vast majority of their time fitting pre-defined solutions to inflexible libraries and systems.

So when you escape from that cycle - the one where everything gets worse and worse over time until you can't recover it any more and you either leave the company or are forced into a full rewrite - web software development gets fun again.

In the first part, I was intentionally vague about how to solve these problems. It's possible the right solution for your situation doesn't look like the solution for mine.
However, architecture is about blueprints, so here's one with more specifics.

There will always be software tools

You could absolutely 100% roll your own solutions to every problem, but it's unlikely that you should for the most complex parts of an application.
That complexity should be irreducible (for example: updating the DOM), and not self-inflicted.

There must always be systems

The original sin of most of the terrible software in modern web development is that they are tools that try to solve problems that can only be solved by systems.
What's the difference?
Tools plug into specific parts of systems. Systems are ideas. You can encapsulate a system purely in documentation. You should probably also have some code that implements the system, but it isn't technically necessary.

De-complexifying systems

Here it is: What does de-complexifying my application look like?

• Anything that solves more than one problem at a time is too complex.
• Anything that deviates significantly from your platform is too complex.

It should be pretty clear why those statements are true, but here's a quick explainer.
If something solves two problems at the same time ("problem" here are classes of application-level concerns, like say: requesting information from an API, request caching, and data binding), any issue with one problem becomes something that affects all of the other problems.
You have an issue with how data binding into your components work? Now you also have a problem with how network requests to your API work. That intertwinedness should actively raise your blood pressure.
In similar fashion, when something deviates significantly from your platform, you become less able to adapt to underlying changes to that platform. This is the problem with all of the VDOM frameworks in use today: they deviated significantly from the platform. The platform has a native component model and they're unable to even come close to the performance or flexibility provided by the platform.
Experienced engineers warned of this a decade or more ago. Code written in defiance of platform norms is zombie code: already dead and a danger to everything around it.

Simple front end application structure

Recall that simple is not the same as easy. Simple systems separate from each other so that they can interoperate more seamlessly by agreeing on standardized interfaces. This allows them to act as discrete black-box systems instead of having their internals deeply intertwined.

Generic

You could break most web applications down into these four (really three) parts:

Specific

It could help to better define these parts of the application, so we can rename each more precisely:

Specifically correct (separation of the parts)

In these images, each part of the application is connected directly to another part. This type of connection is how most applications today are built, and it encourages deeply broken interactions and spaghetti code. This is not only because there aren't clear boundaries about where things should go, but also because it fundamentally doesn't matter: the application's UI is virtually indistinguishable from its core value proposition. If you put application logic directly in a UI component, it doesn't matter, because the UI is the business logic.
This is - of course - madness (See De-complexifying systems, just above).
A robust, scalable architecture requires that each part of a system operate in a predictable way. To keep predictability to a maximum without complex interdependencies causing literal exponential growth of potential outcomes (see: Enumerate, Don't Booleanate for an exploration of explosive exponential growth of interdependent state on a micro scale - within one component), each part should attach to a standardized interface.
You could call this "Signals" or "Actions" or "Events" or "Messages," but whatever they are, the idea is that the application has a backbone of messages passing back and forth, and when a relevant message is noticed, something occurs. Critically, this is the only way the application should operate. Every behavior and data change should be able to be followed through a clear history of messages being passed. I have talked about this multiple times before. I also built gravlift to be this carrier backbone for any system.

Different types of display logic

The "Pages, Components, & Component State" section is doing a lot of work, so we can split that up appropriately.

Defining data

For many people, defining data is difficult. This is made especially difficult because many tools improperly conflate the concepts of application state (what is the application doing right now) and core application data (what is the information that any user is here to see). These tools incorrectly treat these two concepts as the same, and store them the same way and provide the same interfaces for accessing and mutating them.
The clear defining line is "If you wouldn't store it in your database, it's not data." Loading indicators, which sections of a page have been collapsed by the user, which temporary view modification is enabled, and more are all examples of data you wouldn't store in your database. These are component-level state concerns and don't belong in data.
Data is a broad abstraction of three separate things. Your application's value proposition should not be aware of these distinctions; that's why the Data interface is separated by the standardized Messenger.

• Browser Storage
• Single-Concept Utilities
• API

The "single-concept utilities" above requires a bit of explanation. It is very common to need to do certain things with known types of data that should happen based on raw data at this low level of the application. For example, you may be building an app that allows users to view and rate dogs (they are all 12/10). After the application gets data from the API, it may need to import the normalize function from Dog.js and make sure to smooth over any potential rough edges in the data from the API. Dog.js might also provide some helpers like makeNew for when someone is submitting a new dog, and the system needs a "default" Dog value for them to fill in.
The two-fold key is that these utilities are tightly focused (for example, just Dog.js, not normalizeData.js) and they are limited to data. There's a whole part of the app for the value proposition code, this area should just be for unavoidable data mungeing.

More elaborate components and component DOM

Components hold a bit of information (the state of the application, which we covered earlier in that we know it is not data). Component DOM also updates regularly, which means we should define how that happens.

A blueprint for long-lasting, high-performing applications

And so we have arrived at the simple application architecture. Had you followed each of the principles in part 1, you may have arrived at a similar architecture. It is not necessarily easy, but the best things never are. It is undoubtedly elaborate, but most meaningfully complex web applications are elaborate. In this structure, the complexity is kept from seeping into every corner of the code.

1. The application is split into four discrete parts:
   1. Data
   2. the application's Value Proposition (the Business Logic)
   3. the display logic, split between Pages and Components
   4. the rendering artifact of that logic: the UI, which returns reactive signals back to the components
2. Communication is passed among the Data, Value Proposition, and Pages/Components by a message system with a standardized format
3. Data reflects database information. It is not application state. The Data application layer abstracts:
   • accessing the API
   • Browser Storage (local cache, for one example)
   • Single-concept utilities, which help normalize and interact with data constructs
4. Components use Finite State Machines to catalogue non-exponential lists of application states, the non-data type of information your application must manage.
5. Component DOM updates are managed by the family of Web Components browser features (like Shadow DOM, custom elements, and constructable stylesheets). These updates are not full-component re-computes (like VDOM libraries), but instead maintain connections to live versus static parts of the template, and only update those parts that have received changes.