Scalable Enterprise Server Architecture – A Scale Cube

It all began in 1995. Ever since the introduction of client-side scripting (JavaScript) by Netscape back then followed by the introduction of web-application concept using Java Servlet by Sun in 1999, for the business, the web has become a cost effective channel to communicate with prospects and customers. Simply put, a web application is an application-program that is stored in a remote location computer (server) and delivered over the Internet through a browser interfaces like Google Chrome, Internet Explorer, Mozilla Firefox etc. 

Web applications have become such a global sensation and for businesses, it has never been more important than it is right now to have a strong online presence in order to create long-lasting profitable relationship with consumer base across the globe.

Sometime end of last decade there was a huge paradigm shift in how the web solutions are accessed by global customers. Introduction of smartphones and “app stores” by Google, Apple, Windows, and Amazon have set the strategy in mobile software development and guided the path to new areas of revenue. Beginning of this decade the approach to building enterprise (web and mobile) solution is changed to “Mobile First”.

In coming years, enterprise solution needs more complex architecture than what it is now, thanks to IoT (Internet of Things) interface and real-time analytics for quick decision making.

So, how difficult is it to build Enterprise applications? Not that challenging to be honest. The development process is often simplified by open source software platforms and frameworks to facilitate teams with rapid application development without having to worry about scalability and robustness of the hosted application. The development team just needs to focus on their part of the development that is unique to their specific goals.

Enterprise application is hosted in remote locations running a web-server/app-server with a database on the back-end to store enterprise data. Initially the application is hosted in a single physical server that takes care of all services including web-service, application-service and database-service. The application make uses of all server resources such as CPU, disk, memory, network interfaces, etc.

This intertwined approach is known as “Monolithic” architecture. This architecture is good to start with. In this tightly-coupled architecture, each component and its associated components must be present in order for code to be executed. This design is feasible only for enterprise applications that receive very low traffic.

Once the enterprise application gains popularity, the adoption rate is going to be very high. Increase in user base and frequent access to the application is going to raise the web-traffic. This subsequently translates to poor/longer response times for the end-user. At some point the application becomes almost entirely unusable. So it is the responsibility of the engineering team to think about how to scale the platform well in advance to avoid catastrophic failures like this and re-engineer the application to Scale to ever growing need.

So, How to Scale? what is Scalability?

According to Andre B Bondi in a 2000 WOSP paper, “scalability is the capability of a system, network or process to handle a growing amount of work, or its potential to be enlarged in order to accommodate that growth”.

Before scaling an application, the engineering team should have a complete knowledge of current architecture issues, bottle-necks, optimal thread pool size, hardware/software resource usage statistics, user info with traffic flow rate, etc.; this data would help in engineering a better architecture model, defining scalable components and resource capacity planning. 

There are many ways a enterprise application can be scaled:

Vertical Scaling (Scale Up): Vertical scaling means deploying the enterprise application on a server with higher capacity than the one it is currently deployed on. The new server may have a faster CPU, more internal memory with faster memory-bus, hi-speed hard disk with more data to accommodate, etc. However if enterprise application is not configured properly to take advantage of expensive hardware there may not be a significance performance improvement.

This is a simple and quick way of scaling up.

Horizontal Scaling (Scale Out - Technical Architectural Layering - Hardware Load balancing - X axis scaling): What horizontal scaling does is basically clone the current server, share load across multiple instances of the same application and data set. With a little downtime more servers (nodes) can be rapidly added to improve the performance of the application. The input load is managed and allocated to any one of the node using load-balancer. There are many load balancing techniques. The load-balancer is configured in a highly available manner with a passive peer becoming active, should the active peer fail because of hardware or software problems. This approach just distributes the requests across several nodes and it does not address other potential bottlenecks like memory constraints where memory is used to cache information or results. 

Microservice Architecture (Functional Decomposition Segmentation - Business Layer balancing - Y axis scaling): Microservice is like a collection of small Monolithic components that are fully independent but designed to work together to form a single application. Because it is independent, it can evolve on its own and could be deployed separately. 

Each Microservice can have its own choice of technology but should communicate to another Microservice via well-defined API agreements. Amazon, Netflix, Google, Facebook etc. are examples of tech companies heavily relying on Microservice architecture to push new features to production several times per day. Microservice architecture helps to scale the application from a service, or function, or resource perspective. A service could be identified by a set of use-cases. For example, a verb or action like “login” could be a service or a noun like “account balance” could be a service.

Two Dimensional Scaling 

This helps in distributing the memory dedicated to any given application across several systems. However this approach can have a higher cost of implementation with increase in the number of applications/service splitting up. This type of split is often referred to as “swimlaning” an application.

Data Sharding (Data Distribution - Database Layer Balancing - Z Axis scaling): Data Sharding is the technique using which the database is scaled. This is nothing but partitioning the database based on logical separation and grouping of data.

Z-Axis Scaling (Data Partitioning)

For example, the data can be separated and grouped either based on geo data or data category or any other data property. When a request is raised for a data access, a lookup module will identify the node that contains the data and serves to the requester. This is relatively more expensive compared to the other scaling methods and is only recommended for the applications that have exponential growth and high volume of transactions. This technique should not be implemented without implementing Micorservice first. The book “Art of Scalability” authored by Martin Lee Abbott and Michael T. Fisher describes, Data Sharding technique can offer the greatest degree of scalability as the number of “swimlanes within swimlanes” that it creates is virtually limitless.

The Scale Cube – Three Dimensions to Scaling

Software architecture has gone through several iterations of development and refinement in the last three decades. As an enterprise applications architect myself, I strongly believe we are on the verge of a huge seismic shift in the software architecture. 

In my opinion, three-dimensional scaling is the best scaling technique that combines X-axis, Y-axis and Z-axis scaling model together to implement a highly scalable enterprise application. This is in other words, also known as a “Scale Cube”. 

The Scale Cube helps us keep the critical dimensions of system scale in mind when solutions are designed. As with many architectural metaphors and pattern languages, the Scale Cube provides a great framework to work with, thanks to the vast array of options it offers.

Scalability is all about the capability of a design to support ever growing traffic without compromising performance. It is important to understand that while enterprise application architecture might fall under one of the four categories I described above, there can be no “one key opens all doors” approach to designing salable IT solutions.

Amongst many other things, design architects must strategize and take into consideration things like target audience, consumer maturity, product landscape, planned short-term gains and long-term goals etc. when deigning customer centric solution without compromising organization’s key goals and objectives. 

For example, companies like Facebook (social network), YouTube (content streaming), Amazon (eCommerce) , and Bank of America (Financial sector) all have some sort of baseline design similarities with their solutions. But what separates one from another is what their key objectives are as far as their scalability and design requirements may be. It might not be as critical for Facebook to focus on successfully posting a news feed on someone’s wall as it might be for Bank of America to successfully go through with a financial transaction online. Amazon’s availability and the need to be “up and running” 24*7 might be considered a little more crucial than a video failing to load the first time a user tries playing it on the YouTube application. 

Simply put, there’s no “plug and play” option available in architecting enterprise solution that will work seamlessly from the get-go. There’s bound to be some tweaking and design effort involved to get the system to do what we need it to do. 

Conclusion

Technology supported businesses with exponential growth suddenly shriveled because: 

1. Failure to quickly correct the current unavailability issues 
2. No strategy to address the growing traffic
3. Engineering and Development team spending more and more time on technology disasters and lesser time being spent on new products and features
4. More downtime due to the application architecture’s inability to scale up with the ever increasing consumer transactions resulting in business loss

To reiterate, a perfectly scalable system is one that has the capability to scale up and scale down at will with minimum cost/effort and adapt to the ever-changing consumer market with as small a downtime as possible. Because in the end, no matter how good the solution is, if the customer can’t get access to it, it’s as good as not having a solution at all.

Cheers,