A Cloud Architect day can be full of discoveries and interesting Use Cases. In this blog series we outline a real-world experience and address the possibilities for transformation and application modernization. This first part describes the original application as well introduce some key concepts.
“Our application is consuming too much resources and we are facing performance bottlenecks, please increase the server memory and CPU, we need to convert so many pictures during high season that the server is not being able to deal with it. Ah, we also need more storage, please duplicate the current disk.” - Senior Software Developer request.
This request was the starting point for this blog (or blog series). A (still) very common request to the system administrators on most of the Enterprise world. If we were on a traditional on-premises data center the options would not be many besides adding more RAM and CPU as requested and increasing the disk. Some script lines or clicks to tell the Hypervisor we need more power for this specific virtual machine. For our luck, the server was running on AWS, which offers us a lot of interesting possibilities.
Please do not understand me wrong, monoliths are not bad, they were how software and servers were developed and deployed during decades, and still are on most of Enterprises and software companies. Software development evolved through the time from waterfall to agile methods, delivering quickly and adding new features on every iteration but the main breakthrough from the so called DevOps culture or methods are that the boundaries between how the software is coded and how the software is maintained and run have been removed. This means, operations, infrastructure and the software itself are now part of the same process, so developers have to think about scalability and infrastructure as well sys admins need to code their infrastructure and integrate with the application seamlessly. The main point here is, the DevOps culture removed the boundaries between software and infrastructure, so it is time to rethink the monoliths we have in the basement (or Cloud).
When we talk about monoliths we are referring to applications that perform several different tasks and/or contain several business logic elements which are all implemented on the same source code and deployed as one into one or more servers. The business logic has hard-coded dependencies or architectural elements that require them to run together.
Usually such kind of applications, the so called monoliths can only scale up or vertically, this means the server where the application runs is upgraded getting resources by hardware upgrade or increase of virtual cores and memory, for example. In the other hand, serverless or microsservices architectures are designed to scale out or horizontally when needed, this means, instead of a big box running my application, I can have several small instances running different parts of my application and scaling out or in when the load increases, or in the case of scale in, decreases.
Going back to our use case, the application in question is a CMS, generating html static pages from the input of several editors as well including image galleries which are populated by editors and also receive batch jobs from files that are uploaded to a specific share.
The batch jobs were increasing and this feature of the application required the latest vertical scaling events. As well an increase of 4 times in the AWS EC2 and AWS EBS costs as the unpredictability of the load required the instance to be kept online all the time, even if no batch job was received.
The image below shows the initial scenario, with all application modules running in a single box. Notice that there is already an improvement when we compare with an on-premises setup the database layer is decoupled and runs on an AWS managed service, Amazon RDS, or relational database service. We can notice that image persistence is done on the local file system, which makes vertical scaling the only option.
In the cloud journey usually this is how most already existing or legacy applications land in the cloud. And this is also part of the path. The diagram below helps us on this journey, by the so called 6Rs of cloud application migration. We will not enter in details about the 6Rs, for more info please follow this great article from AWS.
(Image credits: AWS, Stephen Orban on Medium)
Our application initially followed the Rehosting approach, or Lift and Shift strategy, where the application is initially migrated with minimum changes, while taking benefit of quick wins, which are small architectural or services improvements that do not require big changes in the application or refactoring. In our case the database layer was replaced from a commercial database into an open-source RDS-hosted engine, reducing significantly the licensing and operational costs for the database engine.
The rehosting was enough to bring the application to the cloud, but now, with the subsequent vertical scalings taking place, it is time to think about refactoring, or at least decoupling part of the services that consume most of resources, in our case image processing and store.
In the next blog from the series we start to decouple image upload and processing, first moving the image storage to a persistent object store outside our server (using Amazon S3), then moving deeper and changing the upload and processing of images to a serverless architecture (leveraging AWS Lambda and S3 Events) and finally removing the image gallery from the server and doing static webhosting directly from the S3 bucket.
Found it interesting? Please contact us and have a talk with our Cloud Architects and DevOps team to discuss possibilities for our applications and workloads. If you are interested to work with these topics, we are actively seeking eager to learn professionals to join our DevOps Team!
- The 12 Factor App - cloud-native DevOps application development best practices
- AWS Blog Post covering Cloud migratoin strategies