Illustrating Microservices in Action: A Closer Look at the Task Manager Application
In today’s rapidly evolving technological landscape, software development approaches have undergone a paradigm shift. One of the most influential trends that has emerged is the adoption of microservices architecture. Microservices architecture is a modern software design pattern that aims to build large, complex applications as a collection of small, independent, and loosely coupled services. Within this project, we have employed this design pattern to circumvent issues of race conditions within the server-side service during scalability enhancements.
The Task Manager is a component within the GenDiseak modules, tasked with handling the delegation of storage and analysis workflows (Figure 1). As the user base expands in the future, the GenDiseak service will require scaling up by deploying multiple replicas within the Kubernetes cluster to accommodate increasing requests. However, a challenge arises due to simultaneous database updates by these GenDiseak replicas, which can lead to potential race conditions (Figure 2).
Figure 1. Task Manager module in GenDiseak
Figure 2. Race conditions in replicas
To address the problem and prevent redundant task execution, the approach involves centralizing the Task Manager (Figure 3). You can visualize the Task Manager as a project manager and the GenDiseak replicas as the workforce. Upon receiving a task from a user or other processes, GenDiseak establishes a task entry in the database by interacting with the Task Manager. Subsequently, the Task Manager continuously scans for pending tasks and delegates them to the GenDiseak replicas (Figure 4).
Figure 3. Centralized Task Manager
Figure 4. Interaction between GenDiseak and Task Manager
A Cron task is under a specific task category. In the case of GenDiseak, it periodically monitors the Illumina BaseSpace for uploaded files and initiates the analysis pipeline whenever new sequencing files are detected. Let’s illustrate how many cron tasks collaborate with the Task Manager using an example. Initially, the Task Manager schedules the very first cron task. Once the scheduled time arrives, the initial task is assigned to GenDiseak for execution. Subsequently, if the task is completed, the Task Manager’s API is invoked to update the task status. This cycle continues for the subsequent tasks in a similar manner (Figure 5).
Figure 5. Cron tasks example
Within the project scope, the Sidecar pattern was employed for the Task Manager and its corresponding database. Additionally, the Strangler Fig approach was leveraged to replace the original Task Manager within the GenDiseak system. To sum up, the adoption of the microservices design pattern presents a revolutionary method in software development. This approach empowers our platform services to construct scalable applications by disassembling monolithic structures into discrete, self-contained components.
