Multi-region web application with custom Storage Table replication

This architecture provides a high availability solution for a web application that uses massive amounts of data. It's a flexible approach that can provide a global solution that distributes applications and data to keep it close to users.

The architecture requires custom replication software. depending on the applications and the configuration, this can be challenging to create.

Here are some possible configurations:

• Active/Passive: There's a primary region that normally provides service to all users. There's also a standby region that becomes active when the primary region can't function. When the primary system is active, a replication service replicates database changes to the standby region.

• Active/Active: There's a primary region that normally is active, providing read service to nearby users and write service to all users. One or more other regions are active and provide read-only service to nearby users. Writes are always directed to the primary region, and reads are always directed to the nearest active region.

  As with the Active/Passive configuration, there's a standby region that becomes active when the primary region can't function. When the primary system is active, a replication service replicates database changes to the read-only regions and the standby region. When the standby region is active, the replication service replicates database changes to the read-only regions.

  One drawback to this approach is the high latency of write operations.

• Multi-active: There are multiple active regions, each capable of providing full service to users. User activity is always directed to the nearest active region.

  Implementation of multi-active is quite challenging, and can require expert design and implementation.

Since replication is a custom implementation, the consistency level can be whatever is needed.

The possible difficulty of implementing custom replication and the time required are important considerations with this architecture.

Potential use cases

The architecture may be appropriate for any application that uses massive amounts of data that must always be available. Examples include apps that:

• Track customer spending habits and shopping behavior (retail industry).
• Forecast weather (agriculture, environment, and media/news industries).
• Offer smart traffic systems or implement smart traffic systems or use smart technology to monitor traffic (automotive and transportation industries).
• Analyze manufacturing Internet of Things (IoT) data.
• Display smart meter data or use smart technology to monitor meter data (energy industry).

Architecture

Download a Visio file of this architecture.

Dataflow

1. The client authenticates with Microsoft Entra ID and is granted access to web applications hosted on Azure App Service.
2. Azure Front Door, a firewall and layer 7 load balancer, switches user traffic to a different Azure region if there is a regional outage.
3. Azure App Service hosts websites and RESTful web APIs. Browser clients run AJAX applications that use the APIs.
4. Web APIs delegate function apps to handle background tasks. The tasks are queued in Azure Queue Storage queues.
5. The function apps hosted by Azure Functions perform the background tasks, triggered by the queued messages.
6. The custom replication software assures tables remain identical among regions.
7. Azure Cache for Redis caches table data for the function apps. This offloads database activity and speeds up the function apps and web apps.
8. Azure Table Storage holds the data used by the web applications.

Components

• Microsoft Entra ID is an identity and access management service that can synchronize with an on-premises directory. Azure DNS is a high-availability hosting service for DNS domains that provides apps with fast DNS queries and quick updates to DNS records. Managing Azure DNS is like managing other Azure services, and uses the same credentials, APIs, tools, and billing.
• Azure Front Door is a secure content delivery network (CDN) and load balancer with instant failover. It operates at the edge close to users, accelerating content delivery while protecting apps, APIs, and websites from cyber threats.
• Azure App Service is a fully managed service for building, deploying, and scaling web apps. You can build apps using .NET, .NET Core, Node.js, Java, Python, or PHP. Apps can run in containers or on Windows or Linux. In a mainframe migration, the front-end screens or web interface can be coded as HTTP-based REST APIs. They can be segregated and can be stateless to orchestrate a microservices-based system. For more information on web APIs, see RESTful web API design.
• Azure Functions provides an environment for running small pieces of code, called functions, without having to establish an application infrastructure. You can use it to process bulk data, integrate systems, work with IoT, and build simple APIs and microservices. With microservices, you can create servers that connect to Azure services and are always up to date.
• Azure Storage is a set of massively scalable and secure cloud services for data, apps, and workloads. It includes Azure Files, Azure Table Storage, and Azure Queue Storage. Azure Files is often an effective tool for migrating mainframe workloads.
• Azure Queue Storage provides simple, cost-effective, durable message queueing for large workloads.
• Azure Table Storage is a NoSQL key-value store for rapid development that uses massive semi-structured datasets. The tables are schemaless and adapt readily as needs change. Access is fast and cost-effective for many types of applications, and typically costs less than other types of keyed storage.
• Azure Cache for Redis is a fully managed in-memory caching service and message broker for sharing data and state among compute resources. It includes both the open-source Redis and a commercial product from Redis Labs as managed services. You can improve performance of high-throughput online transaction processing applications by designing them to scale and to make use of an in-memory data store such as Azure Cache for Redis.

Alternatives

• Azure Traffic Manager directs incoming DNS requests across the global Azure regions based on your choice of traffic routing methods. It also provides automatic failover and performance routing.
• Azure Content Delivery Network caches static content in edge servers for quick response, and uses network optimizations to improve response for dynamic content. Content Delivery Network is especially useful when the user base is global.
• Azure Container Apps is a fully managed, serverless container service used to build and deploy modern apps at scale.
• Azure Kubernetes Service (AKS) is a fully managed Kubernetes service for deploying and managing containerized applications. You can use it to implement a microservices architecture whose components scale independently on demand.
• Azure Container Instances provides a quick and simple way to run tasks without having to manage infrastructure. It's useful during development or for running unscheduled tasks.
• Azure Service Bus is a reliable cloud messaging service for simple hybrid integration. It can be used instead of Queue Storage in this architecture. For more information, see Storage queues and Service Bus queues - compared and contrasted.

Considerations

• The architecture requires custom replication software. This can be challenging to create, depending on the applications and the configuration. The possible difficulty of implementing custom replication and the time required are important considerations with this architecture.

• Because replication is custom-designed, developers have great flexibility in implementing a data consistency strategy.

• There are performance limits on Table Storage that can be overcome by adding Storage accounts. The following circumstances may require more accounts:

  • To implement multi-tenancy to support multiple customers
  • To support customers with higher transaction rates
  • To support customers with large datasets
  • To speed up data access by distributing data across multiple storage accounts
  • To segregate data into hot, cold, and archive tiers
  • To make copies of data for backup and reporting purposes

  For more information, see Scalability and performance targets for Table Storage.

• If your application already contains data, then you need to write routines to copy old data to storage accounts. Make sure that you have timestamp and copy flags to track the progress of migration of data.

Contributors

This article is maintained by Microsoft. It was originally written by the following contributors.

• Nabil Siddiqui | Cloud Solution Architect - Digital and Application Innovation

Next steps

• Guidelines for table design
• Use geo-redundancy to design highly available applications

Related resources

• Web-Queue-Worker architecture style
• Data partitioning strategies
• Build scalable database solutions with Azure services
• RESTful web API design