Exploring Publish-Subscribe Patterns in Azure: A Migration Guide from BizTalk

From BizTalk to Azure: A Shift in Mindset 

In the BizTalk world, the publish-subscribe pattern was an integral part of every workflow. Messages had to be received through a designated receive port, stored in a message box (essentially a database), and then routed to their destinations through orchestrations or send ports using filter expressions. This pattern was mandatory, even for simple point-to-point messaging scenarios. 

Azure, however, offers more flexibility. If you want to directly connect one system to another, you can do so with minimal overhead, using tools like Logic Apps. However, there are still many scenarios where the publish-subscribe pattern is beneficial or even necessary—especially when a single message needs to be routed to multiple downstream systems, some of which may not always be available. 

Azure Service Bus: The Swiss Army Knife of Messaging 

Azure Service Bus is often the go-to choice for implementing publish-subscribe patterns in Azure. As the oldest messaging platform in the Azure ecosystem, Service Bus offers a rich set of features that make it extremely versatile. It may not always provide the highest performance or throughput, but its extensive functionality makes it a reliable choice for many scenarios. 

With Service Bus, you can create topics that serve as message distribution points. Various systems can publish messages to these topics, and multiple systems can subscribe to receive those messages. This setup enables robust publish-subscribe workflows, whether you’re pulling messages off with Logic Apps, Azure Functions, or custom code. Additionally, Service Bus excels in scenarios requiring high-integrity, persistent messaging, such as implementing peek-lock patterns to ensure that messages are not marked as consumed until they are fully processed. 

Event Grid: Lightweight, High-Volume Messaging 

Event Grid provides a different approach to messaging in Azure. Initially designed as a lightweight event-handling system, Event Grid is ideal for high-volume, low-latency scenarios. Unlike Service Bus, which can handle large messages, Event Grid is optimised for small, frequent messages, making it a perfect fit for Internet of Things (IoT) applications. 

Originally, Event Grid was not intended for high-integrity messaging. It operated on a fire-and-forget basis, where events were processed only if listeners were available. However, Azure has since added persistence options, making Event Grid more versatile and reliable. It is particularly well-suited for scenarios where you need to handle vast amounts of data quickly, and losing a few messages is not critical. 

Event Hubs: Streaming Data at Scale 

Event Hubs is Azure’s answer to streaming data, comparable to (and compatible with) Apache Kafka. If you need to handle continuous, high-volume data streams with multiple consumers, Event Hubs is the way to go. This service is designed for scenarios where data arrives in large bursts and needs to be processed in near real-time. 

Event Hubs allows you to stream data into a central hub, where it is written as a sequence of entries. Consumers can then read this data sequentially, and in case of errors, they can reset their pointers and reprocess the data stream. This approach is a significant departure from traditional BizTalk workflows, which tend to be more event-based rather than stream-based. 

Choosing the Right Solution for Your Needs 

Migrating to Azure from BizTalk requires careful consideration of your messaging and integration needs. Whether you need the robust functionality of Service Bus, the high-volume capabilities of Event Grid, or the streaming power of Event Hubs, Azure provides a wealth of options to suit different scenarios.