Welcome to our “BizTalk to Azure Migration Series.” In this session, we’ll explore the powerful publish-subscribe patterns in Azure and how they compare to traditional BizTalk setups. Whether you’re looking to streamline message routing with Azure Service Bus, handle high-volume data with Event Grid, or explore streaming with Event Hubs, this video will guide you through the options available to ensure your migration is both effective and efficient. Let’s dive in!
Today, we’re going to continue with our BizTalk to Azure Migration Series, and we’ll be discussing how to implement publish-subscribe patterns on Azure. If you recall, back in the BizTalk days, everything needed to come in through a receive sign—like a receive location or a receive port. Everything had to come in that way, and then it all went into the message box, which was just a database. From there, either for orchestrations or for send ports, you would have filter expressions that would route the message to go out. This setup made it mandatory to implement the publish-subscribe pattern, whether you wanted to or not. Even if you wanted to go directly from A to B, you had to put publish and subscribe in the middle. Now, with Azure, you don’t have to do that. If you want to go from A to B, you can do so by simply creating a logic app or something similar, just as A to B. However, there are times when you really want to implement a publish-subscribe pattern—such as when one message comes in and you need it to go to multiple different systems downstream.
Some of those downstream systems might not always be available, so you may want to make use of some queuing capabilities. Additionally, some of those systems may only be interested in certain messages, not all of them. These scenarios are well-suited for the publish-subscribe pattern, which is one of the most important integration patterns you can learn. In Azure, we have a wealth of ways to achieve this, but the go-to option is really Azure Service Bus. Azure Service Bus is the oldest messaging platform in Azure, and it’s truly the Swiss Army knife of messaging. It does all these things—though it may not have the highest performance or throughput, which we will discuss later. However, it definitely has the richest functionality. If you want to send a message to your Service Bus, you create a topic; different systems can write to that topic, and different systems can have a subscription to take messages off that topic. Creating subscriptions is a key part, but this effectively implements your publish-subscribe pattern. Whether you want to pull those messages off with a logic app, an Azure function, or some custom code you’ve written, it’s all there. Service Bus is very flexible and allows you to do other things, not just publish and subscribe. It’s excellent for high-integrity, persistent messages. For example, if you want to implement a peek-lock pattern, you can process a message but only mark it as consumed once you’ve finished working on it. This is really good for high-integrity scenarios where losing a message is not an option.
Event Grid, on the other hand, comes from a very different background. It’s much more lightweight, flexible, and scalable as a messaging platform. Originally, it wasn’t designed for high integrity; it was more like events in a programming language—something raises an event, and that event is only processed if there are listeners to it. That’s how Event Grid started. You could raise an event to Event Grid, and if nothing was listening, that event would simply disappear. Since then, it has become more persistent, offering options for scenarios where the downstream system might not be available. However, one of Event Grid’s strengths is in handling high volumes of small messages. Service Bus can handle quite large messages, but Event Grid excels when you have high volumes of small messages. IoT is a key scenario here, as Event Grid is the core foundational technology behind IoT. It’s excellent for high-volume, lightweight scenarios where you might not need every single piece of telemetry data—losing a few items isn’t the end of the world. Event Grid is great for handling tons of data quickly, making it a very flexible option, especially for high-volume applications.
Finally, we have Event Hubs, which is quite different again and more akin to Apache Kafka. Event Hubs are designed for streaming data. You can stream data into an Event Hub, and then write consumers to read that data, with the ability to have multiple consumers. It’s less about events and more about streams. If you have large amounts of data coming in and you want multiple downstream systems to read copies of it, Event Hubs is another technology to consider. Event Hubs excel at handling streaming data with really high volume. It can manage huge amounts of data coming in very fast, making it ideal for busy workloads. Since it’s streaming, it provides a continuous flow of data. It also offers interesting recovery options. For example, with a stream like this, you’re effectively writing a sequence of entries in a table, each with a unique number. Your consumer reads down the sequence—first 1, then 2, 3, 4, and so on. But if you encounter a problem, you can reset the pointer and read that stream again. Event Hubs represent a significant change in mindset, as they don’t operate like traditional BizTalk workloads. Instead, they align more closely with Service Bus or event-based patterns. However, if you need a way on Azure to write data into one place and have multiple different systems subscribe to it, these are your options.
If you want to find out which option is best for you, feel free to reach out to me, and we can find the best solution together.
