• Post #1: Creating an ESB on-ramp that receives from Azure’s AppFabric Service Bus
• Post #2: Sending a Message from BizTalk to Azure’s AppFabric ServiceBus with a Dynamic Send Port

In this third post, we will see how to use an ESB off-ramp to send a message to the Windows Azure AppFabric ServiceBus. We will actually be doing the same thing as we did in the second post, however, we’ll be doing it in a different way.

There is an accompanying video for this post (as I did with the others too), which you can find here.

The sequence used here is:

1. Message is picked up from a file drop (because that’s how most BizTalk demos start:))
2. An itinerary is retrieved from the itinerary repository and applied to the message
3. The itinerary processing steps are performed, and the message is sent to the ServiceBus
4. The message is retrieved by the receive location I wrote about in my previous post
5. A send port has a filter set to pick up messages received by that receive port, and persists the file to disk

The last two steps are not covered here, but are shown in the video.

In this case, we’re not using an orchestration at all, we’ve specified in the itinerary that all processing happens at a messaging level.

The first part of this is the itinerary selection, which is as shown below. As you can see, I’ve “hard-wired” this receive location, and anything coming through it will get the “ServiceBusTest.OrderRelayNetTcp” itinerary applied to it.

The itinerary itself is also quite simple (this one is slightly different than the one shown in the video, but they are functionally equivalent).

All of the magic happens in the “SetProperties” resolver. In this case I am using a STATIC resolver, however, that just means I have statically specified the values. The settings are being applied dynamically by the itinerary processing components. Note that I have specified we are using the WCF-Custom adapter provider. As was the case with the first two posts in this series, I have installed the Windows Azure platform AppFabric SDK, which gave me some additional bindings that let me relay messages through the ServiceBus. In the second post, we used one of those bindings along with the WCF-Custom adapter by setting some values on a dynamic send port. In this one, we will set the same properties, but do so using the out-of-the-box ESB Toolkit WCF-Custom adapter provider. Each adapter provider registered with the ESB has a UI that can be associated with it, and you get there by clicking the Endpoint Configuration ellipses:

If you compare the values above to what I set in the second post in this series, you’ll see they are pretty much the same.

And…. THAT’S IT!!!!! That is all we have to do in order for BizTalk to send a message to the Azure platform AppFabric ServiceBus using an ESB off-ramp. As you can see from the behavior specified, this is a secured channel (in this case I am using “shared secret”). The actual security part of this is provided with the Windows Azure platform AppFabric Access Control Service, which is tightly integrated with the ServiceBus.

Note also that in this example I used the netTcpRlayBinding, which is a high performance .NET-to-.NET, but also requires TCP/IP ports be open, which could be an issue depending on how locked down your network is.

For my demo purposes, I use the ws2007HttpRelayBinding (also provided with the Azure AppFabric SDK), as that’s over port 80.

In this series of posts I have shown you how to bridge the gap between on-premise and Windows Azure AppFabric ServiceBus using out-of-the-box BizTalk and/or ESB Toolkit components. There is much more that can be done, this is only the beginning. We are entering a really interesting time, and I think that “hybrid” applications, with some part on-premise, other parts off-premise, will become increasingly common. As that happens, having a clean integration solution becomes very important so that we don’t create “Point-to-point Spaghetti 2.0”, a more modern version of the chaos that many experienced integration developers have seen all too frequently.

I hope you’ve enjoyed these posts and videos, and that they help you as you explore the new and exciting world of Windows Azure.

This post shows two key interactions with the Windows Azure platform AppFabric Service Bus:

• receiving a message from the Service Bus through a BizTalk receive location
• publishing to the Service Bus from an InfoPath form

It is the first point that I think illustrates what will become a VERY commonly used pattern. This sample is based on the Order Demo I created previously, and you can go watch those videos first to get a sense of what the demo is. It brings together BizTalk, SharePoint, ESB Toolkit, SQL Server Analysis Services, SQL Server Reporting Services, InfoPath… and more, including showing how to use the Business Rules Engine for dynamic itinerary selection (a favorite pattern of mine). I blogged about that here. I like it because it’s a “SharePoint-based ESB-driven BizTalk-powered workflow”, really leveraging the power of the Microsoft stack.

The video for this blog post is available here at the MSDN BizTalk Developer site.

My goal in this post is to take an on-premises business process, and make it externally available. Now, traditionally in a BizTalk environment that would mean TYPICALLY that you expose a schema or orchestration as a Web service, and then reverse-proxy that so it can be reached from the outside world, probably with a load balancer in the mix.

There’s new tool in our toolbox now: Windows Azure platform AppFabric, which includes the Service Bus and Access Control Service. So, I set out to use that as an external relay, and an entry point into my existing order process demo.

Before we integrate with the existing system, let’s get the client->ServiceBus->BizTalk flow working.

Once you’ve signed up for the Azure service and received your developer token, the first thing you need to do is install the Windows Azure platform AppFabric SDK in your BizTalk environment. One of the things this gives you is a set of new WCF bindings, which is what I used in conjunction with BizTalk’s WCF-Custom adapter. I created a new BizTalk project, a new receive port and receive location, as shown below:

I had many options available, but as I was going .NET (BizTalk) to .NET (Service Bus), I opted to use the Net TCP binding for greater efficiency (perhaps not so important for a demo, but, might as well do it right). Note also that transport-level security is enabled.

For credentials, I used “shared secret”. This means that it will use the Access Control Service to authenticate me, and the corresponding issuerName and issuerSecret need to correspond to what you have set up using the Azure portal.

Normally, that would be it, but in my case there was a structural difference between what was being relayed to me from the Service Bus and what I needed for my existing process. So, I used the WCF adapters capability to reach into the message and only pass on what I had specified as being the body based on my XPath expression:

And… that’s it! Enabling that receive location will create an endpoint “listener” (subscriber) in Azure’s Service Bus. Pretty remarkable really, when you consider that I needed to magic at the BizTalk end other than a new WCF binding, and I can now subscribe to message from the cloud. This is the sort of thing where when you first see it work, you sit back in your chair, say “wow”, and then think about what the implications are.

Next step, we need a publisher. My existing demo already had an InfoPath form, so I wanted to use that. However, how do you make InfoPath talk to Windows Azure platform Service Bus? Now, I’m hardly an InfoPath expert, but I thought I’d give it a shot anyhow. Following the lead from samples in the Azure SDK, I knew I had to set up some contracts. I had to embed this right in the formcode for the form, perhaps there’s a better way (any InfoPath experts, I’d love to know), but my goal here was “just make it work”, knowing full well that I’m dealing with CTPs and ultimate production-ready approaches would likely change anyhow. So, with that as my mindset, the following is what I ended up with. Notice the OrderRelayService class, and how that aligns with the URI in the BizTalk receive location.

Next, the “Save” button code just sets the order status, and then calls the SendForm method:

Lastly, we have the sending part. Yes, I have my credentials in the clear here, but once this post goes live those credentials will have changed :)

And, that’s it we can now use an InfoPath form to send messages to the Service Bus, and if the Access Control Service authorizes it, the message will be published. BizTalk will have an endpoint listening and will pick up that message.

The last thing we need to do now is hook this into the existing process flow. If you watched the original Order Demo video, you’ll know that I was creating orders in InfoPath, publishing to a SharePoint document list, and from there they were picked up by a BizTalk receive location that used the SharePoint adapter. As part of that, we use one of the standard ESB Toolkit pipelines to select and apply an itinerary, and we selected the itinerary based on a rules engine decision. So, how hard would it be to integrate that functionality into the messages we’re picking up from the Service Bus? Simple, we just add that pipeline to the receive location, and we’re done.

In this post, and the accompanying video, I have shown you how to take an on-premise BizTalk application and make it available externally via the Windows Azure platform Service Bus, by just adding another receive location. And, a huge benefit you get here against having exposed your own endpoint is the scale benefits of the cloud. You don’t need to worry about having servers in a DMZ, reverse proxies,  load balancers, or any of that infrastructure “goo”.

At this point you may want to sit back, say “wow”, and think about what new possibilities this opens up and the solutions you can now create, combining on-premise BizTalk ESB (and perhaps some of your existing applications) with the Windows Azure platform.

Meeting will be at the Microsoft La Jolla office, 6:00 for pizza, meeting starts at 6:30 and goes until we decide to stop :)

Topic

Recap and discussion of some of the major Connected Systems, SOA and ESB announcements and product roadmaps from PDC

Abstract

PDC saw a flurry of announcements about emerging technologies and future versions of existing ones. Join Microsoft’s Brian Loesgen as he re-caps some of the major announcements, and what they mean for people working in Connected Systems technologies, or people building SOAs. Brian will also spend some time discussing the SOA Manifesto (http://soa-manifesto.com) which he helped co-author and was released at the SOA Symposium conference in Rotterdam recently. He will also show (for the first time) a cool new demo that integrates on-premises ESB with the Azure platform AppFabric Service Bus, a new design pattern we have at our disposal (and will be in fully supported production when Azure goes live Feb 1 2010), and will be widely used.

This will be an informal discussion, with some demos and videos, and will cover a broad range of aspects of future technologies. Should be a fun meeting!

The interview is at Interview Series- Four Questions With … Brian Loesgen

And, in case you’re wondering, although I do tend to eat salsa often, I am now, not nor have I ever been a Salsa dancer :)

Enjoy!

Last week at PDC 2009, Sriram did two things, he announced BizTalk Server 2009 (as I blogged about here), but also, he demonstrated some early prototype work that was being done around bringing together BizTalk, Windows Workflow Foundation, and Windows Server AppFabric. That announcement of R2 and the sneak preview of V.Next are significant enough that I decided to do a separate blog post about each.

Before we start though, let’s set expectations. What was shown in terms of WF/BizTalk integration was a very early prototype. This is NOT something you’ll be able to download next week, and put into production next month. This is early-stage code showing the direction the team is heading in. Have said that, if you follow my blog, you know that’s exactly the stage I like to get involved at :), and I feel it’s very important to keep an eye on the horizon and know what direction you’re heading in.

There are actually a few aspects to what was shown.

First and foremost, was WF integration into BizTalk. Let’s consider some of the motivations behind this:

• a common ask from customers is a low-latency solution (not all integrations require it, but in some cases, it can be a must-have). BizTalk is a reliable messaging platform, once a message is received, it is never, ever, lost. That reliability has a cost, as it means that there has to be durable storage (in our case SQL Server), and no matter how well tuned, persisting a message in durable storage has a latency cost associated with it
• sometimes people aren’t sure which way to go as there are four ways to compose services on the Microsoft stack (usually capabilities lead to a decision, but there can be overlap), with no easy migration path between them:
  1. using BizTalk orchestration
  2. using Windows Workflow Foundation
  3. using BizTalk ESB itineraries
  4. using custom code

Of those four, custom code is an option, but it puts a lot of responsibility on the developer to do “plumbing”. If or example you compose 3 services, and service #2 is unavailable, what do you do? You need to code protectively to respond to that. Or, service #2 returns a fault, what does that mean in terms of the overall composition and flow? Personally, I think a more efficient way to go about this is to let something else worry about the minutiae of the plumbing, which is what we get with the integration server capabilities of BizTalk Server.  So, although there will always be times where it is the right way to go, let’s leave custom code off the list for the rest of this post.

That leaves us with three options, two of which are  BizTalk options, and then WF which is currently completely outside BizTalk. As you probably know, with WF 3.5, if you wanted to host a workflow, you had two choices: use the hosting capabilities in SharePoint, or write your own host (which can be a daunting task depending on far you go towards being a fully featured enterprise-grade host). With .NET 4, that changes with the announcement of Windows Server AppFabric (which I blogged about here). What Sriram showed was a preview of what it make look like if WF were integrated into BizTalk.

From a BizTalker and SOA perspective, I really like the idea of having WF integrated into BizTalk, as it makes it centralizes the service composition mechanisms, and to some degree, helps mitigate the “when to use what” question as that can be reduced to a set of “common” guidelines. However, what he showed goes well beyond just using BizTalk projects as a shell from which to get at the WF designer, the integration goes much deeper than that, even in this early prototype. Bringing WF into BizTalk doesn’t change BizTalk, it extends it.

Here’s look at this from a Visual Studio perspective, notice the XAMLX files in a BizTalk project. Notice also the workflow designer. This is the same workflow designer that people will be used to working with Visual Studio 2010.

Bringing WF into the BizTalk environment brings benefits to the WF developers too. One of those is mapping, the screen shot below shows how you can have either a data contract or a schema on both source and destination sides, and regardless of which you pick, the experience is the same. That also means you get all of the new BizTalk Server 2009 R2 mapper goodness that I blogged about here. From there, as Sriram shows at about the 45 minute mark of his presentation, it’s a one-click operation to generate a workflow custom activity to invoke that map, and you can then work with in the WF designer by just dragging it from the palette onto the design surface.

From a deployment perspective, there was a crude prototype shown, this is obviously not what it would look like in final release. The thing to notice here is the fact that the workflow is being hosted in the BizTalkServerApplication host. So, if you follow that logic a step further (rampant speculation on my part!), then that means you could use the standard BizTalk host instance model to have certain workflows running in certain hosts, and then you have instances of those hosts on selected machines.

There’s a lot of cool stuff in this video, and it points to an interesting convergence of technologies in a future release of BizTalk. It also raises a bunch of questions for me, and adds to a long wish list (which I’ve communicated to the team :)). The bottom line though is that WF integration will give BizTalk developers the ability to handle low latency scenarios in a standard framework supported way, and it also add value for WF developers, as it provides them with the possibility of migrating their workflows into BizTalk and gaining additional capabilities as they do so.

What about Windows Server AppFabric and Windows Azure AppFabric?

What remains to be seen is how this fits into AppFabric, something the team is exploring now. An obvious one to me (rampant speculation!) is that AppFabric is just another host for the workflow (as it could be if the workflow were not part of a BizTalk project), be it Windows Server AppFabric or Windows Azure AppFabric, however perhaps getting there through the BizTalk deployment model and taking things like transformation activities with it. Even today, with BizTalk Server 2009, we have integration from BizTalk into Windows Server AppFabric (and also in the other direction) by going through WCF. So, the integration exists, and will only get deeper in future releases.

Finally….

The last point I want to make is something that may or may not be clear from these two BizTalk announcements. The announcements show Microsoft’s firm commitment to the 10,000+ current BizTalk customers, as well as to the product itself. The capabilities planned for BizTalk Server 2009 R2 and the capabilities shown last week for the release beyond that show the level of commitment and investment that Microsoft is making in the platform. As an insider who’s been involved with BizTalk since before the first release, I can tell you that the degree of enthusiasm and excitement in the product team is very, very real. They have great vision, and the resources they need to execute on that vision. The future keeps getting brighter, this has been a fun ride, and it looks like we have even more fun ahead.

The video of all of this is at http://microsoftpdc.com/Sessions/SVR15. The things I’ve talked about in this post are in the second half of the video.