MVC Storefront Part 19: Processing Orders With Windows Workflow

Yeey! A new one.. and you've come back to showing it on the web too.

Very nice :D

Thank you.

I like how "Teh" has a red squiggly line under it. Powerpoints dictionary is so uncool....

Tip to readers out there: don't get behind on this series. It's really hard to get caught back up with Rob producing them so regularly unless you have free time or something.

Really nice job with these, Rob.

Great show :)

If I got the facts right, you kick off the WF task on a separate thread.

Isn't there a race condition in the unit test then when you check is the order has been verified?

Hi Rob,

Great show! Tx man!

I wonder if you could use structure map to handle the lifetime of WorkflowRuntime wfRuntime instance.

Anyway, the padlock is not preventing others threads to create another instance for wfRuntime. You have some subtle bugs there ...

// Ryan

This was a pretty good intro to WWF but my biggest complaint about this series is that you don't go deep into handling negative scenarios.

I'm not talking about crazy edge cases but really common real-world scenarios like : What if the customer's credit card gets declined by AuthorizePayments??? As it is now, you'll still redirect to the success action and the customer will think their order was successful.

You won't be able to use workflows asynchronously when you need to get a return value. Since workflows are quite expensive to execute (even when you've cached the runtime), performance is going to take a hit while you wait for it to return.

OT, Where in the world is Scott Gu, been very quiet lately?

Rob,

I've used WF and agree in some cases has its place. In the majority of other cases WF seems way too overkill. I'm curious if you've ever looked at the SimpleStateMachine. http://www.codeplex.com/SimpleStateMachine. I've been using it for a few months now and feel that WF missed the mark by not providing something similar. I realize your sample didn't use a state machine, but FYI regardless.

One thing that I have always had problems dealing WF is versioning when using persistence. All existing persisted work flows are hosed unless you provided one of the few suggested workarounds in your typical Google search.

In your implementation I think you did a good job of minimizing the negatives when dealing with WF, no work flow persistence, no code that can't be unit tested elsewhere, etc..

Nice addition to the series!

Great show. I think keeping WF is worthwhile, and I'd like to see you expand its use a bit. I believe keeping the new technologies in the code are worthwhile. Open ID, Cardspace, Workflow, etc are all good technologies that just need a bit of promotion. Obviously you shouldn't try to shoehorn something in that doesn't fit, but I don't think you've done that at all.

Good webcast, It makes me wnat to look into workflow more. But when would using it be overkill. You can almost make everything a wf transaction or whatever. I know K. Scott said that he tends to always use the sequential, but the state machine actually sounds pretty interesting as well.

Ok, I'll be the big @$$hole and tell you why I don't like this...

It seems to me that once you've created your reusable actions, the only thing that Workflow is doing for you (at least in the way you've used it) is stitching together calls to those actions. So essentially you still have the sequential "call this then call that then call the other" code you didn't like to begin with, but now it's been created via a snazzy GUI instead of just typing it in. Oh, and now you have to worry about asynch code running in ASP.NET too...

The reason most people tout Workflow is that "people that don't know code can define processes". Well, I'm pretty sure that people who don't know code aren't going to fire up Visual Studio (or some independent GUI tool), create the workflow, AND figure out how to deploy it. Unless there's a more user friendly way of defining the workflow from the app itself that we're not seeing.

So, the bottom line is this:

This technology is really for developers and it just doesn't make things easier enough for the developer to justify its use. It's just as easy to open up a ProcessOrder method and change the sequential method calls within it as it is to open up the spiffy GUI thing and move boxes around. It's a push.

@Micheal Aird: true enough - I should have waited a bit to flesh out a more proper Checkout scenario - I was hoping it would seem obvious from what I was doing. You can handle fault scenarios quite easily and then send out an email (which is what i'll do) - this is simple stuff.

@Jamie:

>>>It seems to me that once you've created your reusable actions, the only thing that Workflow is doing for you (at least in the way you've used it) is stitching together calls to those actions<<<

It may indeed seem like "this is the only thing" - but that indeed is a massive thing. No matter what I do, the code I write for checkout will not fit everyone's needs - the idea here is to make the process simple pimple to change.

>>>So essentially you still have the sequential "call this then call that then call the other" code you didn't like to begin with<<<

The cool thing is you can erase this and create your own. What didn't I like?

>>>The reason most people tout Workflow is that "people that don't know code can define processes".<<<

I'm surely not making that claim - but you have to admit that it makes developing processes a lot cleaner no?

>>>it just doesn't make things easier enough for the developer to justify its use<<<

Why doesn't it make things easier? It's helped me tremendously. RE the asynch-ness of it, that's something I wanted anyway.

>>>It's just as easy to open up a ProcessOrder method and change the sequential method calls within it<<<

Ahh therein lies the rub. You're assuming that I would have coded it that way :) when indeed I did not. In fact what I had was synchronous - not preferable.

I became a Windows Workflow after being inspired by Omar Al Zabir's first Code Project Google IG like ASP.NET AJAX app (now apparently expanded into that O'reilly portal book).

@Jamie, while I can totally see your point; I feel that discussing the validity of WF is like many conversations in the realm of programmatic tools. They usually end with one sentiment: "It's just another weapon in the arsenal."

However, having a quick visual representation of process steps and code-blocks is always a nice way to reduce confusion. It's the same reason why people love pie graphs and not just columns of excel reporting data. Then again it might be a matter of taste too. I used to love the gui in DTS Packages in SQL Server 2000 to logically look at when various ActiveX script tasks were firing. I recall many of my co-workers hating it.

After watching the video (cool) I had the same question as Stefan, "Isn't there a race condition in the unit test then when you check is the order has been verified?"

And a related question...what if you want to redirect to a different action if the workflow fails. For example, if the payment is not authorized, you may want to redirect to a place where the customer can edit the billing address.

Hi Rob, Love your work.

I'm with Jamie at the moment. I don't see the advantage of having 4 boxes in a WF diagram over having 4 calls in a function. Particularly, when I start to see comments about threads and race conditions. Why do I want all that tricky stuff when I am adding a whole other project just to do the equivalent of 4 lines of code. I must be missing something.

I don't really follow the advantage of being asynch either. If the payment fails, eventually, doesn't the customer need to know? Eg, as a customer I don't want to leave the store until I know my payment has been accepted/approved and the order is safely in the sytem.

Doesn't that mean that a customer wants it to be synch rather than asynch?

Maybe wwf comes in to its own for error/exception handling?

I considered using wwf for another project so that my boss could better visualise a process but she dismissed the flowcharts as 'computer stuff, that's your area'.

(PS me not understanding doesn't mean it is wrong, just that I don't get it)

(PPS I admire the way you handle criticism from your readers)

I think it is a great idea to factor out the pipeline functionality to IPipelineService. Then we can choose to implement it as N procedure calls, an N step sequential workflow, or a state machine.

I'm not lovin the async processing. Many order checkout implementations are going to reach outside for help (authorize a credit card, schedule a package pickup, send emails) and those take time. But the checkout process is usually one big transaction...eventually it either succeeds or fails. If it fails, we will need to let the customer know right away...synchronously with the checkout...not asynchronously via email.

I would like to see a little more on the interface so that we can handle failures. Perhaps the pipeline could return an ActionResult?

Well, you know how I feel about adding a new technology to the solution just because it's cool... However, I am a huge fan of LINQ to SQL, and I will champion using the GUI Designer because it really makes doing what you would already do in code just easier without trying to insert a lot of unnecessary cruft. So, I'm actually intrigued about Workflow and how that adds value/ease to the solution. I'd love to see a refresh on the codeplex to see what you've done here to really evaluate whether there are benefits.

And yes, it's nice to know you are listening and thinking about the choices you are making rather than just going 'oooh.... shiny' when you see a new technology. Obviously I don't agree with some of the choices you've made here, and I stand by my statement that you might be making things too complicated for the intended purpose of selling stuff. And I definitely think your tone in the comments is completely different than the tone of the videos.

But... I follow this series because I still think it's very useful and it's interesting to see where you are going... and where you will end up. I wouldn't even call you the Potsie of Programming. I'd reserve *that* title for another highly visibile 'coding' blogger I won't name... though maybe 'Ralph the Mouth' might be better for him.

Jamie, Stefan, Andy and others - yes, there is a race condition with the Unit Test and I probably should have put in a wait timer, but as it turns out it worked so I'm OK with it being there.

That said - this IS a spike. I actually don't need this test at all since I really don't want to test the tooling, and all of the functionality inside of it has its own test.

>>>And a related question...what if you want to redirect to a different action if the workflow fails. For example, if the payment is not authorized, you may want to redirect to a place where the customer can edit the billing address.<<<

This is based on the assumption that what I want to do is inline with the request - aka synchronous, and it's not. I don't want to run auth inline. To answer you, however, I'd send out an email saying "validation of your card failed" and ask the user to come back and correct the issue (this is very, very common).

>>>I don't see the advantage of having 4 boxes in a WF diagram over having 4 calls in a function<<<

It's only 4 now as I haven't done anything yet. I should have waited to have a better process flow - but the point is that for you to orchestrate it for your needs is simple.

>>>Why do I want all that tricky stuff <<

Is it really that tricky?

>>>I don't really follow the advantage of being asynch either. If the payment fails, eventually, doesn't the customer need to know?<<<

The advantage to asynch processing is that you can run auth/fraud check/PO validation/strip searching offline. Having this happen synchronously is a scaling nightmare - in much the same way a forum site doesn't send emails all at once (rather batches them to another thread). If the payment fails, and email goes out to tell the customer the auth failed - this is really really common.

>>as a customer I don't want to leave the store until I know my payment has been accepted/approved and the order is safely in the sytem.<<

There's nothing here that's preventing this... only the exceptional condition where your card is declined (or fails fraud checking).

>>>Doesn't that mean that a customer wants it to be synch rather than asynch?<<<

Certainly not. In fact I'll go so far as to say that the quick submission (lack of waiting for an external auth service) is much better impression-wise.

>>>Maybe wwf comes in to its own for error/exception handling?<<<

Sure - and moreso. There are lots of different conditionals you can throw in there, and quite simply I might add.

>>>But the checkout process is usually one big transaction..<<<

It actually isn't. It's a sequential firing of events that does not need to complete to still be valid. For instance you can auth the card and stop the process. You can then get a call from the user to up the shipping - you can then change it and charge extra to cover it - this isn't transactional at all. It's just a simple sequence of events.

BUT - more to the point, if you want it to be transactional you can certainly throw in a TransactionScope and toss all the activities inside it - then you have a transaction (without writing a line of code), which is killah.

>>If it fails, we will need to let the customer know right away...synchronously with the checkout...not asynchronously via email.<<

Go order a Kindle from Amazon with a bad credit card :). See what happens...

>>>I would like to see a little more on the interface so that we can handle failures. Perhaps the pipeline could return an ActionResult?<<

Currently the pipe isn't returning anything - if it did I'd use some WebServiceInput/Outputs but that gets mess. Tying an ActionResult in there seems a tad messy to me.

>>>I stand by my statement that yo

In terms of my tone - having my decision process relegated to a "fascination with new toys" doesn't usually elicit happy feelings... but feel free to express yourself as you see fit, and I'll do the same.

Great set of videos!

By chance are you going to be tackling validation?

I was hoping to see you integrate a library that cuts across the entire application, where the validation that is enforced at the domain is exposed on the UI. Something similar to how DynamicData implements is validation framework.

Just a suggestion!

You have done such a great job here, that I figured I would ask for more greatness!

Bryan

Those of you advocating that this be a synchronous process rather than asynchronous are forgetting that card validation / fraud check errors are the exception, not the rule - in most cases, financial transactions proceed just fine. Running them synchronously really only optimizes for the exceptional case while 'punishing' the common case.

However I would still like to see a synchronous example - not for order processing, but for something that requires a return value from WF. In my own fiddling with WF I've seen that if you start a new WorkflowConsoleApplication the default code in Program.cs creates a wait handle, wires up the WorkflowCompleted event to Set the wait handle, starts the workflow, then waits on the handle. This sort of solution seems like it would work even in a web application.

Erik, thanks for the explanation. It suddenly become clear to me that the sync vs async in this case is a business decision not a programming decision. Thus it's clearer when viewed as such. Synchronously = programming for the worst case scenario while async = most case scenario so it's easy to see which one would provide the most benefit.

Rob, good work on the series.

Another vote from me for a synchronous example from WF

Erik:

It's a sad but true fact that payment services being not available is not that exceptional a case. Verisign for example was down for two full days at a time during the last year.

Keep up the good work Rob.

I see that the latest update on codeplex was done on June 19. Is this latest code available on anywhere? Btw... Great stuff... thanks for sharing it...

The code will be updated soon - and for those of you who saw Ayende's post - this is a spike (if I didn't make that clear) - the test I wrote isn't really a good and valid test.

Rob and Erik - OK, I understand now why we might choose an async process. Like you I have worked with several ecommerce packages. Some of them do perform the auth step outside of the request processing because 1) it takes awhile and 2) as a form of fraud prevention (if you tell the person that has stolen a cc that the address is wrong, they may try again with a different guess or with a different card). But even with those systems, there are merchants that insist on providing sync feedback to the customer.

Does a workflow (or at least WWF) prevent a sync approach?

Hey Rob,

I like the series. I didn't like this episode, but I understand why you made it.

Keep up the good work.

Nathan it helps if you tell me why..

Andy - it's not prevented really; you can use a WebService input and output to do this. In fact it's dreadfully simple since you can set the action of the form to the ASMX that gets generated for you.

Anyway - it is possible to do this inline but I would offer two things:

1) It remains a scaling issue, which is something the client may not completely understand until it's too late.

2) You can probably push this out of the workflow - or just toss the workflow altogether

Good discussion...

Rob,

I originally wanted to post this reply to your "Subsonic and the Repository Pattern" post, but it appears that comments are locked (and I don't your e-mail to contact you directly), so I apologize for posting it here. Please feel free to delete this or move this as you see fit.

I just finished up a quick and dirty framework called Blunt Architecture that uses some of the concepts you presented in your "Subsonic and the Repository" post. I removed Subsonic from the solution to keep it as generic as possible and broke apart the data layer in your original post into two projects, one for the domain and the other for the data. Considering your pretty vast experience with ASP.NET MVC, I would love to get your thoughts on the design.

You can find my blog posting (with links to the Google Code project, documentation, and source code) here: www.kevinwilliampang.com/.../Blunt-Architect

P.S. Great work on the MVC Storefront series! I've enjoyed it immensely and I think it's a great resource for developers interested in developing against the ASP.NET MVC framework. :-)

Hi Guys,

Is it possible to put up a "live demo" of the user side/admin side of this on the site so everyone can test and use it ?

Cheers,

Andy

I'm all for more WF examples btw - I really liked this episode, mostly because I don't see enough people taking WF seriously, and you gave it a shot with this spike. Keep taking those risks - they pay off in the end, even if the price is pissing people off sometimes. =)

Rob,

A friend and I were just talking about how Windows Workflow (WF) would be utilized in an ASP.NET MVC application and here you are doing it, amazing. I would say that I would like to see the WF used in more of a transitional or orchestral fashion. What I mean by that is if I user decides to abandon the checkout process and come back, they will be placed at the same step as when they left. This probably requires a state machine and a synchronous process.

A side note to Rob Conery. There will always be haters and that is what they like to do. Katt Williams said if people are hating on you then you must be doing something right. You might not have everything right and perfect all the time, but you are putting rubber to the road and that's more than a lot of the haters do.

I also have created and experimented with Activities in the past and I have created a PayPal activity library, along with other activities; if you would like to see it drop me and email.

1st: I'm trying to push TDD here at our company, and we're running into the exact bug you ran into, where all tests run when the user chooses 'run tests in context'. We're not doing any threading or WF, so I don't think that's the issue. In my experience, this typically happens on the first run, then not for subsequent runs. If you get to the bottom of that, please post the solution!

I'd vote to see WorkFlow kept in the solution. However, the thing I think is missing here is that interaction with the UI & user is almost always part of the workflow. For example, I can imagine a client saying that one of the "validation" rules is optional, and he wants the user to have the chance to bypass that particular rule, before either continuing with the checkout or canceling it (but only on Mondays after a full moon, if the user's name starts with "W" =). I'm pretty sure WF supports this sort of thing, and it would be awesome to allow this change in a visual designer, but this is the point where I usually give up and decide it's still easier to code than to use WF.

First, i am NOT a WF expert by any means. I took a 2 hour class on it, and from the little I could tell, I would hate using it. It looks hard to configure, hard to refactor, hard to test. Maybe this is my ignorance, but even the instructor looked lost at times and he had been using it a while.

Hi Fregas - there is no configuration to it really; File/New Project is about as far as I had to go. What did experience?

Yes, it's not easy to test - but hopefully using it the way I did mitigates that.

I'd like to know more about why you feel this way..

Rob - I really dig the idea of a workflow that is easily editable. One of the largest nightmares when dealing with a stream of customers is how something small like changing the order of operation really tends to bog down development. I'll be purchasing the book. I understand the scenario of synch validation but have you done any research into asynch processing?

If you wanted to inject a call to a view into the work flow and await a response, how would you go about that?

Rob, good job getting a good start with WF. Wait until you really get comfortable with it, you'll find yourself wanting to make everything a Workflow. This is a great example of how it is applicable.

I think some of the commenters are missing the point you're. Your order processing workflow is just four steps in a sequential diagram. The order processing workflow for an electronics circuit fabricator might be a state machine with fifty possible states.

In another scenario, the only step in the workflow might be a single CodeActivity that does everything in a block of code. Basically the point is, Rob's app instantiates the workflow and fires it off, it makes customizing this particular piece of the application painfully simple.

The naysayers are also missing the laundry list of free benefits that WF gives you like automatic serialization of a running workflow. If a specific process is waiting for input, instead of sitting around looping until that input arrives, the workflow engine automatically serializes the process to a datastore until it comes in.

Also, the Workflow itself is serializable (by default it uses XAML, the same language of WPF). So if you want to make a change to the process, rather than redeploying your application, just change the workflow definition and have the engine reload it. Think of Workflow Foundation as a distilled Biztalk (in fact the next version of Biztalk will be using Workflow at its core).

The one thing I would do differently is make your order processor a separate service that runs outside of the ASP.NET engine, allowing for long running workflows. Take for example an ordering process that requires manual verification. If you get 100 of these to process, suddenly you're hosting 100 workflows within ASP.NET. Also, you might have problems recovering should IIS crash when hosting within IIS.

@Mike, @Rob,

First, the caveat. I haven't ever needed to use something as heavy as Workflow in our applications, but I have investigated it enough to know that I haven't needed it.

That out of the way, I think Mike has the best solution. Use the Windows Workflow in a seperate processing service, not inside the webapps code. The webapp really could just use a simple coordinator object (I usually call these workflows, but they really are just simple state machines) that deals with saving to DB and updating properties, validation, etc. At the end of that coordination you could even send a message off to the workflow service (though this is rarely needed as simple polling usually suffices).

I understand this is WAY out of scope for your project however.

My understanding of Workflow is that, like WCF and WPF, its really for standalone applications (rich clients/servers), and not well suited to Web Applications. Anyone else confirm or rebuke this?

Lastly, just to reaffirm what others are saying, I can personally attest to the importance of testing your workflow/statemachine in isolation from everything else. I find that the nastiest toughest to find bugs always end up in these areas and with out good SoC in your workflow, it can be... painful to trace.

Thanks,

-Lucas Goodwin

Thanks Lucas - definitely don't want this thing happening "inline" with the web app - I'm working up a new demo and as my understanding increases, the more i like this.

I need to kick it off in the web app, but I'll do it in a different way then I showed here - I'll try to keep the process 100% separate.

I think what would really be nice is to create a service that "scrapes" the DB and looks for the orders that have been submitted, and then tosses them into the WF runtime bin.

That's the way to handle this (which is what you guys are saying as well) and I completely agree. The problem is that I need this thing to work with as little setup as possible - I think it will work out just fine.

I'm a tad worried about the "GoDaddy" factor and spinning up a separate thread though.

Rob, since you're going into new technology and work with experts on this technology, can you also cover when it is a good idea to use WF and when is it not (an overkill that would complicate things)?

you've briefly touched on the subject. to me WF looks like a way to model any (or almost any?) algorithm. I think you can use it to replace your hard-coded logic in many places. it's probably a bad idea, or not (because of overhead, added complexity to init and launch workflow, etc.)? where're you planning to use it in your web app and why?

thank you.

@Rob,

Ahh, yes, the bain of all Software Engineers working in the web world: the "GoDaddy" factor ;) I know exactly what you mean.

When you aren't doing shared location hosting (or hosting yourself) alot of what's REALLY cool in .NET and IIS6+ just can't be used (custome HttpHandlers for non-standard file-types for instance).

And spinning off a thread from with-in IIS will just eat one of your already limited number of available request threads. Perhaps seperating the processor (workflow) and ecommerce site into seperate sites, the later being the website, the former being a webservice would provide the widest range of application hosting options? this would allow someone to setup the workflow hosting site with it's own app-pool or even on a seperate server as well...

Essentially the webservice call would be an Async notification...Just a thought. With a host like discountasp.net you can setup seperate "maintenance" services to run on a schedule, which is how I'd normally do this.

Addendum:

I concur with Slava, it'd be really valuable to hear some guidance on WF from the experts in a real application like you did with IoC, Mocking, and TDD with Ayende.

Rob,

Is it an idea to let wf determine the flow of pages? For instance you are in the new action of a certain controller. When you're done with inserting the new object to the store, you ask the workflow which view the controller should go to. Go back to the Index page or go to a page with a message indicating successful adding to the store. Or something like a page wizard where you have created the step sequence in wf.

Nice work Rob, but I think that you skipped important parts of this tutorial, at least for us who don't know how to create workflow and wire it up with code.

I think you should put in video the whole creation of workflow, and all those stuff that made it work, how it's depending on your custom classes (Order, ...), wiring basically.

I would really like to see step-by-step creation of it. Now I still don't know how to create one :( Something like you did with StructureMap, which was great show.

@Ronnie Hoogland

<<Is it an idea to let wf determine the flow of pages? For instance you are in the new action of a certain controller. When you're done with inserting the new object to the store, you ask the workflow which view the controller should go to. Go back to the Index page or go to a page with a message indicating successful adding to the store. Or something like a page wizard where you have created the step sequence in wf>>

That sounds similar to what the patterns and practices team have come up with as an extension to the Web Client Soft ware factory <a href="http://www.codeplex.com/websf/Wiki/View.aspx?title=Page%20Flow%20Application%20Block">Page Flow Application Block</a>