Erin,
Thanks for the suggestions. An ideal solution would be to modify the
original orchestration so that it was resumable. Technically that could be
done. But it is a large complicated orchestration with many, many
potential
points of failure and the effort to make it resumable at this point is
more
than my client wants to spend. We have invested a fair amount of effort in
the past month or so identifying and eliminating root causes of failure in
an effort to reduce the failure rate. That has been successful, which of
course has reduced the business' appetite for making the orchestration
handle failures better. So we've chosen the approach of allowing the
orchestration to fail, but attempting to eliminate the huge mass of
suspended messages that currently results.
I've considered the possibility of going to one-way receives and sends.
The
problem I think I have with that is that a one-way HTTP Receive will send
an
HTTP 200 response when the message is received and that isn't necessarily
what I want to happen during this "wait and see what IT wants to do"
period.
The reason for the message to IT is that in some rare cases, we have
suspended orchestration instances that actually are resumable and in that
case I don't want the third party to have gotten a successful response to
their message. I will eventually be sending HTTP 200 responses to the
third
party messages, but only after IT has indicated that's what they want the
Responder to do.
I think the path I'll follow is what Dan Rosanova suggests in the
subsequent
reply - to add some code in the section of my Responder orchestration that
is waiting for the IT response. That code will listen for messages from my
third party and send them negative (maybe HTTP 5xx) responses. That will
keep IIS happy because I'm responding in a timely fa****on and will keep my
third party resending messages until IT has time to respond. I'm going to
give that a try.
Thanks again for your suggestions... and for taking the time to read
through
and understand my situation.
Zoe
"Erin Miller" <emiller@[EMAIL PROTECTED]
> wrote in message
news:usRCOO5uIHA.5892@[EMAIL PROTECTED]
> Hi Zoe,
> At the risk of stating the obvious, I'm just going to check one thing...
> You can't modify the original orchestration, can you? For instance, you
> mentioned that if something goes wrong, you have to go terminate the
> orchestration because it can't recover. So the easiest way to solve
that
> (it seems to me) is to add compensation to that original orchestration
so
> that you CAN recover...then you don't need to worry about all this other
> stuff.
>
> Okay, assuming you've already thought of that and it's not viable, it
> seems like there are a few possibilities. So right now, the Responder
> orchestration is getting dehydrated while you're waiting for someone in
IT
> to decide what to do, correct? And all those messages from System B are
> piling up, but when the orchestration re-hydrates, they all fail because
> the IIS timeout has occurred?
>
> Ultimately, because you have to wait for IT to intervene, you're doing a
> long-running transaction and you can't use the HTTP request/response
> paradigm. It just won't work--it will always have the problems that
> you're describing because the orchestration will always get dehydrated
> because you're waiting for a human to decide what to do, and that's
going
> to take longer than an HTTP request/response. The Responder ends up
just
> adding in more dehydration and suspended messages.
>
> Assuming I'm understanding all this correctly (and forgive me if I'm
> not!), it sounds like the goal of the Responder app is to figure out if
> the orchestration is suspended or not, and if it is suspended. to get
> System B to stop sending all those messages. A couple of thoughts:
>
> What about just subscribing to the suspended message queue? And then
> sending off the "stop sending to me" message when your app determines
that
> it should? I think that's what your goal was with the Responder
> orchestration, but because you introduced a delay, the HTTP
> request/response didn't work anymore.
>
> The other options, of course, are to use one-way sends/receives; your
> Responder will still dehydrate, but at least when you rehydrate, you
won't
> be dependent upon the same IIS session.
>
> I hope this helped...I'm not sure if I fully understood everything, or
if
> I just muddied the water further.
>
> Best,
> Erin
>
> Zoe Hart wrote:
>> As I said, it's complicated. I'll step back and try to give you the big
>> picture.
>> I have an orchestration that is managing a long (really long) running
>> transaction between two systems - call them A and B. Sometimes
something
>> goes wrong, an exception occurs, and the orchestration gets suspended.
>> When this happens, we typically have to terminate the orchestration
>> because it wasn't designed in a way that lets it recover on resume.
>> System A then continues to process the long running transaction and a
>> manual process is used to update System B. But System B, in response to
>> events triggered through the manual process, still attempts to send
>> messages to System A through the automated, BizTalk hosted process.
These
>> fail because there's no orchestration subscribed to them. When they
fail,
>> System B resends them... over and over again for two weeks. The result
is
>> a really large number of routing failures and suspended messages every
>> time an orchestration fails. When something happens that causes a large
>> number of orchestrations to fail, the number of suspended messages
aimed
>> at those suspended/terminated orchestrations becomes ridiculously
large.
>>
>> Enter my new Responder orchestration. When the main orchestration gets
an
>> exception it drops down into my exception handler. In the exception
>> handler I use a StartOrchestration shape to start an instance of the
>> Responder orchestration. I pass in enough information so that the
>> Responder knows what order it is responding to messages for.
>>
>> The first thing the Responder does is send a message to IT Sup****t. The
>> message contains the order number, which initializes a correlation set
>> used to receive subsequent messages from System B. The message gets
>> mapped to an HTML email that contains two links, each of which comes
back
>> to the BTSHttpReceive listener. The query string parameters basically
>> tell the Responder to do one of two things:
>>
>> 1) Terminate the original suspended orchestration and begin listening
for
>> and acknowledging all messages from System B for this order, or
>> 2) Do nothing - used primarily in rare cases where the original
>> orchestration was successfully resumed.
>>
>> So the Responder orchestration sends the message and waits for the
>> message back that tells it what to do. If the message back tells it to
>> take Action 1, it first terminates the original orchestration (the
>> orchestration InstanceId was passed in as a parameter). It then has a
>> Listen shape listening for any one of about six different messages from
>> System B and, for each message, it simply needs to send a simple
>> request-received acknowledgement message that keeps System B happy and
>> prevents System B from sending and resending the messages.
>>
>> The problem is that while the Responder orchestration has sent its
>> initial message and is waiting for the subsequent message that will
>> instruct it what action to take, System B is sending messages - either
>> messages in response to some event triggered through the manual
interface
>> or resending the message it was sending when the initial exception that
>> caused the main orchestration to fail occurred. The Responder
>> orchestration is not yet at the Listen shape that will receive these
>> messages but, for reasons I don't fully understand, the messages get
>> queued up for the Responder orchestration. If I right-click a
dehydrated
>> Responder orchestration in the Admin Console, one that I know is
awaiting
>> its "instructions" message, and Show Messages I can see the messages
from
>> System B - usually 1 to n resends of the same message queued up. Hours
>> can go by before IT Sup****t reads the mail message and clicks the link
>> that instructs the responder to terminate the original orchestration
and
>> respond to messages from System B. Now the Responder orchestration
moves
>> forward, terminates the original orchestration, and reaches the Listen
>> shape to listen for messages from System B. It takes the first queued
up
>> message and attempts to send an acknowledgement, but that Send fails
>> with:
>>
>
>> ...
>> Additional error information:
>> A batch item failed persistence Item-ID ... OperationType
>> MAIO_CommitBatch Status - 1061151998 ErrorInfo The published message
>> could not be routed because no subscribers were found.
>>
>> I believe this exception occurs if the orchestration attempts to send
the
>> response after the IIS process that received the original request has
>> timed out. The end result is that my Responder orchestration, which was
>> intended to clean up the mess that results when my main orchestration
>> suspends, now ends up also suspended so I've got one more thing to
clean
>> up.
>>
>> Now do you understand my issue? If not, don't worry about it. As I say,
>> it's complicated.
>>
>> Thanks for trying.
>> Zoe
>>
>> "Dan Rosanova" <dnova7@[EMAIL PROTECTED]
> wrote in message
>>
news:15e7fee4-f9c7-4b45-927f-662567477fac@[EMAIL PROTECTED]
>>> OK, now I'm more confused. So is receive from the HTTP request is not
>>> an Activating Receive, i.e. it's not the first shape in the
>>> orchestration? If not then you must be using some sort of correlation
>>> to receive this message? I'm affraid I don't understand your issue.
>>> Kind Regards,
>>> -Dan
>>
|
