Responding to Srinivas’ comment requesting more on debugging, I thought I’d post on one common question when using the service integration bus: where has my message gone? By this I mean the case when a message has been sent but not received by the consumer.
If using an explicit JMS receive to retrieve the message, the first thing to check is that the connection from which the message consumer’s session was created has been started. For some unknown reason, the designers of the JMS API decided that the start and stop methods should control delivery to both asynchronous and synchronous consumers. Even more unfortunate, is that they didn’t consider it to be an error to call receive on a consumer whose connection is currently stopped. If you are using the default messaging provider you will, however, see a warning output in the application server logs if this is the case.
The next place to look is at the resources defined on the bus. Has message receipt been disallowed on the target destination? Are there any mediations defined on destinations through which the message should have passed? If so, could they have consumed or redirected the message, is the mediation queue point started and is there at least on mediation handler application running for that destination? Again, looking at the destinations through which the message should pass, has an administered forward routing path redirected the message away from the consumer?
Another factor that may impede progress of a message is security. If bus security is enabled, did the sender have permission for all the destinations on the path to the target destination?
Finally, the message may have been deliberately discarded by the bus. Did the producer set too short a time to live on the message that would cause it to expire before reaching the consumer? Was the message sent with a reliability of best-effort non-persistent? If so, could shortage of memory have caused the message to be discarded. Are you using publish/subscribe messaging and, if so, was the consumer’s subscription active at the time the message was sent?
In an attempt to locate your missing message there are a few places to look. The first is the default exception destination for each messaging engine between producer and consumer. These are the ones prefixed with _SYSTEM.Exception.Destination and ended with the messaging engine name. Destinations may also be explicitly configured with their own exception destinations and these should be checked as well. A tool such as the Service Integration Bus Explorer is very useful at times like this for checking multiple queue points distributed across the bus (ensure that you have View > View System Objects checked).
One typical scenario where the exception destination is employed is if a message is received under a transaction which then rolls back. If the message redelivery count exceeds the maximum failed deliveries limit on the destination then the message will be moved to the exception destination. If this is happening then you need to determine why the transaction is being rolled back repeatedly. Is the receiving application, for example, trying to access some database that is not currently available?
If the message has to pass through multiple messaging engines, one of which is unavailable, then the message may be sat on a remote queue point waiting to be delivered to the messaging engine that is down. For those with a WebSphere MQ background, a remote queue point is like a transmission queue only it does not need to be explicitly defined. These remote queue points can be located on the runtime tab of the messaging engine panel in the administrative console.
One final tip for determining where messages are going is to set the following trace specification for application servers on the route between producer and consumer: SIBMessageTrace=all=enabled. This outputs an entry when each of the following messaging actions are performed:
- Message sent by a producer
- Message delivered to a consumer
- Message placed on a queue point
- Message committed
- Message rolled back
- Message placed on an exception destination
- Message discarded
- Message expires
- Message transmitted or received
- Message distributed to subscribers
- Message transformed/re-routed/modified by a mediation
- Message delivered to a mediation
- Message passed on by a mediation to the next destination
- Message discarded by a mediation
- Message processed by a mediation and new messages are generated by the mediation
this is a great blog. thanks.