Exactly-Once-On-Order - Conversation Group Question
Sep 30, 2007
Hi,
A SSB newbie question:
Looking in a various resources, i see that a conversation group guarantees receiving messages EOID, which means that each message will be received only once, and that messages will be received in the order they were sent.
However, the documentation explicitly states that only dialogs guarantee messages order. So, I€™m a little bit confused, what's the "In-Order" stand for in "Exactly Once In Order"?
I have not been successfull in getting conversation group to work. My understanding is that I can specify a 'guid' for a conversation group id in the create dialog and when I send a message on this conversation it will have that specific guid for its conversation group. When I do this it does not appear this way in the "target" queue.
I am looking for an example to help me understand how to use a conversation group. The MSDN has not really provided that run able example that I can run and verify and tweak.
The idea that I would like to try is that the initiator must send 5 different XML messages to a target. These 5 messages are all related and must exist together. What I assume is that if I want the target to get all 5 messages together out of the queue all messages must be sent in their own conversation but all linked with the same conversation group Id. I have not been able to get this to work.
The communication is really a one way where the initiator sends the data to the target and does not process or need a message sent back from the target.
Is there any way to ensure that messages sent on different dialogs have the same conversation group id on the target queue? I was attempting to set the conversation group id on the dialog before sending but learned that this only works on the initator end.
I have messages that could be sent from different applications (and at slightly different times) that need to processed exclusively (i.e. have the same conversation group id).
I'm trying to use Service Broker to relate a set of messages together and was trying to use a related conversation group id. From what I can gather (looking at other threads) I can't use this.....
Basically, my ideas was.... I have several tables being updated within a database transaction. These tables will have triggers associated with them which send a message to a SB queue detailing the table that has been affected and the key information.
After the database transaction commits, I wanted to retrieve the group of messages in order to identify exactly what happened to the database during the transaction (for business reasons). I don't need necessarily need them in the same order, but do need them grouped by database transaction.
Service Broker seemed to be ideal i.e. the messages wouldn't commit if the database transaction rolled back and I wouldn't be able to access them until the entire transaction was committed........ My only problem is that I don't seem to be able to associate them with each other!!!!
Can anyone help with a way I can do this with Service Broker, or am I just trying to use the wrong technology???
I'm using service broker and keep getting errors in the log even though everythig is working as expected
SQL Server 2005 Two databases Two end points - 1 in each database Two stored procedures: SP1 is activated when a message enters the sending queue. it insert a new row in a table SP2 is activated when a response is sent from the receiving queue. it cleans up the sending queue.
I have a table with an update trigger In that trigger, if the updted row meets a certain condition a dialogue is created and a message is sent to the sending queue. I know that SP1 and SP2 are behaving properly because i get the expected result. Sp1 is inserteding the expected data in the table SP2 is cleaning up the sending queue.
In the Sql Server log however i'm getting errors on both of the stored procs. error #1 The activated proc <SP 1 Name> running on queue Applications.dbo.ffreceiverQueue output the following: 'The conversation handle is missing. Specify a conversation handle.'
error #2 The activated proc <SP 2 Name> running on queue ADAPT_APP.dbo.ffsenderQueue output the following: 'The conversation handle is missing. Specify a conversation handle.'
I would appreceiate anybody's help into why i'm getting this. have i set up the stored procs in correctly?
i can provide code of the stored procs if that helps.
I have a a grid (Fig-1) where i have LineID and corresponding RankValue. I want to sort out the Grid like (Fig-2) where It will be sorted based on Rank Value(Higher to lower) but LineID group should maintain. I am using SqlServer 2008.
I have the following statement that is working just fine:
SQL = "select* from products GROUP BY name ORDER BY price"
I have many products with the same name but with different prices and I want to display unique records along with other information but I want to display the cheapest product first. It is displaying the first record in the table with that name but not the cheapest distinct record in the list of all distinct product names. Not sure of the above makes sense so this is the kind of data in the table.
My service broker was working perfectly fine earlier. As I was testing...I recreated the whole service broker once again.
Now I am able to get the message at the server end from intiator. When trying to send message from my server to the intiator it gives this error in sql profiler.
broker:message undeliverable: This message could not be delivered because the Conversation ID cannot be associated with an active conversation. The message origin is: 'Transport'.
broker:message undeliverable This message could not be delivered because the 'receive sequenced message' action cannot be performed in the 'ERROR' state.
Hello, I'm trying to find the most optimal way to perform a trickyquery. I'm hoping this is some sort of standard problem that has beensolved before, but I'm not finding anything too useful so far. I havea solution that works (using subqueries), but is pretty slow.Assume I have two tables:[Item]ItemID int (Primary Key)ItemSourceID intItemUniversalKey uniqueidentifierPrice int[Source]ItemSourceIDPriorityI'm looking for a set of ItemIDs that match a query to the Price(something like Price < 30), with a unique ItemUniversalKey, taking thefirst item with each key according to Source.Priority.So, given Item rows like this:1 2 [key_one] 152 2 [key_two] 253 1 [key_one] 15and Source rows like this:1 12 2I want results like this:2 2 [key_two] 253 1 [key_one] 15Row 1 in Item would be eliminated because it shares an ItemUniversalKeywith row 3, and row 3's Source.Priority is lower than row 1.Help!?
CREATE TABLE #Turnover ( location varchar(50), Total int )
insert into #Turnover (location,Total) values('A', 500) insert into #Turnover (location,Total) values('AB', 200) insert into #Turnover (location,Total) values('ABC', 100) insert into #Turnover (location,Total) values('BA', 100) insert into #Turnover (location,Total) values('BAC', 500) insert into #Turnover (location,Total) values('BAM', 100)
Now i want output order by total but same time i want to create two groups. i.e. location starting with A and order by total and after locations starting with B and order by total.
We have implemented our service broker architecture using conversation handle reuse per MS/Remus's recommendations. We have all of the sudden started receiving the conversation handle not found errors in the sql log every hour or so (which makes perfect sense considering the dialog timer is set for 1 hour). My question is...is this expected behavior when you have employed conversation recycling? Should you expect to see these messages pop up every hour, but the logic in the queuing proc says to retry after deleting from your conversation handle table so the messages is enqueued as expected?
Second question...i think i know why we were not receiving these errors before and wanted to confirm this theory as well. In the queuing proc I was not initializing the variable @Counter to 0 so when it came down to the retry logic it could not add 1 to null so was never entering that part of the code...I am guessing with this set up it would actually output the error to the application calling the queueing proc and NOT into the SQL error logs...is this a correct assumption?
I have attached an example of one of the queuing procs below:
Code Block DECLARE @conversationHandle UNIQUEIDENTIFIER, @err int, @counter int, @DialogTimeOut int, @Message nvarchar(max), @SendType int, @ConversationID uniqueidentifier select @Counter = 0 -- THIS PART VERY IMPORTANT LOL :) select @DialogTimeOut = Value from dbo.tConfiguration with (nolock) where keyvalue = 'ConversationEndpoints' and subvalue = 'DeleteAfterSec' WHILE (1=1) BEGIN -- Lookup the current SPIDs handle SELECT @conversationHandle = [handle] FROM tConversationSPID with (nolock) WHERE spid = @@SPID and messagetype = 'TestQueueMsg'; IF @conversationHandle IS NULL BEGIN BEGIN DIALOG CONVERSATION @conversationHandle FROM SERVICE [InitiatorQueue_SER] TO SERVICE 'ReceiveTestQueue_SER' ON CONTRACT [TestQueueMsg_CON] WITH ENCRYPTION = OFF; BEGIN CONVERSATION TIMER ( @conversationHandle ) TIMEOUT = @DialogTimeOut -- insert the conversation in the association table INSERT INTO tConversationSPID ([spid], MessageType,[handle]) VALUES (@@SPID, 'TestQueueMsg', @conversationHandle);
SEND ON CONVERSATION @conversationHandle MESSAGE TYPE [TestQueueMsg] (@Message)
END ELSE IF @conversationHandle IS NOT NULL BEGIN SEND ON CONVERSATION @conversationHandle MESSAGE TYPE [TestQueueMsg] (@Message) END SELECT @err = @@ERROR; -- if succeeded, exit the loop now IF (@err = 0) BREAK; SELECT @counter = @counter + 1; IF @counter > 10 BEGIN -- Refer to http://msdn2.microsoft.com/en-us/library/ms164086.aspx for severity levels EXEC spLogMessageQueue 20002, 8, 'Failed to SEND on a conversation for more than 10 times. Error %i.' BREAK; END -- We tried on the said conversation, but failed -- remove the record from the association table, then -- let the loop try again DELETE FROM tConversationSPID WHERE [spid] = @@SPID; SELECT @conversationHandle = NULL; END;
Using SQL Server 2005 (9.0.3042), I can not get a full-text search query to work with order by or group by. For example, if I run the following query...
SELECT s.networkID, i.interID
FROM Interactions i
INNER JOIN CONTAINSTABLE(Interactions, text, 'ipo') t ON t.[key] = i.interID
INNER JOIN Sessions s ON i.sessionID = s.sessionID
WHERE i.startTime BETWEEN 1051772400000 AND 1054537199000
It returns two rows immediately.
If I add an "order by" or "group by" like...
SELECT s.networkID, i.interID
FROM Interactions i
INNER JOIN CONTAINSTABLE(Interactions, text, 'ipo') t ON t.[key] = i.interID
INNER JOIN Sessions s ON i.sessionID = s.sessionID
WHERE i.startTime BETWEEN 1051772400000 AND 1054537199000
ORDER BY s.networkID, i.interID
The CPU shoots to 100% and I have to kill the query. I have tried many variations of the above and I can not get the query to work with order by or group by. Is this a bug in SQL Server? Is there a fix or workaround?
In Outer join, I would like to add the outer columns that don't exist in the right table for each order number. So currently the columns that don't exist in the right table only appear once for the entire set. How can I go about adding PCity, PState to each order group, so that PCity and PState would be added as null rows to each group of orders?
if OBJECT_ID('tempdb..#left_table') is not null drop table #left_table; if OBJECT_ID('tempdb..#right_table') is not null drop table #right_table; create table #left_table
I have an SSRS 2012 table report with groups; each group is broken ie. one group for one page, and there are multiple groups in multiple pages.
'GroupName' column has multiple values - X,Y,Z,......
I need to group 'GroupName' with X,Y,Z,..... ie value X in page 1,value Y in page 2, value Z in page 3...
Now, I need to display another column (ABC) in this table report (outside the group column 'GroupName'); this outside column itself is another column header (not a group header) in the table (report) and it derives its name partly from the 'GroupName' Â values:
Example:
Value X for GroupName in page 1 will mean, in page 1, column Name of ABC column must be ABC-X Value Y for GroupName in page 2 will mean, in page 2, column Name of ABC column must be ABC-Y Value Z for GroupName in page 3 will mean, in page 3, column Name of ABC column must be ABC-Z
ie the column name of ABC (Clm ABC) Â must be dynamic as per the GroupName values (X,Y,Z....)
Page1:
GroupName          Clm ABC-X
X
Page2:
GroupName          Clm ABC-Y
Y
Page3:
GroupName          Clm ABC-Z
Z
I have been able to use First(ReportItems!GroupName.Value) in the Page Header to get GroupNames displayed in each page; I get X in page 1, Y in page 2, Z in page 3.....
However, when I use ReportItems (that refers to a group name) in the Report Body outside the group,
I get the following error:
Report item expressions can only refer to other report items within the same grouping scope or a containing grouping scope
I need to get the X, Y, Z ... in each page for the column ABC.
I have been able to use this - First(Fields!GroupName.Value); however, I get ABC-X, ABC-X, ABC-X in each of the pages for the ABC column, instead of ABC-X in page 1, ABC-Y in page 2, ABC-Z in page 3, ...
select DATENAME(WEEKDAY,[date]) as DAY, sum(earnings) as Earnings from myTable where DATENAME(WEEKDAY,[date] in ('monday','tuesday','wednesday','thursday','friday') group by DATENAME(WEEKDAY,[date]) order by DAY
this query works very well except the last line. the last line causes SQL-Server to order the days in an alphabetical order.What do I have to do that SQL-Server orders the days according to their natural appearance (monday, tuesday, wednesday, thurs......)
I have a system that will post a message to a queue, but does not need to wait for a response - just needs to make sure the message arrived properly in the queue, not that is was processed at the receiving end. A second service will poll the queue to retrieve outstanding messages and will then move the message to an outside system. The movement of the message to the outside system will be wrapped in a transaction and if the process is successful, then the transaction will be commited otherwise it will be rolled back.
1) is it appropriate for the service that posts the message to send an END CONVERSATION ? This way the sending service will not be waiting for a response.
2) in the data movement phase, is it appropriate to issue and END CONVERSATION when commiting and not issue when ROLLBACK occurs. Or should ROLLBACK occur with a following END CONVERSATION with error message?
I am attempting to learn Service Broker from Bob Beauchemin's book "A Developer's Guide to SQL Server" - Chapter 11. I'm finding it to be very good but I'm confused over the concept of closing a conversation. Could someone answer the following questions for me?
When a conversation is ended, can the conversation handle that was created when the conversation was created still be used? (I assume not) Beauchemin says, on page 511, that when a conversation ends, "Any messages still in the queue from the other end of the conversation are deleted with no warning." Does this mean that if I send a message that expects a reply, but I end the conversation, the message is still sent, it is still received by the other endpoint, the other endpoint processes it, but I'll never receive the reply? Beauchemin says that if no lifetime is specified, the conversation is active for the number of seconds which can be represented by the maximum size of an integer. Does this mean that if I don't specify a lifetime, a conversation is active for many, many years?
I want to reuse conversations to minimize overhead during bursts of activity. Remus' article on reusing conversations (http://blogs.msdn.com/remusrusanu/archive/2007/05/02/recycling-conversations.aspx) is great. (I know you are reading this Remus, thanks.)
I was wondering if there is a simpler way of ending a cached conversation - Quiesce the conversation (Stop using it), then after some period of time, end it.
I create a conversation, cache it in RLY_Conversations, and use it for 50 seconds. After 1 minute, the dialog timer servicing proc ends the conversation. There will be no messages sent around the time the End Conversation takes place, thus no race conditions.
Do you see any problems with this method?
Select @DialogHandle = [conversation_handle] From RLY_Conversations Where TableName = @TableName and IsActive = 1 And
CreatedTmstp > dateadd(ss, -50, getdate())
if @DialogHandle is null Begin -- initialize a conversation and record it in our reuse table BEGIN DIALOG CONVERSATION @DialogHandle FROM SERVICE FirstHostRelayService TO SERVICE 'SecondHostRelayService' ON CONTRACT RelayContractSentByAny WITH ENCRYPTION=OFF ;
-- cache the dialog handle to minimize dialog creation overhead. Insert into RLY_Conversations ( TableName, conversation_handle, conversation_id, is_initiator, service_contract_id, conversation_group_id, service_id, lifetime, state, state_desc, IsActive, CreatedBy, CreatedTmstp ) Select @TableName, conversation_handle, conversation_id, is_initiator, service_contract_id, conversation_group_id, service_id, lifetime, state, state_desc, 1, 'Setup', getdate() From sys.conversation_endpoints Where conversation_handle = @DialogHandle;
-- initiate housekeeping process BEGIN CONVERSATION TIMER ( @DialogHandle ) TIMEOUT = 60; End
when you move a conversation to a conversation group, that conversation_group has to have been created previously, ie, you cant specify a non-existing conversation_group, right?
I ask because I am trying to develop an application where I use optimally one conversation related to many given conversation_groups, so that when I receive, I lock only a small determined subset of messages. What I could have used was a way to send messages on a conversation, specifying a conversation_group_id.
Message ordering is of utmost importance in our application.
As i found in testing the only way to ensure message ordering is if they are in the same conversation.If you send multiple messages in different conversations there is no garantee which will be processed first.
Therefore i will be creating conversations that last "forever", that is using a single conversation.
I plan on doing a BEGIN DIALOG CONVERSATION when an inititator site is setup and writing the conversation handle guid to a table.
I will them simply SEND ON SONVERSATION using the guid, i will never issue a end conversation from target or initiator.
Is this theory solid, ie: is there a better way or best practice to do this?
I know that conversatons persist with sql server restarts, however what happens if an initiator site db is restored ?
I was thinking of adding logic to first check if a conversation endpoint exists with the specified guid if not , then start another conversation. But is this the best way?
I am thinking of updating my SQL monitoring application to use Service Broker.
Right now I loop through my list of servers performing various checks on each server. Things like 'check last database backup', 'check for new databases', 'check for server restart'. I loop through, one server at a time, doing one check at a time. The more servers I have the longer it is taking.
So, I want to multi-thread the servers, but single-thread the checks on each individual server. This way I can check say, 5 servers at a time, but on each server I will only do one check at a time. This way I won't flood an individual server with multiple checks.
Is this possible? It looks like Conversation groups might be the way to go but I'm not sure.
I'm having some troubles with conversation groups. I need to send two messages on the same conversation group so I have the following in my SP....
BEGIN DIALOG CONVERSATION @providerConversationHandle FROM SERVICE [ProviderDataService] TO SERVICE 'CalculatedDataService' ON CONTRACT [ProviderDataContract] WITH ENCRYPTION = OFF , LIFETIME = 600;
BEGIN DIALOG CONVERSATION @curveConversationHandle FROM SERVICE [ProviderDataService] TO SERVICE 'CalculatedDataService' ON CONTRACT [ProviderDataContract] WITH RELATED_CONVERSATION = @providerConversationHandle , ENCRYPTION = OFF , LIFETIME = 600;
SEND ON CONVERSATION @providerConversationHandle MESSAGE TYPE [ProviderDataMessage] ( @providerMessage );
SEND ON CONVERSATION @curveConversationHandle MESSAGE TYPE [ProviderCurveMessage] ( @curveMessage );
When I query the queue I see two messages, but they don't have the same conversation_group_id.
I have a conversation that I want to know has not ended so I am using LIFETIME.
When the conversation times out I then have three records in the queue.
1. The original conversation record that has not been received.
2. Error message to the initiator.
3. Error message to the target.
Both message bodies on the error records say that it was a lifetime error.
If I end the conversation on the initiator side after it is sent, I still get the target error record but the message_body field is null.
So say I don't end the conversation on the initator side. My next receive on the target side will pull the original record. Then it will pull the initator record and then it will pull the target record. Nothing on that record says that it had timed out.
What is the best practice for handling lifetime errors?
END CONVERSATION DIALOG CONVERT(uniqueidentifier, '58C1A7AA-C0D7-DB11-B4C6-005056C00008')
using ICommand.Execute method, I got an error:
Incorrect syntax near the keyword 'END'.
I use SQL Native Client as OLE DB Provider.
In profiler I can see that my statement becomes like this:
exec END CONVERSATION CONVERT(uniqueidentifier, '903586ED-C0D7-DB11-B4C6-005056C00008')
So provider add 'exec' before my statement, maybe because NCLI don't know such statetment and thinks that it is a stored procedure call with the name 'END'?
Is there is a way to avoid it except of using following construction:
I am new to service broker and would like a little help please. I have a SP which gathers information from a collection of tables. Depending on the data gathered it may or may not begin a dialog conversation with a service broker queue. What i'm needing to know is should at the end of the SP once the required message has been sent should i end the conversation or not?
Given that the conversation states are as follows: (Thanks Rushi!)
Event Initiator Endpoint state Target Endpoint state
BEGIN DIALOG SO -- from Initiator to Target
SEND message(s) CO -- from Initiator to Target
Target receives a fragment CO SI of the first message sent or receives out of order message
Target received entire CO CO first message
END conversation at CO DO target
Initiator receives EndDialog DI DO message from target
Target receives ACK for the DI CD EndDialog message sent
END conversation at CD CD Initiator
When does the 30 minute timer start for clearing the conversation from the sys.conversation_handles table? Is it the same for both sides (initiator and Target) ie, the end conversation at the Initiator. I guess it must be just in case a resend is necessary.
