SQL Server 2005 Service Broker - Activation Stored Procedure Memory Usuage
Nov 9, 2007
We are looking for some guidance with an issue we have picked up with our implementation of Service Broker here on the ABSA Capital project and I am hoping you can help or point us in the direction of someone.
The architecture we have implemented for service broker is to make use of an Activation stored procedure on two queues (1 SP per queue) to process the messages received. What we have found is that the activation stored procedure runs on a background session and its CPU time and memory just grows to the point where it brought one of our UAT servers to a grinding halt.
Is there anyway we can reduce the memory consumption of the activation stored procedure or is this one of those things that still need to be ironed out in Service Broker?
I use Try ... catch blok in my activation stored procedure. When SQL Server raise error (e.g. Primary key violation) in Try blok, XACT_STATE in Catch blok has value 1 = commitable transaction and I can use rollback transaction to savepoint. But when I use Raiserror() in Try blok, XACT_STATE in Catch blok has value -1 = uncommitable transaction and I can't use rollback transaction to savepoint. When I drop automatic activation for given queue and I run this stored procedure with Raiserror(), XACT_STATE has value 1 = commitable transaction.
What a problem may cause this different behavior ?
I'm having problems with activation. I have a CLR stored procedure that runs fine when run directly (and consumes messages from the queue). When I try to enable activation nothing happens. I've tried different execute as options and looked in the sql server logs and I don't see anything indicating why.
I've checked sys.service_queues and all options look correct including activation options. The sys.dm_broker_activated_tasks view is empty.
Without any errors I'm having difficulty tracking down the problem. I've seen references to service broker shutting down the procedure if it doesn't consume any messages but I haven't seen where this is indicated.
I'm a Chinese fan of Microsoft SQL Server 2005. Through studying for MS SQL Server 2005, now I have a question about €˜internal activation of service broker€™.
At the moment I have done a test. The description of test is as following:
Firstly I set the status of the activation to be off (ALTER QUEUE [dbo].[TargetQueue] WITH STATUS = ON , RETENTION = OFF , ACTIVATION ( STATUS = OFF , PROCEDURE_NAME = [dbo].[Usp_HelloWorld] , MAX_QUEUE_READERS = 5 , EXECUTE AS N'dbo' )). And filled the queue with100000 Messages. Then that running another application caused the usage of the CPU achieve 100%. Secondly reset the status of the activation to be on and monitored the €˜sys.dm_broker_activated_tasks€™. The instance of the stored procedure got the maximum number 5 very quickly. The server was much slower at current.
The SQL Server 2005 Books Online says: €™an activated stored procedure typically processes one or more messages and returns a response to the service that originated the message or messages. When messages arrive faster than the stored procedure processes messages, Service Broker starts another instance of the stored procedure, up to the maximum number defined by the queue.€? But it doesn€™t mention the performance counter of the server, for example: If the usage of the CPU is very high, the queue readers should be as few as possible, even don't process the message of queue until the system is idle. Because a client choice service broker means he don€™t care immediately process message, contrarily he care the throughput of the system and don€™t bother the natural application.
So my question is whether the strategy of internal activation of service broker doesn€™t care the performance counter of the server, just care the speed of process message. And the priority of queue monitor in SQL Server€˜s internal processes. Unfortunately I can€™t find any information about these from books online and Internet.
I am trying to find a solution to my woes creating CLR activation stored procedure that references Enterprise library data access dlls. Does anyone know best practices to deploy .NET 2.0 assemblies as SQLCLR. I am using VS2005 SqlClrProject to deploy my .NET 2.0 assemblies to Sql Server and it fails with error such as "Assembly was not found in the sql catalog". If I manually try to load a referenced assembly within SqlServer by browsing to the target location, like for example, System.Management.dll it fails with similar error, trying to resolve its dependencies. Even if I manage to walk through the dependency tree and resolve it to the final dependent dll, I am only able to load it in "UNSAFE" mode. Any recommendations, suggestions and feedback are welcome.
i am having a problem with service broker. here is what i have
i have two databases (DB1 and DB2) and both are on seprate servers (seprate instances) in DB1 i am putting raw data from plc using rssql. once data gets into the table, i am executing a trigger which bind the data and send it to the target service. in the target service i have a activated stored procedure which unbind data and then call a stored procedure which is on in DB2( i am using Linked Server ). on DB2 i am doing really really complex calculations thats why i have to use service broker to seprate the calculation to data input for performance.
i ran this query on both databases so i won't have to create the certificate. (ALTER DATABASE MyDatabaseName SET TRUSTWORTHY ON)
now my problem is, its not executing the stored procedure from DB2 and disabling the service everytime i enter the data into rawdata table(DB1). But if i run the stored procedure manuly from TSQL it works fine.
I have a stored procedure (a) which calls another stored procedure (b).
Stored procedure (b) is a C# stored procedure which simply writes out to a file data in XML format. Internally, it calls...select fld1, fld2, fld3, fld4, fld5from #tmptable for xml auto, elements
If I call stored procedure (a) from Query Analyser / SQL Management Studio everything works fine. Perfect.
But....we need this all to run asynchronously. So we used the Service Broker, configured the queues and messages and off we went. All worked as planned except our XML files were empty.
Further investigation showed that if we call select fld1, fld2, fld3, fld4, fld5from #tmptable
- without the 'xml' bits, we got a resultset back. But if we call it with the for xml auto, elements, the reader was empty. No errors are visible in the profiler, but the XmlReader refuses to read.
The binary / extended stored procedure is the same pysical binary that is called from Query analyser that works, but via the Service Broker refuses to do anything XML based. Outputting the data as normal text is cool, but not what we want.
----------------- UPDATE -------------- I changed the code so the CLR Stored proc was fired as a trigger on an update to a table. If I update the table in Query analyser, the trigger fires, the CLR Stored proc is called, the XML is generated.
If I update the table as part of my message handling in the Service Broker queue, the trigger is fired, the CLR Stored proc is called, the XML is generated EMPTY!!! The other TSQL statements work fine, but selecting for xml simply will not work for a procedure called, either implicitly or explicitly from the service broker.
I have implememted service broker in my application.My sqlservr.exe comsumes more than 2 gb of memory which eventually cause my computer to hang.I unable to track the problem. Can anyone help me out of this. Looking at the SQL Server logs i found following error message logged multiple times ;
The query notification dialog on conversation handle '{09B96045-99A4-DC11-802F-0013EFF142EF}.' closed due to the following error: '<?xml version="1.0"?><Error xmlns="http://schemas.microsoft.com/SQL/ServiceBroker/Error"><Code>-8470</Code><Description>Remote service has been dropped.</Description></Error>'. -Aravind cheziyan
I've been running SQL2000 for the past few years with 2gb of RAM and just recently upgraded to SQL2005 with 4 gigs.
My old system would always have the sqlservr.exe process using as much RAM as it could, this number would never really drop, maybe if it did it was in the single digit percentages.
I was just setting up new maintenance plans on my server, and I noticed from the time I started to the time I finished the SQLSERVR.EXE process dropped from using 3.5gb of RAM to just 1.5GB of RAM right now. It's pretty steadily going up too.
Does sql2005 have some new advanced memory management features? Did creating a sql maintenance plan release some memory? Did sqlserver crash ? I've never seen this before.
I have installed sql server standard edition 2005.I want to enable service broker feature or services after intalling the sql server.Please let me known any tool or utiltiy to enable the feature
Alter datatabase set enable_broker is not working.
I created the table dbo.customer from the orginal table sales.customer. dbo.customer works but sales.customer does not. WHY? We took the time to design our tables under different schemas to help control, organize and assign permissions.Private Function GetCustomers() As DataTable Dim sqlconnection As New SqlConnection(WebConfigurationManager.ConnectionStrings("connectionstringAlliance").ConnectionString.ToString)WORKS:Dim command As New SqlCommand("SELECT CustomerID, CustomerType FROM dbo.Customer WHERE CustomerID=1234", sqlconnection)DOES NOT WORK:Dim command As New SqlCommand("SELECT CustomerID, CustomerType FROM Sales.Customer WHERE CustomerID=1234", sqlconnection)
Dim dependency As New SqlCacheDependency(command) Dim dad As SqlDataAdapter = New SqlDataAdapter(command) Dim customers As New DataTable dad.Fill(customers)Cache.Insert("Customers", customers, dependency) Return customersEnd Function Protected Sub Page_Load(ByVal sender As Object, ByVal e As System.EventArgs) Handles Me.LoadDim customers As DataTable = CType(Cache.[Get]("Customers"), DataTable)
If customers Is Nothing Then customers = GetCustomers() Label1.Text = System.DateTime.Now.ToString() End If gridviewCustomer.DataSource = customers.DefaultView gridviewCustomer.DataBind()
Hello, I receive this error "The SQL Server Service Broker for the current database is not enabled, and as a result query notifications are not supported. Please enable the Service Broker for this database if you wish to use notifications." I attach the database in Management Studio to query and enable the broker using the scrip below but to no avail. ALTER DATABASE DataName SET ENABLE_BROKER ‘''<<------successfulandSELECT is_broker_enabled FROM sys.databases WHERE name = 'Database name' ‘'''<<-------value is 1 Global.asax ... Sub Application_Start(ByVal sender As Object, ByVal e As EventArgs) System.Data.SqlClient.SqlDependency.Start(ConfigurationManager.ConnectionStrings("dataConnectionString1").ConnectionString) End Sub...Web.config ... <connectionStrings> <add name="dataConnectionString1" connectionString="Data Source=.SQLEXPRESS;AttachDbFilename=|DataDirectory|jbp_data.mdf;Integrated Security=True;User Instance=True" providerName="System.Data.SqlClient" /> <add name="ASPNETDBConnectionString" connectionString="Data Source=.SQLEXPRESS;AttachDbFilename=|DataDirectory|ASPNETDB.MDF;Integrated Security=True;User Instance=True" providerName="System.Data.SqlClient" /> </connectionStrings>... Hope you could help. cheers,imperialx
How to prevent the hang on the initator service broker if the target service broker is not started?
Our case has two service brokers (two databases), sometime, the target is need to turn off. But the sitation is the initator service broker (in fact, the message is sent from triggers) become hang, I want to prevent this case and continue to operation, and the messages should queue and will continue to send to target service broker when it startup. How should I do?
Back in July I released the (then) latest version of SsbAdmin. For you who don'r know what it is; it's a tool which allows you to graphically administer SQL Server Service Broker.
I have now uploaded a new version which is compiled against the released versions of SQL Server 2005 and .NET 2.0.
You can download it from here: [0].
After download, un-zip to some directory and read the README.doc file.
"Could not impersonate the client during assembly file operation."
The CLR function is invoked from Service Broker internal activation stored procedure.
"SELECT user_name()" returns dbo just before CREATE ASSEMBLY execution.
SqlContext.WindowsIdentity.Name is "NT AUTHORITYSYSTEM" as the Data Engine runs with the LocalSystem account.
How do I create a the necessary security context for "CREATE ASSEMBLY" to succeed ?
Service Broker Queue activation with EXECUTE AS = "SELF", "OWNER", domain account or dbo, all result in the above error. The Service Broker assembly having the internal activation stored procedure is registered "unsafe".
I need to create a stored procedure (SQL 2005). I did it in C# (Visual Studio). The .dll file (inside) uses web services of Project Server 2007 (I just need to log on, create a project and log out). I prepared 2 files (exactly the same code): dll an exe. the .exe works fine. the problem is with .dll file (stored procedure in SQL)...
by the way: I created serialization file with sgen tool.
the problems start when I am trying to register these 2 .dlls in SQL server. I can do that in unsafe mode only. when I am trying to register them in externall access (normal dll) and safe mode (serialization .dll) I am getting errors:
CREATE ASSEMBLY failed because method "add_QueueCheckInProjectCompleted" on type "ConsoleApplication11.ProjectWebSvc.Project" in external_access assembly "ConsoleApplication11" has a synchronized attribute. Explicit synchronization is not allowed in external_access assemblies
thus I did it in unsafe mode (both of them). than I created the procedure.
Now, I tried to call that procedure. I am getting error:
A .NET Framework error occurred during execution of user defined routine or aggregate 'Main':
System.Net.WebException: The request failed with the error message:
<h2>Object moved to <a href="http://my_comp:22278/projectserver/_layouts/1033/error.aspx?ErrorText=Object%20reference%20not%20set%20to%20an%20instance%20of%20an%20object%2E">here</a>.</h2>
</body></html>
--.
System.Net.WebException:
at System.Web.Services.Protocols.SoapHttpClientProtocol.ReadResponse(SoapClientMessage message, WebResponse response, Stream responseStream, Boolean asyncCall)
at System.Web.Services.Protocols.SoapHttpClientProtocol.Invoke(String methodName, Object[] parameters)
at ConsoleApplication11.ProjectWebSvc.Project.QueueCreateProject(Guid jobUid, ProjectDataSet dataset, Boolean validateOnly)
at ConsoleApplication11.Program.Main()
That means "Object reference not set to an instance of an object". the problem occurs when calling QueueCreateProject method
I dont understand that error. I wrote the same code for .dll and .exe. the .exe file is working ok (no errors) but .dll (as a stored procedure) shows this error
my code in C#:
using System;
using System.Collections.Generic;
using System.Text;
using System.Data.SqlClient;
using System.Net;
using PSLibrary = Microsoft.Office.Project.Server.Library;
We have a number of large binary xml messages. I would like to compress the XML before converting to VARBINARY as much as possible. What is the recommended way to handle this if the conversions are happening in TSQL? I will also need to be able to decompress in .NET code, so I'm wondering if the best way is to just use CLR Integration for the marshaling routines, rather than building the marshaled structure in SQL. Any suggestions on reducing the XML payload size for extremely large messages?
IF (@message_type = 'http://schemas.microsoft.com/SQL/ServiceBroker/EndDialog') BEGIN END CONVERSATION @handle; RETURN 0 END ......................... ......................... After this I do process the message and some other processing
And then
END CONVERSATION @handle;
Note I do have single conversation group
Is their a problem in the way I am receiving and processing messages. Is it possible because of the delay between RECEIVE and END CONVERSATION same message is read by two different threads.
I have around 100 packages (all [packages run at same time) each package calls a stored Procedure once the Stored proc Execution is completed the package will write the log information into a log table, here how can i capture the CPU and Memory usage for execution of each stored proc.
I am struggling with the position SSB could take in an SOA. If I would want a broker in the general sense, meaning an intermediary sitting between applications which exchange information through messaging, would SSB be a good candidate? I know Biztalk is probably the primary candidate, but in my scenario I would end up with Biztalk apps with empty orchestrations. Also, I think Biztalk is more expensive to manage. So I am looking for a lightweight broker for a simple SOA targeted at application interoperability, no fancy business processes in sight.
Can't believe no one has commented on this in here yet. If you're into SQL Server Service Broker (and you are if you are reading this forum), you must go over and take Rushi's Service Broker Challange.
Recently there has been questions on some of the newsgroups about examples for SQL Server Service Broker. So, yesterday on a flight back to England I crofted up three different SQL Server projects as examples: LocalSampleRemoteSampleServer1RemoteSampleServer2 All three examples are jus your very basic €œHello World€? example, but they show: LocalSample €“ communication between two databases on the same SQL server instance. MasterKeys in a database and the database being TrustWorthyRemoteSampleServerX €“ These two samples are meant to run on two different machines/instances and in the samples you set up both transport security and dialog security by using Certificates. Download the zip file from here [0], unzip and read the README.txt file, and €“ Have Fun!! [0] : http://staff.develop.com/nielsb/code/servicebrokerexamples.zip
I have a initiator and a target service broker peer.
Both are controlled by a C# unit test. The initiator uses the Microsoft.Samples.SqlServer class. The target service uses stored procedure activation.
Sending a message from the initiator to the target, saves the content of the message, along with its conversation handle in the target's database specific table.
The unit test needs - at a later time - to instruct the target to send a message back on the same conversation handle to the initiator service.
For this the C# unit test creates a Conversation off of the saved conversation handle:
Service client = new Service("cleintservicename", conn, tran);
Conversation dialog = null;
dialog = new Conversation(client, convHandle); Sending the message on this dialog generates an error "Message body: <Error xmlns="http://schemas.microsoft.com/SQL/ServiceBroker/Error"><Code>-8495</Code><Description>The conversation has already been acknowledged by another instance of this service.</Description></Error>". Is the error due to the fact that a service - using the activated stored procedure already picked up the conversation, so that a new reference to the service can not be created through the Service class in CLR? If so, I might need then to skip the activated stored procedure in favor or a CLR service, alltogether? Any help - greatly appreciated.
I have configured a non-SQL linked server (via an OLE DB provider) and I wish to insert data into it via Service Broker but I am getting the following error in the SQL Server log: The activated proc [dbo].[sp_ mytableServiceProgram] running on queue TestDB.dbo.mytableQueue output the following: 'Cannot promote the transaction to a distributed transaction because there is an active save point in this transaction.'
As you see below, my strored proc. is not issuing any 'save trans' statements, so why is it not allowing me to wrap my code in a transaction? How else can I use a transaction (in order to not lose anything from the queue) and yet still be able to insert to the linked server?
CREATE PROC sp_mytableServiceProgram AS SET NOCOUNT ON;
-- This procedure continues to process messages in the queue until the -- queue is empty.
WHILE (1 = 1) BEGIN BEGIN TRANSACTION; --BEGIN DISTRIBUTED TRANSACTION; --Tried this but didn't help.
-- Receive the next available message WAITFOR ( RECEIVE TOP(1) -- just handle one message at a time @MessageTypeName = message_type_name, @MessageBody = message_body, @Dialog = conversation_handle FROM mytableQueue ), TIMEOUT 2000 ;
-- If RECEIVE did not return a message, roll back the transaction -- and break out of the while loop, exiting the procedure. IF (@@ROWCOUNT = 0) BEGIN ROLLBACK TRANSACTION; BREAK; END ;
SET @XML = CAST(@MessageBody AS XML);
INSERT INTO LINKEDSERVER.dbname.user.mytable SELECT tbl.rows.value('@doc_no', 'INT') AS doc_no, tbl.rows.value('@queryid', 'NVARCHAR(50)') AS queryid, tbl.rows.value('@ar_num', 'NVARCHAR(50)') AS ar_num, tbl.rows.value('@status', 'NVARCHAR(20)') AS status, tbl.rows.value('@creationtime', 'DATETIME') AS creationtime, tbl.rows.value('@note', 'NVARCHAR(250)') AS note, tbl.rows.value('@posted', 'NCHAR(1)') AS posted, tbl.rows.value('@kms', 'INT') AS kms, tbl.rows.value('@schresid', 'NVARCHAR(50)') AS schresid, tbl.rows.value('@resolution_code', 'NCHAR(8)') AS resolution_code, tbl.rows.value('@page_count', 'INT') AS page_count, tbl.rows.value('@new_serial_number', 'NVARCHAR(20)') AS new_serial_number, tbl.rows.value('@taskresolution', 'NVARCHAR(250)') AS taskresolution FROM @XML.nodes('/inserted') tbl(rows);
-- If the INSERT did not insert any rows, rollback. IF @@ROWCOUNT = 0 BEGIN ROLLBACK TRANSACTION; BREAK; END COMMIT TRANSACTION; END GO
I have an application that is reading a message from a Service Broker Queue. When I use integrated security with an NT account it works fine. When I use a SQL Server User through Management Studio I can select from the Queue however, when I use this same account through the web app with the SQL Server User, I cannot see the Queue. Is there a grant that I must do to this account to get it to see the Service Broker Objects?
I am writting a huge Service Broker Application. I notice that each message that is processed appears in the SQL Server Logs. Currently I am just testing with a single queue and tons of messages are showing up in the log. Is there a way that I can turn these informational messages off? I think in the end I may have something like 15 or 20 queues. These messages are very useful for debugging but I may not desire this extensive logging to be running all the time on all of my queues.
