I've an asp.net application that generates tons of messages (a few hundreds/Second, depending on load of requests). Basically, I need to consume those messages (doesn't have to call "message" even) into sql server table. Each message being a new row in a table.
I tried a few things but failed to satisfy the performance. I tried to put the messages in MSMQ and then have either the MSMQ trigger to process the message or Windows C# service to process message. The performance is horrible.
I'm thinking to send messages directly to SSBQueue and have sql server process it. Is there any async way of doing this so the performance would not be a problem? Anyway I can use WCF + SSB to solve this problem?
Can anybody suggest better approach? Buying a decent hardware or SQL Server 2005 is not a problem.
Rushi, if you are reading this post, sorry for the repeatation, I know I emailed you a few hours ago...
I'm in the process of doing the initial research for the architecture of a large scale, transactional messages routing platform.
My initial design called for a series of MSMQ queues and Windows Services, written in C#, to process the messages in these queues. There will be incoming and outgoing queues, queues to store unroutable messages, etc.
My application will be routing many hundreds of thousands (and eventually millions) of messages per day. These message are very small (< 200 bytes each) and must be routed very quickly. (<1 second processing overhead per message for high priority messages.)
Using the term "routing" may be a bit misleading. The messages arrive via TCP socket connections. I will just need to take in a message, examine its intended destination, and send it to one of several outgoing socket connections, likely on different machines. Some messages require higher priority routing than others, but I don't need multi-hop routing or anything like that.
Of great concern to me is that there are absolutely no single points of failure in the system. Because of this I was considering using a combination of MSMQ and Windows Services in a Clustered environment.
Can the Service Broker provide me with this kind of functionality? If so, how well does it perform and scale? Is it a better choice for messaging applications that require high transactional throughput than MSMQ?
I'm just trying to get an idea of what products/services I should look into further.
I've been trying different things to "READ" the recordset from the "Message Queue". I can read it but with some weird characters. I've tried
ActiveXMessageFormatter
BinaryMessageFormatter
XMLMessageFormatter
So, far I have no luck.
MessageQueue msgQ3 = new MessageQueue("SERVERNM\" + msg.Label, false);
Message msg3 = new Message();
msg3.Formatter = new ActiveXMessageFormatter(); msg3 = msgQ3.Receive(new TimeSpan(0, 0, 30));
byte[] b = new byte[msg3.BodyStream.Length]; msg3.BodyStream.Read(b, 0, (int)msg3.BodyStream.Length); System.Text.ASCIIEncoding enc = new System.Text.ASCIIEncoding();
returnVal = enc.GetString(b);
RESULTS: I try to convert it to String and then deserialize later but I'm getting some junk. <?xml version="1.0" encoding="utf-8" ?>
I want to Export a List of the Public Queues using the command prompt. Can this be done. I don't want to always click the Export List button. I eventually want to automate the Export List so I can create an Environment HealthCheck.
Used to work fine: (windows 200 server, sql server 2000, JDBC driver SP2.) Two java processes, one sending messages (using jdbc and sql as temp storage), second is receiving and placing on msmq.
After installing SP3a for SQL server, under high load (200 a sec) there are missing messages in msmq. Installed everything from scratch on separate machine, works fine until SP3a is applied. Any help is highly appreciated
Firstly the disclaimer: im a total SSIS bunny. Ok, with that out of the way... What i want to do is have data read from an MSMQ message (which will be a simple XML message) and have it transformed and inserted into a simple table. Sound simple, but i cant work out how to do it.
I started a new Integration Services Project and dropped a Data Flow task on the Control flow tab, then double-click it to go to the data flow tab. I have added a MSMQ connection manager which points at my private queue and i can hit the test button and it sees it tests ok. How do i wire that into my data flow?
On my dataflow tab i have a SQL Server Destination (set to the local SQl server instance) and ive been trying a DataReader source and try setting the Advanced Editor "Connection Manager" column on IDBConnection row to point to my MSMQ connection manager. I get some error "cannot aquire a managed connection from the run-time connection manager". How do i get a Data Flow source which reads the XML message off my MSMQ connection manager??
On the Control Flow tab you can add a Message Queue Task and i can bind it to my MSMQ connection manager and set it up as a recieve task etc, but how do i use that?
I'm working on the design of a system that will have two queues. The first queue will take requests to do something. The second queue will hold the results of having done something. Both queues won't necessarily be on the same physical server. Clients reading/writing to the queue will not be using SQL server. Web service/WCF interfaces will be developed to interrogate/manipulate queues.
My question is: which is better - MSMQ or Service Broker; if I want reliable queues. I can affort to loose items off the first queue, but cannot afford to loose any result data on the second queue.
I have an SSIS package which writes to a configured public MSMQ. When run directly through Visual Studio, file system or msdb it works fine (I guess these use me as the user). However, when I schedule it through an SQL Agent Job it fails with the following:
Failed to write message ... to the queue.
This seems like a permissions thing to me, but I've tried giving full control to everyone in the MSMQ permissions, also explicitly giving full control to the user that the agent job executes as. Neither made a difference.
Any help would be greatly appreciated. Cheers, Tony Vaughan.
I had a question thrown at me today that I am unable to find an answer to.
Can MSMQ 2.0 and SQL Server 2000 co-habitate on the same server? That is, can they both be installed on the same server and run along side each other with a big smile on their face?
My instinct is that, "yes", they can because they are both Microsoft products, but I have not been able to find anything that confirms this. Lots of information on the web about the two being used together, but nothing about them being on the same box.
Would someone be able to confirm whether they can/cannot run side by side on the same server? If you know of a website I can look at please let me know that also.
Thanks for the help! I greatly appreciate it! -- Shava
