How Xml Could Help Us With Our ETL Processes On Daily-basis?
May 26, 2006
Hi all of you,
I'm just a newbie with XML. Now we're moving all our DTS to SSIS packages. Most of them are simply processes that takes a plain file and loading these data into Sql table or in inverse way. So that I wonder what role will play XML here.
I mean, are there big differences between store data as XML format and nvarchar?
I have to import data into 2 tables on a daily basis. The data is provided as a flatfile. In order to fullfill this task the tables have to be truncated first. Are there any possibilities to do this job automatically with dts, or do I have to write an Interface in VB or something like that?
I have recently upgraded one of our servers using the in place upgrade method to SQL Server 2005 from SQL Server 2000.
We are currently having an issue with stored procedures as they appear to be running slowly and the only way to correct this issue is to recompile them on a daily basis.
Has any one else had a similar issue, any ideas what could be causing this to occur.
I'm creating 15 reports based on one data source. Once I created, I'm supposed to run this on daily and monthly basis. I'm going to use snapshot option in Report Manager. Is it fine? Do I need to create a data base too?
Also when I change the target URL address in the report property box, can I deploy it to any other server? Do I have to change the settings of report configuration manager too, inorder to publish reports on another server(not in my local machine)
I have a query which sends email notification of count of pending name and suggested name.. on a daily basis. So this works fine but now I want like if the pending and suggested are ' o count' I don't want the email notification. It should not send an email, if it is o count, but if we have the number for pending and suggested this email should be sent.
We are facing lot of problems with Blocking,can any one help us in this matter,The problem is as follows
We have SQL Server 7.0 running on Nt4.0, and three web servers and 5 application servers are accessing SQL server. Till Yesterday everything was fine,Suddenly today more than 18 processes were blocked by other(Like chain),First i killed some blocking process,then it was fine,once again it started and continuously some processes are blocked by other,and i found that all blocking process are running from webservers.I ran SQL Profiler to get some information,but no use. I am not understanding why suddenly it happend,because we have't modified anything.Is there any way to overcome this situation,this is production server. because of this users are getting slow responce/no responce.
----Here i want to know why it happend? ---How to trace the problem and fix it
How do I format a date value in SQL 2000 sp? Tired the following none are returning the quivilent rates. What is the syntax please? SELECT *FROM tblCompanyWHERE ( C_LastBuyDate = '03/21/2008') SELECT *FROM tblCompanyWHERE ( C_LastBuyDate = '2008/03/2008') SELECT *FROM tblCompanyWHERE ( C_LastBuyDate = '03-21-2008')
hello guys ! i need a little help in an update statement.
i have two tables as
table1(id1,name) table2(id2,date,val2)
what i want is to set val2=45 in table2 where date=4/4/2007 (say) and id2, now id is to be obtained from table1 based on the name;
like table1 --------- (1,"akmal") (2,"peter") (3,"kamal")
I know the name as peter, but i want to used its id1 (2) in the update statement. You might have got the idea that id2 is a foreign key referring to id1.
Hi All I am using following code for my application SELECT convert(varchar(10),acr._eventTime,120) [Date], sum(case r.Severity WHEN 'Warning' THEN 1 end)[Warning], isnull(sum(case r.Severity WHEN 'OK' THEN 1 end),0)[OK], FROM ActiveCheckIssueResult r GROUP BY convert(varchar(10),acr._eventTime,120)
it gives me Result as,
Date warning OK --------- --------- ----- 1/1/1008 1 1 2/1/2008 0 2
i want count on weekly basis Could any one help me?
I want to set my job agent to schedule for second basis but in EM i found by right clicking job in EDIT Recuring Job Schedule i m getting option in Hr and MM but not in SS so where i can set this in seconds (SS) or through QA????
Currently database1 and database2 are planned to be synchronized with the interval of 5 seconds!! ( dont know exact, but i select run continoulsly on subscriber screen ) This could cause serious performance issues as both applications will be live on difference servers.
how can i Synchronization should be performed on day basis or it should be modification basis to avoid any performance issue on live application
i am using merge replication Local distributor and subscriber on another server .
On a webpage, there are filters to choose from. Like date, amount, SSN (multiple filters can be choosen).I have a single query so far. SqlCommand cmd = new SqlCommand("SELECT [column1], [column2], [column3], [column4], [column5] FROM [table] WHERE [column4] = 'condition4' AND [column5] = @total_bill AND [last_change] >= @txtStartDate AND [last_change] <= @txtEndDate ", Conn) ; cmd.Parameters.Add(new SqlParameter("@total_bill", total_bill1.Text)); cmd.Parameters.Add(new SqlParameter("@txtStartDate", txtStartDate.Text)); cmd.Parameters.Add(new SqlParameter("@txtEndDate", txtEndDate.Text)); I want to break the query so that it executes on the basis of different sets of conditions (filters). If I dont select date filter, then the above query will not execute properly.Please help.
I currently have a series of stored procedures that capture stock prices on a daily basis, then summarize the results into a daily, and further, a weekly summary of the "index" of a group of stocks. The data is accumulated from a (to use a highly technical unit of measurement...) bunch-O-individual rows of data using aggregate functions such as AVG and SUM.
The problem is that I occasionally get a warning on such aggregate statements which is the common one complaining thusly: "Warning: Null value is eliminated by an aggregate or other SET operation"
I know where it comes from, and I know how to code to protect the aggregate from complaining (i.e., AVG(ISNULL(yadayada,0)) ) but I am interested in figuring out a way to REPORT the statement that contains null values. I can, of course, capture ERRORS in selects, but is the same mechanism used to capture these NULL warnings on my aggregate statements? I don't necessarily want to know which individual row is causing it, just want to "tag" somehow the statement that results in the warning so I can go back after the run and check into it (after capturing local "pointer" info at the time the offending aggregate is invoked).
The code I use to capture errors and trace information follows: UPDATE PortfolioPerformance SET PrevDate = @PrevDate, DailyPerChg = GPP.DailyPerChg, DailySumPriceChg = GPP.DailySumPriceChg, SumCurrPrice = GPP.SumCurrPrice, SumPrevPrice = GPP.SumPrevPrice, StockCount = GPP.StockCnt, AvgHighPriceRatio = GPP.AvgHighPriceRatio, AvgLowPriceRatio = GPP.AvgLowPriceRatio, Volume = GPP.Volume FROMPortfolioPerformance PP (nolock), VIEW_Get_PortfolioPerformance GPP WHERE PP.PortfolioID = GPP.PortfolioID AND (PP.CreateDate = GPP.CreateDate AND PP.CreateDate = @CreateDate) AND PP.PrevDate IS NULL
NOTE: the aggregation in the above code is performed in the view referenced as "GPP", but that's outside the realm of the question, I think, so I won't bore you with the details of that just yet.
So I think if I can capture the warning like I do the errors, I can accomplish what I want to accomplish. I haven't yet been able to find any guidance in the Books Online, so do any of you have any pointers?
How would you solve this problem? Would you use replication? SSIS? Schedule stored procs? A third party data tool?
We have about 50 databases located at offices around the world. There are several key tables which we need a central copy of at our Head Office.
We need only the new INSERTS or UPDATES for the day to be copied to head office. The copying process should happen after business hours for that office.
Some cleaning of the data needs to happen before updating head office.
Our database is being installed at new offices every other month, so it's important to us that this be easy to add a new database into the schedule, ideally without needing a SQL DBA or Developer.
We want something robust, performant and easy to troubleshoot.
On what basis, SQL server decides the sequence of these rows?
I need to fetch data from a table with very large number of rows. Because of the datasize I need to do this in chunks. I am thinking of passing row counter and fetch N rows at a time. I want to know if there is a need for sorting a table before I apply the above logic or I can rely on default sorting.
I have 10 oracle o/p tables. I have to transfer data in monthly or adhoc basis. Each table will have millions of records. How to transfer Oracle to SQL Server 2005. Which is the best way to transfer the data.
Here is my Query, I don't know whether I'm getting it right?
--Quarter 1 SELECTD.MerchantName, A.MID, A.TID, ISNULL(SUM(A.SumTrxnMon), 0) AS SumTrxnMon, E.FullName, E.DxBEmail INTO#Quarter1 FROMdbo.tblRPT_Spend AS A INNER JOIN dbo.tblMer_DeployORetrieveTerm AS B ON A.MID = B.MID AND A.TID = B.TID INNER JOIN
now I want to call all the sp on the basis of input like If filename is Fdoor then it shold fire the SP_Archive_using_merge_Fdoor , if file name is Fdoop then it shoilud fire the SP_Archive_using_merge_Fdoop like that .
below is the 2 sp .
--First SP ALTER PROCEDURE [dbo].[SP_Archive_using_merge_Fdoor] AS BEGIN SET NOCOUNT ON DECLARE @Source_RowCount int DECLARE @New_RowCount int
Anyone have any experience in connecting to a Basis database? I want to build an SSIS package to ETL data out of a Basis database and into a sql server 2005 db. I have downloaded the ODBC driver, but am not usre how to plug it in to SSIS to do teh extraction.
Also I am totally new to Basis...it is not an RDBMS at all but the literature says it supports SQL queries, so I am not sure about where to start.
I've just spent the most frustrating 4 hours trying to install SP2.
I kept getting "unable to start SQL Server service....see BOL ...re starting the service manually" messages; with the only options being to Retry or Cancel. Retry resulted in the same message. Cancel resulted in all components except SQL (sqlexpress) being installed - therefore totally useless
No mater what I did i could start the SQLservice as it did'nt show in the list of Services; nor did the SQL EE programme show in the Add/Remove Programs...evenn though the other components did.
I deleted/removed everything that was istalled; cleaned up the Registry; and tried again...
..and again
..and again
FIVE install / re-install failures; numerous registry cleanups!!!!
I was at the point where it looked like i'd have to reformat my hard disk, and completely re-install XP, all aplications i.e. Office 2003 etc in order for the thing to work.
I gave it one more shot, but this time, during install
- selected all components except SDK
- I unchecked the Start SQL / BROWSER at startup
- Checked the Enable User Instances and Add to Administrators boxes
Lo and behold....it installed!!!
I'm still very worried as the registry is full of "duplicate" entries i.e. Blah blah blah and blah blah blah.1
But at least the thing is installed.
Would interested to hear if anyone has had similar or other problems with the SP2 install/upgrade
----------------------
For info:
I ran the install as Administrator
IIS not installed
Win XP Sp2 fully updated
Dell Inspiron XPS M1710 laptop with 2gb Ram; 2ghz Duo core processors
I would like to force only certain users on certain databases to use encrypted connections for accessing our 2005 SQL Server. I've found a clunky way to do it, via registering a login event notification and using the kill statement. However, that's ugly, and I don't think there is any guarantee that the event will always fire before an evil client could get off a few evil commands. Is this supported, or is the only supported feature the forceencryption option, which is server-wide?
I have upgraded a MS SQL database from 6.5 to 7.0. The database functioned fine in 6.5, now I have a table that is locking due to a blocking process. If I kill the process all is fine, but am trying to determine what is causing the process to hang. Has anyone experience any similar situations.
Very long story and I will not bore you with it. What I need to do is call a stored procedure and from within that stored procedure, initiate other sp and get out before those other sp are finished running.
I know I can execute a job but my volume would be about 1 job per second with a life of 2-5 seconds. So in a very short period of time, I can stack up a large volume of jobs.
How to kill process??? I turn off workstation but the process still available in the sysprocesses table and in the Curent activity window. I can't to kill this (and I can't to restart serever, because the users) .
If anyone know something about this problem. Thanks
I have an SQL server with several (100) users connected. When I run a long running process, it severely impacts user performance. The long running process is a store procedure with several cursors.
Are there SQL configuration settings that would reduce the impact of these long running processes on other Users?
/* The Following Stored Procedure helps to Kill All Processes in a Particular DataBase With Out Current Process */
Create Proc Sp_KillAllProcessInDB
@DbName VarChar(100)
as if db_id(@DbName) = Null begin Print 'DataBase dose not Exist' end else
Begin Declare @spId Varchar(30)
DECLARE TmpCursor CURSOR FOR Select 'Kill ' + convert(Varchar, spid) as spId from master..SysProcesses where db_Name(dbID) = @DbName and spId <> @@SpId and spId > 50 and dbID <> 0 OPEN TmpCursor
FETCH NEXT FROM TmpCursor INTO @spId
WHILE @@FETCH_STATUS = 0
BEGIN
Exec (@spId)
FETCH NEXT FROM TmpCursor INTO @spId
END
CLOSE TmpCursor DEALLOCATE TmpCursor
end /* The Above Query Helps TO Change a Database in Single USer Mode Quickly*/
I have been needing to disconnect users often, before backing up a database or setting it to restricted users. Each time, I have to go to Current Activity & kill one process at a time. Is there a way, by which I can kill all processes on a database or force out all coonections to it?
Can you kill processes in SQL Server 2K without stopping SQL Server and restarting it? I am using sp_who2 to get a list of active users and I see some accounts that are logged off but still showing up and I am trying to find a way to Kill these accounts and the processes they are doing.