Improvement Sysdtslog90?
Apr 12, 2007
hello
im trying to create a ssis monitoring tool (using sysdtslog90, reporting services and system tables) including sql server agent ssis schedules/history. but im having some problems evaluating sysdtslog90. im using the built in SSIS logging without any custom logging tasks (i only log OnError and OnWarning) to keep the logging table as small as possible (one sysdtslog90 table for all packages running on the server).
and there is the problem:
there are some executions without any events except OnError. the source of this event is the name of the failed task, so how do i get the name and id of the failed package?
"normal" executions have "PackageStart" and "PackageEnd" where the sourceid and source are equal with the id and name of the package
i could add logging for OnPreValidate and catch the first line, because no package can fail before validation!?
but isnt there any better solution?
i would love it to have two more columns in sysdtslog90.. package id and name. would be much easier to evaluate sysdtslog90 :/
- paul
View 8 Replies
ADVERTISEMENT
Dec 4, 2007
Hi,
is there any way to use dbo.sysdtslog90 table for SSIS package logging?
View 2 Replies
View Related
Oct 30, 2006
I have enabled logging using database.
When I try to query the sysdtslog90 table using System::ExecutionInstanceGUID to link executionid - it is not matching.
Any resolution?
Thanks
View 7 Replies
View Related
Jun 13, 2006
The documentation says:
"An integer value that identifies the event associated with the log entry. The value 0 indicates the event provided no identifier."
When I look at the sysdtslog90 table and I filter for 'PackageEnd' events, I see a value of 0, 1 or 3 in the datacode column. Is there any information I can glean from these values, e.g. package success or failure?
View 7 Replies
View Related
Nov 14, 2007
In our business we receive debt from clients that we work in order to try and reclaim some of the debt on their behalf. When a debt file comes in from a client it is loaded onto the system and every entry in that debt file is stamped with the same batch id. Using the batch id we track each debt file and monitor how well we are working that debt file. So as an example we could receive a debt file with 100 records in it each of these records would be stamped with a batch id of say 100001, the next file that is loaded onto the system, each record would then be stamped with 100002. so we can track how each batch is doing by grouping by batch id.
I have written my query that basically groups all the accounts on the system by batch id. This is fine when I only want to return totals and sums in the select list. The way I have the written the query If I want to calculate the payments we have received for each batch or the commission we will make on the payments (or any other info per batch) I have to use subquerys in the select list using the batchid I have grouped by . Is this the best/only way to achieve what I want to do.
Any input most welcome.
sql below:
Select c.Name,ib.ImportBatchID BatchID,
---------------------------------------------------------------
CONVERT(VARCHAR(10),ib.UpdatedOn,103) LoadedOn,
---------------------------------------------------------------
Count(d.DebtID) Reg_Count, --No. Reg
---------------------------------------------------------------
Sum(d.DebtValue) Reg_Value, --Value Reg
---------------------------------------------------------------
(Select sum(da.Amount) from dbo.DebtAdjustment da WITH (NOLOCK), dbo.ImportBatchItem bi WITH (NOLOCK) where
bi.ItemID=da.DebtID And bi.ImportBatchID=ib.ImportBatchID) Adjustments,
---------------------------------------------------------------
(Select count(dh.DebtID) from dbo.Debthistory dh WITH (NOLOCK), dbo.ImportBatchItem bi WITH (NOLOCK) where --No. OBC's
bi.ItemID=dh.DebtID And dh.Note like 'OBC:%' And bi.ImportBatchID=ib.ImportBatchID) OBC_Num,
---------------------------------------------------------------
(Select count(dh.DebtID) from dbo.Debthistory dh WITH (NOLOCK), dbo.ImportBatchItem bi WITH (NOLOCK) where --No. ICC's
bi.ItemID=dh.DebtID And dh.Note like 'ICC:%' And bi.ImportBatchID=ib.ImportBatchID) ICC_Num,
---------------------------------------------------------------
(Select count(dh.DebtID) from dbo.ImportBatchItem bi WITH (NOLOCK), dbo.DebtHistory dh WITH (NOLOCK)
Where dh.DebtID=bi.ItemID AND bi.ImportBatchID=ib.ImportBatchID AND dh.UserName='Letter Server' AND dh.Note like '%Letter%') ItemsMailed,
---------------------------------------------------------------
Cast((Select sum(CASE
WHEN dp.ReceivedByID = 1
THEN dp.Amount * (tF.Rate /100)
WHEN dp.ReceivedByID = 2
THEN dp.Amount * (tD.Rate /100)
ELSE
dp.Amount * ((tF.Rate + tFe.Rate) / 100)
END)
From
dbo.DebtPayment dp JOIN dbo.Debt d ON d.DebtID=dp.DebtID
JOIN dbo.ImportBatchItem bi ON dp.DebtID=bi.ItemID
LEFT JOIN dbo.mTrackerFeeChange tF ON tF.ClientID=d.ClientID
AND tF.ContractID=d.ContractID AND dp.ReceivedByID=tF.RateType
AND (
(dp.PaymentOn >= tF.StartDate AND dp.PaymentOn <= tF.EndDate)
OR
(dp.PaymentOn >= tF.StartDate AND tF.EndDate IS NULL)
)
LEFT JOIN dbo.mTrackerDirectChange tD ON tD.ClientID=d.ClientID
AND tD.ContractID=d.ContractID AND dp.ReceivedByID=tD.RateType
AND (
(dp.PaymentOn >= tD.StartDate AND dp.PaymentOn <= tD.EndDate)
OR
(dp.PaymentOn >= tD.StartDate AND tD.EndDate IS NULL)
)
LEFT JOIN dbo.mTrackerFieldChange tFe ON tFe.ClientID=d.ClientID
AND tFe.ContractID=d.ContractID AND tFe.RateType=dp.ReceivedByID
AND (
(dp.PaymentOn >= tFe.StartDate AND dp.PaymentOn <= tFe.EndDate)
OR
(dp.PaymentOn >= tFe.StartDate AND tFe.EndDate IS NULL)
)
where bi.ImportBatchID=ib.ImportBatchID) AS decimal(10,2)) ComRate,
---------------------------------------------------------------
(SELECT sum(Debt.OutstandingValue) TotalValue From Debt WITH (NOLOCK), DebtStatus WITH (NOLOCK), ImportBatchItem bi WITH (NOLOCK)
WHERE Debt.Status=DebtStatus.DebtStatusID AND bi.ItemID=Debt.DebtID And bi.ImportBatchID=ib.ImportBatchID AND
DebtStatus.Description LIKE ('%' + 'Arrangement' + '%') AND Debt.AgreementPaymentMethodID = 1 AND Debt.OutstandingValue > 0) Girobank,
---------------------------------------------------------------
(SELECT sum(Debt.OutstandingValue) TotalValue From Debt WITH (NOLOCK), DebtStatus WITH (NOLOCK), ImportBatchItem bi WITH (NOLOCK)
WHERE Debt.Status=DebtStatus.DebtStatusID AND bi.ItemID=Debt.DebtID And bi.ImportBatchID=ib.ImportBatchID AND
DebtStatus.Description LIKE ('%' + 'Arrangement' + '%') AND Debt.AgreementPaymentMethodID = 2 AND Debt.OutstandingValue > 0) StandingOrder,
---------------------------------------------------------------
(SELECT sum(Debt.OutstandingValue) TotalValue From Debt WITH (NOLOCK), DebtStatus WITH (NOLOCK), ImportBatchItem bi WITH (NOLOCK)
WHERE Debt.Status=DebtStatus.DebtStatusID AND bi.ItemID=Debt.DebtID And bi.ImportBatchID=ib.ImportBatchID AND
DebtStatus.Description LIKE ('%' + 'Arrangement' + '%') AND Debt.AgreementPaymentMethodID = 3 AND Debt.OutstandingValue > 0) CreditCard,
---------------------------------------------------------------
(SELECT sum(Debt.OutstandingValue) TotalValue From Debt WITH (NOLOCK), DebtStatus WITH (NOLOCK), ImportBatchItem bi WITH (NOLOCK)
WHERE Debt.Status=DebtStatus.DebtStatusID AND bi.ItemID=Debt.DebtID And bi.ImportBatchID=ib.ImportBatchID AND
DebtStatus.Description LIKE ('%' + 'Arrangement' + '%') AND Debt.AgreementPaymentMethodID = 4 AND Debt.OutstandingValue > 0) DirectDebit,
---------------------------------------------------------------
(SELECT sum(Debt.OutstandingValue) TotalValue From Debt WITH (NOLOCK), DebtStatus WITH (NOLOCK), ImportBatchItem bi WITH (NOLOCK)
WHERE Debt.Status=DebtStatus.DebtStatusID AND bi.ItemID=Debt.DebtID And bi.ImportBatchID=ib.ImportBatchID AND
DebtStatus.Description LIKE ('%' + 'Arrangement' + '%') AND Debt.AgreementPaymentMethodID = 5 AND Debt.OutstandingValue > 0) Cheque,
---------------------------------------------------------------
(SELECT sum(Debt.OutstandingValue) TotalValue From Debt WITH (NOLOCK), DebtStatus WITH (NOLOCK), ImportBatchItem bi WITH (NOLOCK)
WHERE Debt.Status=DebtStatus.DebtStatusID AND bi.ItemID=Debt.DebtID And bi.ImportBatchID=ib.ImportBatchID AND
DebtStatus.Description LIKE ('%' + 'Arrangement' + '%') AND Debt.AgreementPaymentMethodID = 6 AND Debt.OutstandingValue > 0) PostalOrder,
---------------------------------------------------------------
(SELECT sum(Debt.OutstandingValue) TotalValue From Debt WITH (NOLOCK), DebtStatus WITH (NOLOCK), ImportBatchItem bi WITH (NOLOCK)
WHERE Debt.Status=DebtStatus.DebtStatusID AND bi.ItemID=Debt.DebtID And bi.ImportBatchID=ib.ImportBatchID AND
DebtStatus.Description LIKE ('%' + 'Arrangement' + '%') AND Debt.AgreementPaymentMethodID = 7 AND Debt.OutstandingValue > 0) Cash,
---------------------------------------------------------------
(SELECT sum(Debt.OutstandingValue) TotalValue From Debt WITH (NOLOCK), DebtStatus WITH (NOLOCK), ImportBatchItem bi WITH (NOLOCK)
WHERE Debt.Status=DebtStatus.DebtStatusID AND bi.ItemID=Debt.DebtID And bi.ImportBatchID=ib.ImportBatchID AND
DebtStatus.Description LIKE ('%' + 'Arrangement' + '%') AND Debt.AgreementPaymentMethodID = 8 AND Debt.OutstandingValue > 0) BankersDraft
---------------------------------------------------------------
From
dbo.Client c WITH (NOLOCK), dbo.Debt d WITH (NOLOCK), dbo.ImportBatchItem ib WITH (NOLOCK)
Where
c.ClientID=d.ClientID
AND
d.DebtID=ib.ItemID
AND
(@ClientID IS NULL OR c.ClientID = @ClientID)
Group by c.Name, ib.ImportBatchID,CONVERT(VARCHAR(10),ib.UpdatedOn,103),DATENAME(Month, ib.UpdatedOn) + ' ' + DATENAME(Year, ib.UpdatedOn)
Order by ib.ImportBatchID
View 2 Replies
View Related
Dec 31, 2007
I did a trace on a production DB for many hours, and got more than 7 million of "RPC:Completed" and "SQL:BatchCompleted" trace records. Then I grouped them and obtained only 545 different events (just EXECs and SELECTs), and save them into a new workload file.
To test the workload file, I run DTA just for 30 minutes over a restored database on a test server, and got the following:
Date 28-12-2007
Time 18:29:31
Server SQL2K5
Database(s) to tune [DBProd]
Workload file C:Tempfiltered.trc
Maximum tuning time 31 Minutes
Time taken for tuning 31 Minutes
Expected percentage improvement 20.52
Maximum space for recommendation (MB) 12874
Space used currently (MB) 7534
Space used by recommendation (MB) 8116
Number of events in workload 545
Number of events tuned 80
Number of statements tuned 145
Percent SELECT statements in the tuned set 77
Percent INSERT statements in the tuned set 13
Percent UPDATE statements in the tuned set 8
Number of indexes recommended to be created 15
Number of statistics recommended to be created 50
Please note that only 80 of the 545 events were tuned and 20% of improvement is expected if 15 indexes and 50 statistics are created.
Then, I run the same analysis for an unlimited amount of time... After the whole weekend, DTA was still running and I had to stop it. The result was:
Date 31-12-2007
Time 10:03:09
Server SQL2K5
Database(s) to tune [DBProd]
Workload file C:Tempfiltered.trc
Maximum tuning time Unlimited
Time taken for tuning 2 Days 13 Hours 44 Minutes
Expected percentage improvement 0.00
Maximum space for recommendation (MB) 12874
Space used currently (MB) 7534
Space used by recommendation (MB) 7534
Number of events in workload 545
Number of events tuned 545
Number of statements tuned 1064
Percent SELECT statements in the tuned set 71
Percent INSERT statements in the tuned set 21
Percent DELETE statements in the tuned set 1
Percent UPDATE statements in the tuned set 5
This time DTA processed all the events, but no improvement is expected! Neither indexes/statistics creation recomendation.
It does not seem that Tuning Advisor crashed... Usage reports are fine and make sense to me.
What's happening here? It looks like DTA applied the recomendations and iterated, but no new objects where found in DB.
I guess that recomendations from the first try with only 80 events were invalidated by the remaining from the long run.
I couldn't google an answer for this. Help!!!
Thanks in advance.
++Vitoco
View 1 Replies
View Related
Feb 15, 2006
I am in the process of evaluating 2005 to upgrade a DTS and A/S solution for accounts receivable data imported from a legacy application on various AS/400s in our enterprise. Our current process uses staging and mart databases with views providing a layer of abstraction. Instead of using upgrade wizards, I would prefer to implement a "ground up" rearchitecture that better incorporates the 2005 toolset.
One of the challenges is building a fact table for historical a/r balances since the legacy application will only provide transactions, meaning that there is no data entry for a month where there might be a balance, but no transaction (payment, invoice, credit, etc) occurred. Our current implementation uses a date table as its base, perfoming a left join to the transaction table. I have to use subqueries to calculate previous balances, which is also used to calculate ending balance.
Should I replace the current view with data flow logic that would perform this calculation? If so, what would be the best method? A merge join? A calculated column? If it helps to provide sample schema snipets, I would be happy to do so.
View 1 Replies
View Related
Jul 25, 2007
Hi,
when I select sql server provider pointing to connection mydatabase for Logging the sysdtslog90 table is not getting created. However if I point the connection to msdb - sysdtslog90 it works fine.
Can I have sysdtslog90 under the application database? what are the steps to do this?
Thanks in advance.
Prabhakaran
View 4 Replies
View Related
Oct 26, 1999
Can anyone suggest hwo can I improve the performance of the replication process and make it faster.
Pran
View 1 Replies
View Related
Jul 23, 2005
Hi,I wonder if you could shed some light into this.I have the following table.Id, ContentId, VersionDate, ContentXmlThere are several ContentIds in the table.SELECT *FROM tblVersionsWHERE (VersionDate =(SELECT MAX(tblVersions2.VersionDate)FROM tblContentVersion AStblVersions2WHERE tblVersions2.ContentiD =tblVersions.ContentiD))ORDER BY ContentIdThis query works to select the latest versions (MAX) of every content,but I do not like it, any other way to do this properly?I also want to do this knowing a set of ids (probably using IN )SELECT *FROM tblVersionsWHERE (VersionDate =(SELECT MAX(tblVersions2.VersionDate)FROM tblContentVersion AStblVersions2WHERE tblVersions2.ContentiD =tblVersions.ContentiD AND tblVersions.ContentiD IN (1, 2, 3, 6, 7, 8)))ORDER BY ContentIdAny ideas for improvements on this query?ContentXml is of ntext typeThanks,/ jorge
View 4 Replies
View Related
Aug 31, 2007
After reading an article today on SQL Server Central about choosing the best connectors for ssis (obviously written to drive sales, but hey it looks like it's working) I downloaded an evaluation copy of DataDirect's 64-bit DB2 driver and started some preliminary testing.
http://www.sqlservercentral.com/columnists/mfrost/3209.asp
Here are some inital findings/opinions:
-Rather easy to install (downloaded, briefly read documentation, and was up and running in about 10 mintues)
-With Microsoft's IBM OLE DB provider for DB2 (32-bit) we can extract 1 million records in approx. 9 minutes
-With the 64-bit driver we can extract 1 million records in approx. 2.25 minutes! 4 times as fast as the 32-bit driver!
-Was unable to get the driver to work through linked server. Tech support opened an issue to look into it.
-Sales rep didn't have exact pricing, but she thought they charged per core.
-DataDirect also has 64-bit Oracle and Sybase drivers available.
For those of you extracting large amounts of data from DB2, Oracle, or Sybase with SSIS and are looking to improve performance I'd recommend at least checking this product out.
Also, I'd be interested to hear if anyone else started testing this or any other 64-bit driver for DB2.
View 2 Replies
View Related
Aug 20, 2007
I have been using the SSIS Log Provider for SQL Server quite a lot this year with little to no problems. Recently however as I edit or create packages the logging seems to keep going to the master.dbo.sysdtslog90 table as opposed to the [db].dbo.sysdtslog90 as setup in the package. I tried totally deleting the connection from the package and recreating and still had the problem.
Is there some setting on the SQL server itself that can cause this? Our "dba" likes to make changes without letting anyone know about them.
Any ideas?
View 3 Replies
View Related
Feb 5, 2008
Sorry to be an ignorant American here, but I'm having a challenge I just can't seem to wrap my head around.
I am attempting to make a nice query for a report that will show how our SSIS jobs are doing. All my packages use the native SQL Server logging capability. The table is sysdtslog90 and it stores the message to a field called 'message' (crazy, I know) with a datatype of nvarchar(4096).
The challenge I'm having is that I'd like to grab all lines where my message is X. The following queries will return no results. The first is me typing in the message I see in the window, the second is copying the value and pasting it in. I know there are instances where that message exists, sadly, in the log table.
Code Snippet
SELECT
L.*
FROM
sysdtslog90 L
WHERE
L.message = N'A commit failed.'
SELECT
L.*
FROM
sysdtslog90 L
WHERE
L.message = N'A commit failed. '
If I were to change that clause out to a like, it'd work just fine. Is that the appropriate way to work with unicode literals? It doesn't feel right.
Code Snippet
SELECT
L.*
FROM
sysdtslog90 L
WHERE
L.message LIKE N'A commit failed.%'
I have exported my sysdtslog90 table to a unicode flat file and looked at this message in a hex editor and it looks fine, nothing looked awry. I then opened the file up in SSMS, thinking perhaps it's a silent conversion issue with pasting, but to no avail. Anyone have some guidance for me?
View 6 Replies
View Related
Jul 17, 2007
Hi,
I am running MSSQL 2005 Standard edition on a two processor Intel Xeon 3GHz (dual-core) with 8GB RAM.
I notice in "Windows task manager CPU performance" while running a long SQL statement (takes 1.5 hours), only 1 logical (out of 4) is utilised at >70%. The remaining 3 logical processors hover around 10%
Using Performance monitor, the average read queue, write queue, and pages/sec also hovers around 25%, indicating no heavy physical disk/memory loading.
How can I set to utilise more physical/logical processor to improve the MSSQL performance ?
Thanks.
View 2 Replies
View Related
Mar 25, 2014
Am customizing SQL server MGMT tools 2012 for Mass deployment.Client had asked to remove Customer Feedback option from help menu.how to disable that.
View 6 Replies
View Related
Oct 17, 2007
Hi,
I decided to use the SQL Server log provider to store logging data of all my Integration Services packages. I also created some reports about this data for operating purposes.
I have a problem occurs the name of the executing package is not always written to the log,but the name of the single task which failed. But that is not very useful information for operating, because I do not see any chance to get the name of the package by the information which is logged in the sysdtslog90 table in the database which I defined for SSIS Logging.
How do I configure the package to always log the package information into the table, too?
Best regards,
Stefoon
View 5 Replies
View Related
