how can i get the package execution start or end time along with DATE as a derived column into the output table.
i tryed using getdate() as expression but it gives diff time as my pkg runs in a loop.
and i see a sys variable STARTTIME which just gives time.
any help appriciated
thanks
Hi, all experts here,
Thanks for your kind attention.
I want to use time series algorithm to mine data from my case table and nested table. Case table is Date table, while nested table is the fact table. E.g, I want to predict the monthly sales amount for different region (I have region table related to the fact table), how can I achieve this?
Thanks a lot and I hope it is clear for your help and I am looking forward to hearing from you shortly.
With best regards,
Yours sincerely,
Hi, all experts here,
Thank you very much for your kind attention.
I am confused on key column of case table and key time column of nested table by using Time Series algorithm.
In my case, the case table structure is as below:
Territory key text (the ID is actually dimrisk_key, in this case, I use the name column binding to combine the Territory column of case table Dimrisks),
While the nested table structure is as below:
Cal_month key time (in this case, actually the ID is dimdate_key, again, I used name column bining property to bind the Cal_month to the ID)
So my question is, as the key column of case table has been set to be Territory, as a result, does the model training still cover all the cases (rows) based on the ID of the table?
Also, in the nested table, as the key time column has been set to Cal_month rather than Dimdate_key of the nested table, as a result, would the single series based on the cal_month?
Hope it is clear for your advices and help.
And I am looking forward to hearing from you shortly.
With best regards,
Yours sincerely,
I have an SSIS package that moves data from SQL Server to an legacy Access database. In SQL Server, there is a date/time column that I need to split into a separate date column and time column in the access database. Initially I just created a derrived column for the time and set the expression equal to the source date/time column from SQL Server. Unfortunately, that just makes the date column and time column the same having the full date/time.
What expression can I use during a derrived column transformation to assign just the date to a derrived column and just the time to another derrived column?
Thanks,
Steve
Hi, all experts here,
Thank you very much for your kind attention.
I am confused on key time column selection. e.g, I want to predict monthly sales amount, then what column in date dimension should I choose to be the key time column? Is it calendar_date (the key of date dimension) column or calendar_month?
Thanks a lot for your kind advices and help and I am looking forward to hearing from you shortly.
With best regards,
Yours sincerely,
How can I calculate a DateTime column by merging values from a Date column and only the time part of a DateTime column?
View 5 Replies View RelatedHow can I obtain just the time portion from a date/time column?
My data contains "2008-05-19 09:30:00.000"
Actually, all I want/need is the hh:mm part of it.
Thanks,
Walt
I have two columns in a table:StartDate DateTime and StartTime DateTime.The StartDate column holds a value such as 07/16/2004The StartTime column holds a value such as 3:00:00 PMI want to be able to add them in a stored procedure.When I use StartDate + StartTime I get a date two days earlier than expected.For example, instead of 7/16/2004 3:00:00 PM StartDate + StartTime returns7/14/2004 3:00:00 PM.Can anyone point out wht I'm doing wrong with this one?Thanks,lq
View 2 Replies View RelatedHow can i configure DTS package elements at run time through VB6..?
Like Specifying the Source,Destination etc
Hello all, I have a SSIS package that is importing data from a DB2 database. I am using SSRS (BIDS/VS2005). I want to be able to access the last time a particular package ran in my report, like in the footer of the page. I have found the globals.executiontime for grabbing the time the report was ran.
I want to do the same thing, except I want to call the date/time the last import was made to the database. Is there some easy way to grab that from SSIS, or will I need to maybe reference the DB creation time? (The database is dropped and recreated (for now anyway) in the SSIS package.)
Thanks for your help.
Okay, we have are running our Master Package (and therefore all related Child packages) through a .bat file. The .bat file is scripted using the following logic for an entire month of daily runs:
Code Snippet
DTExec /FILE E:ETLFinancialDataMartMaster.dtsx /DECRYPT masterpwd /SET Package.Variables[ReportingDate].Value;"2/01/2007" > E:ETLErrorLogsProcessingetl_20070201log.txt
IF NOT %ERRORLEVEL%==0 GOTO ERROR%ERRORLEVEL%
mkdir E:ETLErrorLogsArchive20070201
move E:ETLErrorLogsProcessing*.txt E:ETLErrorLogsArchive20070201
DTExec /FILE E:ETLFinancialDataMartMaster.dtsx /DECRYPT masterpwd /SU /SET Package.Variables[ReportingDate].Value;"2/02/2007" > E:ETLErrorLogsProcessingetl_20070202log.txt
IF NOT %ERRORLEVEL%==0 GOTO ERROR%ERRORLEVEL%
mkdir E:ETLErrorLogsArchive20070202
move E:ETLErrorLogsProcessing*.txt E:ETLErrorLogsArchive20070202
etc...
Generally it takes about 40-45 minutes to run one days worth of data. However, we have found unpredictable instances where the job will take 3 hours or even 6 hours and appear to hang....
The weirdness sets in when we kill the job and rerun it. In all instances of a rerun, the job will execute in the normal 40-45 minute time frame. Obviously, we would like to institute some sort of logging, monitoring and error handling....including if need be a method to timeout a process and restart it.
I am reviewing the WMI (Windows Management Instrumentation) Task but I'm not entirely convinced that it's the right tool for the job.
Questions:
Has anyone else experienced the type of processing behavior that I described?
Has anyone been successful at using WMI or another process to monitor and timeout packages? If so, are there sample packages or a good tutorial that maps it out?
Unrelated to this issue, we also have instances incomplete processing logs. The logs don't finish writing and the weird part is that they all end at the same point, does anyone have experience with incomplete job logs?:
Code SnippetProgress: 2007-06-20 12:46:49.87
Source: Update factFinancial Data Flow
Cleanup: 11% complete
Thanks in advance!
I''m working on a data conversion process and trying to convert data from SQL Server to Oracle 10g. The problem I'm encounterin g is that when I go from SQL to Oracle with a date into a Timestamp field in Oracle, it errors out. Not because it's only Timestamp, Oracle accepts dates in that field type. I don't know why this is happening, but I'm sure someone else has run into this...thoughts?
View 1 Replies View RelatedHello All,I am am having a strange problem. I am trying to execute a DTS packagefrom VB .NET (.aspx) page. The DTS package takes 21 minutes to execute.When the aspx page sends a command to MS SQL server to execute this DTSpackage it starts executing. After 8 minutes the .aspx page shows error"Cannot Find Server" and errors out with a white screen. Theinetresting thing is the DTS package is still being executed in backendand completes execution successfully.I have no clue why this is happening because there are other DTSpackages that I am executing through the same .aspx page that takesabout 3 minutes to complete execution and are working absolutely fine.By the way I have a stored procedure wrapper over every DTS package. Sowhen i have to execute a DTS package I execute a stored which inturnexecutes a DTS package.Thanks for reading my query. If you have any suggestions please let meknow.Thank you in advance.
View 1 Replies View Related
Hi,
I have a DTS in SQL Server 2000, Where I am importing some data from a remote server (SQL 2000) to local server (SQL 2000).And this is working fine. it is taking max of 1 min to execute the package.
Now I have created the same DTS in sql server 2005 (SSIS) where the source server is sql server 2000 and the destination server is 2005.and I have created the ssis in that server. The same logic which i have created in sql 2000. But here it is taking almost 10 min to execute the package.
Where as the same in sql server 2000 taking max of 1 min. Why this happening.. Is there any configuration to execute the SSIS package.?
Dear all,
This is a beginner's question so bear with me - I have a SSIS package that builds an Analysis Services fact table - depending on user input, this table is either re-built from scratch or any additional records since it was last run are created.
Where is the best place to store the time a package was last run - in a seperate table in the database or somewhere in the fact table?
Sam
Dear friends,
Our package which at the time of normal execution takes 2-2:30 mins for fetching data on VPN with some select queries. But some times its job runs for hours and hours. What could be the exact reason behind it? I guess its queries are stuch somewhere, but not when we run from the BIDS or run the job manually.
Please help. Thanks.
I have created a DTS package. I want to know how to call/run this package from a stored procedure.
Please let me know how to call this DTS from a Stored Procedure.
Thanks.
-Rajesh
I have a package variable that is a datetime... how am I able to set the time of that variable? In visual studio on the variables screen if I click on value it brings up a calendar control - I can't seem to edit the time portion of the variable.
When I first click open the variables window it will say 9/1/2006 12:00:00 AM - but as soon as I click on the value box the 12:00:00 AM part will disappear and I can't edit it. I've tried on someone elses PC as well to make sure there isn't something wrong with my visual studio
Ideas?
I have several sequence containers in one package that fire off execute package tasks. I would like each of the sequence containers to start at the same time when the job starts running. However when I set them up to do that, i get an error that the variable cannot be read because it is locked. I have the variables setup as readonly so not sure why they are being locked. When I run the package and have each sequence container fire off after the previous one ends it runs fine.
Any ideas on how to get around this?
Thanks!
Hi All,
I have to create a task, and in that task i have check exection time of package and if it more then specified time, send a mail with custom message to some specific users.
or
Can i write a event which occur after specific time and send a mail?
Please help me.
Thanks in advance....
From
Manish
Does the validation time of DTSX package depend on the amount of data?
I have a data transfer task which contains a oledb source data from a SQL Server 2005 view. Then the data go through 3 Lookup transfromations before going into another view, also on SQL Server 2005 but a different database.
The purpose of this package is to load fact data, so it has to deal with few million of rows. Before setting DelayValidation to true, it takes few minutes just to open the package in BI Studio. Now with DelayValidation set to false, I can open the package without any problem. But it takes more than 5 minutes during Validation, Prepare for Execution and Pre-execution Phase. During the time the memory usage and cpu time on SQL Server goes up significantly. The CPU doesn't hit 100% though. My client machine doesn't have any significant activity.
I have similar packages ( Oledb view -> 3 or 4 lookup Transformations -> Oledb Table) but dealing with dimension data. With DelayValidaion set to False, those packages can be opened in BI Studio within few seconds. They also take only few seconds during those 3 phases before starting actual execution phase.
So I have an impression that the validation time depends on the amount of data in the database. Shouldn't it just depends on the metadata?
Hi
Can I limit an SSIS package that should run only for specific time, if it exceeds then should stop.
Ex. mySSIS.dtsx should run only 5 min, if exceeds 5 min it should stop running and exit.
Thanks,
Madhu
I load SSIS package using following code:
Application app = new Application();
Package pac = app.LoadFromSqlServer(packageName, serverName, null, null, null);
For simple package containing 2 tasks this code executes about 20 seconds.
If I load old version (SQL2K) package from SQL2000 then it takes 5 seconds.
Is any way to increase loading speed for SSIS packages?
I have SSIS Projects taking a long time to open with packages with a large number of data flows. Is there a way to turn off validation of metadata when a package opens? Turn off validation during execution on SSIS Service (after previously validated in dev)? Or be able to control when validation takes place in general?
In my one package (1 of 5) I have 43 data flows (with a single source to target mapping) in 4 sequence containers, and it takes approximately 2-3 seconds per source to target mapping and sequence container to validate which will translate to 1 ½ to 2 ½ minutes to open. When the project with all 100+ tables for the data warehouse goes through validation, I can make coffee in the time it takes to open the project. I have to delete *.suo file (or verify all packages are closed in the designer and save the project file), and when I open the project, I have to jump immediately to SSISÃ Work Offline to set it to not validate the metadata to be able to work in a timely fashion. DelayValidation=TRUE does not help much.
Running in debug mode, has an effect of causing packages that were not open and validated to go through validation though I am not running those packages. Validate once during design and run forever.
Even if I re-open a package that I just closed from designer and had gone through validation, it will go through the validation process again.
It would be great if there could be an on-demand option off the menu bar to allow one to control when validation can take place for a project, or a more granular validation option for a specific data flow or container.
Hi,
I want to include the running time (Start Time and End Time) of the Package in my script task that sends out an email after job completion.
How do I get the start time and end time?
thanks a lot
cherriesh
Hi All,
I am trying to import data from flat file to sql server 2005 DB table using SSIS. There are 4 different text files in the input folder. I am using for loop to iterate reading and importing 4 times. In order to do this I have set "Connection String" property of a connection manager to a package level variable using expression. There is a script which supplies the source file name and assigns to this variable.
The package works fine. The data gets imported successfully into the destination table. However, when I close the package, reopen it and then run, the Data Flow task fails with the error "Can not open the source file". If I enter a valid file name in the Flat File Source task and run the package it works again. As soon as I close the file/package, reopen and then run, the data flow task fails with the same error.
How do I make this working. I am planning to schedule a job which will execute the package programatically. In that case no user intevention is possible.
I would appreciate your help on this. I can provide further details if required.
-SGK
Can somebody please help answering this question?
I'm planning to consider writing a SSIS package for a new project that requires downloading large chunk of data and transform into the diverse databases such as MS SQL or Oracle depending on the Client's Datbase.
For the clients having SQL Server installed at their end, i had no issues in deploying this package on their server and run it in their licensed instance.
What should be the case for others having Oracle database? Wouldn't installing the SQL 2005 client tools install the necessary run-time services for running SSIS packages? What i understood from the MSDN library (http://msdn2.microsoft.com/en-us/library/ms403355.aspx) is that there's no run-time support available for running the SSIS packages (unlike DTS run-time support) in production environment!
Would that mean that it requires a SQL Standard edition, at minimum, (as Integration Services is OOTB from Standard Edition onwards) to be installed at the production site to run this package?
If so, the client wouldn't be ready (which is fair too) to buy the new license just to run this package. Is there any work-around/suggestions for this case?
If not, can somebody please point me to the right location where i can download the run-time support for running SSIS packages?
Any thoughts on this is highly appreciated.
Thanks for your time.
~Sagar
I have an SSIS package that imports data from an Excel file, replaces any value in Excel that reads "NULL" to "", then writes the data to a couple of databases.
What I have discovered today, is I have two columns of dates, an admit date and discharge date column, and what I need to do is anywhere I have a null value in the discharge date column, I have to replace it with the value in the admit date column.Â
I have searched around online and tried a few things using the Replace funtion in Derived columns but no dice so far.Â
I am using SSIS to populate a star schema.
The issue is in the data flow for loading and setting the Fact table dimension keys (the dimensions are all loaded fine). After 16 rather pedestrian Lookup Transformations, I have an escalating problem adding additional Lookup transforms to the Data Flow. The problem is not in execution; the problem is adding more transforms in design mode.
Lookup # Fields in Data Flow Time to validate that lookup
<17 47 Sub-second
17 48 2 sec
18 49 4 sec
19 50 8 sec
20 51 16 sec
21 52 32 sec
22 53 64 sec
While I€™m intrigued by the mathematical progression that is forming here, the issue is that I have at least 6 more Lookups to perform. I hope you can see my dilemma.
I have gone to where it takes a little over 4 minutes each to validate the lookup transform and its associated Derived Column transform and Union transform (Total 12 Minutes). Not only does this add up to many idle minutes to each design step, BUT it breaks the debugger as it pre-validates the ENTIRE data flow before it ever switches into debugging mode.
Some notes:
1. It doesn€™t matter what order the Lookup transforms occur in, the timings are exactly the same.
2. I tried many Data Flow execution optimizations, but they don€™t improve the validation times (or even get a chance to improve the execution times!)
I realize this may be somewhat of a unique problem.
Thanks for any help you are able to lend.
-Dave
Hi,
I have a package designed as bring data tables over to SQL Server. There are 9 data flow tasks that runs parallel, to bring 9 datatables over. In BIDS, when I execute the package, it runs like 8 minutes. Or if I start the scheduled job manually, it runs around 8 minutes too. But it runs about 30 minutes at the scheduled time at midnight.
I wonder what I can do to speed up the scheduled job.
Thanks
Trying to get the records where the date = today and the time is only
within 15 mintues
So if the time was 10:20 i only get records 2 and 3
COLUMN STAT_DATE
1.2008-06-10T10:30:00.537
2.2008-06-10T10:15:00.530
3.2008-06-10T10:00:00.760
I am in between of creating a dynamic SSSIS package which will run for multiple zones having different source connection.My source is in Oracle.I am having 3 DFT with the 3 different source tables.I want to create a package with above DFT dynamically so that my single package can run for the entire zone with dynamically source connection change.I have created a Master table which stores the zone source connection string and zone name. I have 2 different connection.so if in future any new zones come so only newly zone details need to be add in master table without opening the package.
View 3 Replies View RelatedHi
I am trying to Execute an SSIS Package from C# Dot Net.
Below the code i am using.
try
{
Application app = new Application();
Package package = app.LoadPackage("DTSPackage/WinServeMonitoring.dtsx", null);
DTSExecResult result = package.Execute();
label1.Text = "Package Execution " + result.ToString();
}
catch (Exception exp)
{
label1.Text = exp.Message.ToString();
}
I have added reffrence for DTS Run time (using Microsoft.SqlServer.Dts.Runtime;)
But I am getting an Error like this
Error 1 'Application' is an ambiguous reference between 'System.Windows.Forms.Application' and 'Microsoft.SqlServer.Dts.Runtime.Application' C:Documents and Settingsvahqmd10My DocumentsVisual Studio 2005ProjectsWindowsApplication1WindowsApplication1Form1.cs 23 17 WindowsApplication1
Error 2 'Application' is an ambiguous reference between 'System.Windows.Forms.Application' and 'Microsoft.SqlServer.Dts.Runtime.Application' C:Documents and Settingsvahqmd10My DocumentsVisual Studio 2005ProjectsWindowsApplication1WindowsApplication1Form1.cs 23 39 WindowsApplication1
Error 3 The type 'System.Windows.Forms.Application' has no constructors defined C:Documents and Settingsvahqmd10My DocumentsVisual Studio 2005ProjectsWindowsApplication1WindowsApplication1Form1.cs 23 35 WindowsApplication1
Can any one Help me !
Regards
Deepu M.I