Hi,
I need to estimate the effort required in writing some SSIS packages.
Could anyone provide some pointers to the various factors (E.g. number of tables, source, etc.) which influence the effort estimate for SSIS packages?
Thanks in advance.
Regards,
B@ns
Hello there... Happy New Year!
A client of mine, a group of 4 doctors, are interested in implementing a small office network in which they will switch to a digital filing system. I have recommended a Microsoft Server 2003 system, therefore the patient information will be stored in the SQL server.
I have never administered, administrated?? (see, I really need help here!) a database, but have built small databases from scratch in C and with Access, so I feel confident that I could set up a system for them. I need to have some kind of estimate, both a fee and implementation schedule. Can you help me get a handle on the scope of this?
Many thanks,
Bryn
I have a little problem that I think I solved with a sledge-hammer andthought I'd post here to see if there was not perhaps a more elegantand probably quicker solution.Essentially I have a unique display condition on a table that mustreturn data in a certain way. The problem column in question is thesecond col of a compound PK which thus makes the data in this columnunique for any given col1 value. The problem is that regardles of whatdata is in col2, if one of the values is 'NO' ,then this MUST appearat the top of any resultset, query, or report that can be generated,otherwise standard alpha sort is fine (it is a varchar(30) field)My inelegant solution was to create a third column in the table, achar(1) that can only ever have 2 values, a '1' or a '0'. So allentries for col2 which are 'NO' get assigned the value 1, all othersare 0 so when you sort using this extra column based on any col1value, 'NO' always gets to go on top.Can I lose the extra column which is there purely for sorting/logicand thus is irrelevant to the table data or its meaning(<hangs head indue deference to normalisation>) It obviously works OK the way it is,but it makes all queries/sprocs cumbersome because they always havethe overhead of sorting an additioal column, or an inefficient indexwhere nearly all the values in a vital column are always '0' except teodd few, also all the updates and inserts need to be wary of changesor additions that are make a 'NO' value for col2 and thus reset or setthe sort column to a '1'.Some sample DDL and values below.CREATE TABLE [dbo].[tmpTest]([colA] [varchar](30) COLLATE Latin1_General_CI_AS NOT NULL,[colB] [varchar](30) COLLATE Latin1_General_CI_AS NOT NULL,[colN] [varchar](50) COLLATE Latin1_General_CI_AS NULL CONSTRAINT[DF__1] DEFAULT ('UNK'),[Reference] [char](1) COLLATE Latin1_General_CI_AS NULL DEFAULT('0'),CONSTRAINT [PK_tmpText]PRIMARY KEY CLUSTERED ([colA] asc , [colB] asc))ON [PRIMARY]insert into tmpTest (colA,colB,colN) values ('1','a','z')insert into tmpTest (colA,colB,colN) values ('1','b','z')insert into tmpTest (colA,colB,colN) values ('1','c','z')insert into tmpTest (colA,colB,colN,reference) values('2','NO','z','1')insert into tmpTest (colA,colB,colN) values ('2','a','z')insert into tmpTest (colA,colB,colN) values ('2','b','z')insert into tmpTest (colA,colB,colN) values ('2','c','z')insert into tmpTest (colA,colB,colN) values ('2','d','z')select * from tmpTestwhere colA='2'order by colA,reference desc,colB
View 1 Replies View RelatedHi,
I have a question regarding a project I am working on and I'm not sure what the best way to do it may be, I am new to SSIS and would like to gain some exposure using it, but don't know if this project is overkill or even possible using SSIS.
*** PROBLEM ****
Basically I have a large number of flat files each file contains different columns and data. What I want to do is import all of the data from the flat files into corresponding tables in SQL Server. The tables in SQL server are already defined with data structure to match each file. The first 10 characters of the flat file are also the same name as the SQL table. So it seems easy enough to just import flat file to table one by one, but I cannot figure out how to match the flat file structure to each table without setting up a task for each table (there are roughly 50-60 tables). Another reason I want to do it with SSIS is these flat files have to be imported on a weekly basis.
**********************
Is it in my best interest to just create a stored procedure that basically loops through the files and imports the data? Or do you think using SSIS is the best solution? I have looked at SSIS a bit and thought maybe I could use the bulkinsert task, but it appears it cannot dynamically change due to column specs. I figured out how to loop through the flat files using the foreach task, but I could only get it to import into one SQL table. I also started looking at the dataflow task and thought about creating a script task, but I'm not sure if that would work before I spend a lot of time trying to figure that out.
Any suggestions or thoughts are appreciated.
Thank you
Hey, I've a few jobs which call SSIS packages. If I run the SSIS package, it runs fine but if I try to run the job which calls this package, it fails. Can someone help me troubleshoot this issue? None of my jobs that call an SSIS package work. All of them fail.
Thank you
Tej
Hello,
Plz explain me what are the key factors or conditions one should consider while defining indexes on a field.
Regards,
Shailesh
I have a scenario in which a schedule is recorded like the top table below. Notice the start and end times, the meeting length, and the fact that you could book more than 1 meeting (book factor) during the times slot. The second table is the result needed. I have it working using the dreaded cursor, but I know there's got to be a more elegant solutions.
empID
bookFactor
mtgLen
mtgStart
mtgEnd
1
2
15
7/1/2015 8:00
[code]....
Hello All,
I am new to SQL Server 2000. I am eager to learn what factors/parameters are key for obtaining good retrieval performance of SQL Server 2000 (prompt response to user query).
I recall that someone told me that a recordset with asOpenStatic cursor type has higher speed than that of a recordset with other cursor types.
Is this true or false. Are there really some key parameters for perfomance tuning .
Thanks
I'm currently working on a project at work to test the effects of database compression, trying to obtain measurable data on the impact of the compression on other server resources, and therefore whether the reduction in space used is worth the extra overhead. This has involved taking a trace of a production customer's workload for a period of time and replaying it against a backup using Distributed replay in synchronised mode.
I'm then taking a trace of that replay, as well as using perfmon to record useful data about the server, before and after compression is enabled. Finally, I'm loading the traces into a tool called Qure to analyse the impact of the compression on reads, writes, CPU, overall duration etc.
What I'm finding is that even across 2 different 'baseline' runs, which are replaying the exact same workload against the exact same database, performance etc differs to a significant enough degree that it calls into question the validity of the test. I can only put this down to the fact this server is on a VM, which is affecting available resources, which in turn affects execution plans the workload is generating and causes different replays of the same workload. I'm therefore looking at doing this on a standalone server, but I still can't be sure the differences will go away.
How to make tests such as this as similar as possible on multiple runs, when elements outside of SQL Server are in effect out of my control?
Hi,
In our project we have two SSIS package.
And there is a task (Execute SSIS package) in First package that calls the execution of second package.
I m continuously receiving an error "Failed to decrypt protected XML node "PackagePassword" with error 0x8009000B "Key not valid for use in specified state.". You may not be authorized to access this information. This error occurs when there is a cryptographic error. Verify that the correct key is available."
As we are running first package by job, job runs successfully logging above error
The protection level of second package is set to "EncryptSensitiveWithUserKey"
Can anybody please suggest how to handle it?
I have a table with 23 columns. 8 columns are not always filled by users and so valued to default. Would this affect the overall performance of querying this table? Should I separate those 8 columns and link with a one-to-one relation?
Thanks
I have a table "PageInfo" wich has columns "PageID, Title(has "AboutUs" as one of the values), DateModified, Active". My other table is "AboutUs" and has columns "ID, WhatsNew, Welcome, Active".
My goal here is that if table "AboutUs" is affected as an "INSERT" or an "UPDATE" on any row, I want "PageInfo.DateModified" WHERE "PageInfo.Title = AboutUs" to be updated with "getDate()". Don't know if I'm clear enough but thanks in advance.
Gazzou
I have the following code:
Code Snippet
INSERT INTO [CICC].[dbo].[972createtest]
SELECT
Always81,
AccountNumber,
CAST(SUBSTRING(DateOfPayment,5,2) + SUBSTRING(DateOfPayment,1,2) + SUBSTRING(DateOfPayment,3,2) as datetime), CAST(TotalPmtAmt AS decimal(9,2))/100,
Collector,
PmtCode,
CAST(AmtPdToAgency AS decimal(9,2))/100,
CAST(AmtPdToClient AS decimal(9,2))/100
FROM [CICC].[dbo].[972create]
I'm trying to insert data into the table called 972createtest. It has the following columns and datatypes:
[Always81] [smallint] NULL,
[AccountNumber] [nvarchar](8) NULL,
[DateOfPayment] [nvarchar](6) NULL,
[TotalPmtAmt] [decimal](11, 2) NULL,
[Collector] [nvarchar](3) NULL,
[PmtCode] [smallint] NULL,
[AmtPdToAgency] [decimal](11, 2) NULL,
[AmtPdToClient] [decimal](11, 2) NULL
And here's a sample of the data stored in those columns:
81 01068713 092107 40.00 NJW 31 4000.00 0.00
81 01068713 111207 2000.00 NJW 31 2000.00 0.00
81 01068784 110806 10000.00 KSD 31 10000.00 0.00
81 01068784 121506 10000.00 KSD 31 10000.00 0.00
81 01068784 030507 10000.00 KSD 31 10000.00 0.00
81 01068784 033007 14100.00 KSD 31 14100.00 0.00
81 01068784 051807 7400.00 KSD 34 7400.00 0.00
Everytime I try to run the code, I get a message telling me "0 row(s) affected". What stupid thing am I doing wrong?
I am trying to use SQL Express to support unit testing of functionality which requires database access.
I created an express database (.mdf) which contains a database snapshot which is the database environment that each test expects to see when it starts. Through the execution of the test the database will be modified arbitrarily and I need to reset the database to the original state before the next test executes.
Here's what I do today:
Before Test:
The original database is copied to a unique location in the temp directory
A unique database name is generated (GUID-based)
I connect to the new database file with:
Server=.SQLEXPRESS;AttachDbFilename="PathToTempFile";Database="GuidDBName";Initial Catalog="GuidDBName";Trusted_Connection=Yes;User Instance=true
During the test all database access is routed to the temporary database GuidDBName
After Test:
Close all active database connections and clear the connection pool
Connect to the master database and detach the database:
Server=.SQLEXPRESS;Database=master;Initial Catalog=master;Trusted_Connection=Yes;User Instance=true - sp_detach_db 'GuidDBName'
Delete the temporary file
When I do this, everything seems to work correctly but when I re-open the original database it has changed unexpectedly and includes the modifications that were made by the test as it ran.
Can anyone explain why this is happening and/or provide a solution?
Situation: In this stored procedure, I have to calculate in some manner: Font, FontSize, BoldText, ShowBox and number of characters to see how many lines it will take on a Crystal Report. Wondering if you have seen some like this on Web or have an ideas? Measurements(length, width) and character count seem appropriate. How about a function?
Uses Arial and fontsize can be 8pt - 28pt.
Thanks. TAEG
how long does it take to convert a DTS package to ssis package?
View 9 Replies View RelatedIs there a way to copy the structure from one database to anotherwithout affecting the actual data?For example, I added new fields and stored procedures to a db in mydevelopment environment. There are a lot of them.I now want to make the same changes to my production environment db butdon't want to affect the data. Only the fields, stored procs &constraints and stuff pertaining to the structure of the db should bechanged.Does anyone know of a program that can do this or can this be done inMS SQL Enterprise Manager.Any help would be appreciated.7078895
View 2 Replies View RelatedHi,
I would like to know how to add SELECT row to a table variable. It's not for my SELECT syntax(code following is just an ugly example) that I want help it's for the use of table variable.
Your help will greatly appreciate!!!
ex :
DECLARE @MyTestVar table (
idTest int NOT NULL,
anotherColumn int NOT NULL)
SET @MyTestVar = (SELECT idTest, anotherColumn FROM tTest)-- This cause an error :-- Must declare the variable '@MyTestVar'. ???? What?
I have a SSIS job, one of the last steps it performs is to execute a SQL 2000 DTS package. This has to be done as a SQL 2000 DTS package as it is performing rebuilds of SQL 2000 Analysis Services dimensions and cubes. We've found that when the DTS fails the SSIS job is happily completing showing as a success, we would prefer to know it went wrong.
As far as I'm aware SSIS merely starts the DTS off and doesn't care about it's result. I've taken a look in to turning on the logging for the execute DTS package and thought that the ExecuteDTS80PackageTaskTaskResult would give me the answer I need...but is merely written to the log not available as an event-handler. It also looks like it is not safe to put a SQL task in as the next item to go look at the SQL 2000 system tables to look at the log for the DTS package as the SSIS documentation warns that the DTS package can continue to run after the execute DTS package task has ended.
Ideally I want any error raised within the DTS package to cascade up to be an error in the SSIS job, I can then handle it appropriately. I cannot find a way to do this. Is there a way?
If not, can anyone suggest how in the remainder of the SSIS tasks I can be sure that the DTS has completed before I start any other tasks that will check for the SQL 2000 log of its execution?
Hi,
I have developed an SSIS package for ETL purpose. I am invoking the SSIS package through .Net console application by referencing the ManagedDTS Assembly. I am able to execute the package in Sql Server 2005 Developer Edition and it runs fine till completion.
But when i try to execute the packahe in Sql Server 2005 Standard edition, by invoking the package through .Net console application the status of the package is failure.
Can any one help me how to over come this problem.
Hi All,
I am in the process of moving from a 32-bit SQL Server 2005 Enterprise (9.0.3054) to a 64-bit SQL Server 2005 Enterprise (9.0.3054 with 4 CPUs and 8GB of memory on Win 2003 SP2) and the process has been very frustrating to say the least. I am having a problem with packages that I created on my 64-bit SQL Server. I am importing a few tables from the 32-SQL Server into the 64-bit SQL Server using the Task --> Import to create the package.
Sometimes when I am creating a package I get the following error in a message box:
SQL Server Import and Export Wizard
The SSIS Runtime object could not be created. Verify that DTS.dll is available and registered. The wizard cannot continue and it will terminate.
Additional information: Attempted to read or write protected memory. This is often an indication that other memory is corrupt. (System.Windows.Forms)
Other times when I run a package that has run successfully before I get the following error:
Faulting application dtexecui.exe, version 9.0.3042.0, stamp 45cd726d, faulting module unknown, version 0.0.0.0, stamp 00000000, debug? 0, fault address 0x025d23f0.
Other times I get this error message:
.NET Runtime version 2.0.50727.1433 - Fatal Execution Engine Error (79FFEE24) (80131506)
And still other times
The package appears to hang when running. By this I mean that the Package Execution Progress shows progress up to a point then it just stops. (The package takes about 17 seconds to run normally) CPU usage is at 1% and the package cannot be stopped.
I have deleted and re-created the package several times and I have also re-installed the service pack on the SQL Server (9.0.3054) but that did not help.
Does anyone have any other suggestions to try?
Thanks.
I would like to standardize SSIS development so that developers all start with the same basic template. I have set it up so it is an available template ( http://support.microsoft.com/kb/908018 ) but I would like it to be the default when a new project or package is created. Is this an option?
View 4 Replies View RelatedI would like to fetch the data flow component name while package is executing. Since system variable named [System::SourceName] only fetches name of the control flow tasks? Is there a way to capture them?
View 5 Replies View RelatedHi there
We have a SSIS run which runs as follows
The master package has a configuration file, specifying the connect strings
The master package passes these connect-strings to the child packages in a variable
Both master package and child packages have connection managers, setup to use localhost. This is done deliberately to be able to test the packages on individual development pc€™s.
We do not want to change anything inside the packages when deploying to test, and from test to production. All differences will be in the config files (which are pretty fixed, they very seldom change). That way we can be sure that we can deploy to production without any changes at all.
The package is run from the file system, through a job-schedule.
We experience the following when running on a not default sql-server instance (called dkms5253uedw)
Case 1:
The master package starts by executing three sql-scripts (drop foreign key€™s, truncate tables, create foreign key€™s). This works fine.
The master package then executes the first child package. We then in the sysdtslog get:
Error - €œcannot connect to database xxx€?
Info - €œpackage is preparing to get connection string from parent €¦€?
The child package then executes OK, does all it€™s work, and finish. Because there has been an error, the master package then stops with an error.
Case 2:
When we run exactly the same, but with the connection strings in the config file pointing to the default instance (dkms5253), the everything works fine.
Case 3:
When we run exactly the same, again against the dkms5253uedw instance, but now with the exact same databases defined in the default instance, it also works perfect.
Case 4:
When we then stop the sql-server on the default instance, the package faults again, this time with
Error - €œtimeout when connect to database xxx€?
Info - €œpackage is preparing to get connection string from parent €¦€?
And the continues as in the first case
From all this we conclude, that the child package tries to connect to the database before it knows the connection string it gets passed in the variable from the master package. It therefore tries to connect to the default instance, and this only works if the default instance is running and has the same databases defined. As far as we can see, the child package does no work against the default instance (no logging etc.).
We have tried delayed validation in the packages and in the connection managers, but with the same results (error).
So we are desperately hoping that someone can help us solve this problem.
Thanx,
/Nils M - Copenhagen
hi,
I am interested in Passing value from a child Package variable to the Parent package that calls it in ssis.
I am able to call the Child package using the execute package task and use Configurations to pass values from the parent variable to the child, but I am not able to pass the value from the child to the parent.
I have a variable called datasetId in both the parent and child. it gets computed in the child and needs to be passed to the parent...
Any suggestions?
Thanks for any help in advance..
smathew
Deployed Report having SSIS package as source do not work when Indirect Package configuration is used in ETL package. It seems ETL package when called/executed from Report manager does not recognize environment variable to pick up the dtsconfig file.
The Report works when Direct package configuration is used to same dtsconfig file.
What could be the reason? Any solution for this? This will cause our build/deployment to QA and Prod very difficult.
Dear All,
I now have two SSIS package, "TESTING" and "LOADING". The "TESTING" package have an execute package task that call the "LOADING" package. When I want to execute the TESTING package, how can I setup the connection string so that I can edit the password of the database connected by the "LOADING" package?
Regards,
Strike
I have a ordersystem with repeating orders. Each order has a orderline
with a frequency and a intervaltype (day, week, month and so on) for
indicating how often a task should be done.
When an task/job is done, an equivalent receipt with receiptlines is
generated.
When a new job/task should be suggested, it has to calculate the
shortest next intervallength for each repeating orderline. This
intervallength must then be added to the date of the last receiptline.
I want to to this all in a SPROC. My result should be the one next
orderline that should be carried out with a date for when this is.
RESULT
OrderID TaskID TaskDesct EstDate
5000 GETGROCERIES Pick up groceries for miss Lama 19.12.2003
************************************************************************
Is this possible to do all in a SPROC and not having to do any in .NET
(which this is developed in), and can you help me with some ideas?
************************************************************************
To se my starting point, se the diagram at:
http://www.promotion.no/div/diagram.gif
My mind is stuck and this is so far I've reached:
ALTER PROCEDURE dbo.GetNextOrders
AS
CREATE TABLE #Interval
(
ivID int IDENTITY(1,1),
ivTaskID char(20),
ivTaskDescr nvarchar(200),
ivHour int
)
INSERT INTO #TemporaryTable
SELECT RepID, RepIntName
FROM RepeatInterval
SELECT InvoiceDetails.iID, InvoiceDetails.ReceiptID,
InvoiceDetails.OrderID, InvoiceDetails.TaskID,
InvoiceDetails.TaskDescr, InvoiceDetails.DateRegistered
FROM InvoiceDetails LEFT OUTER JOIN
Orders ON InvoiceDetails.OrderID = Orders.OrderID
WHERE (Orders.DateStart < GETDATE()) AND
(Orders.DateEnd > GETDATE() OR Orders.DateEnd IS NULL)
to do a database repair on SQL server v7.
This is what I did
dbcc checkdb ('mydatabase', REPAIR_REBUILD) with estimateonly
It gave me error ....
estimateonly is not an option.
What did I do wrong? Thank you for your help!!!!!!!!!!
I would like to ask some help for estimating necessary hardware for a database, I can tell you about usage profile the following:
- 2 tables, approximately 100.000 records, the tables joined in 1:1 relation, one record is approximately 500 bytes long
- typical scenario:
- select what returns 1 or zero record
- if it returned 1 record an update performed on this record
- the event logged in a third table (one insert)
- this typical scenario needs to run 100-300 times in every second
Can you tell me any advice on sizing of hardware for this? We can rely on a well designed database structure (proper indexes, etc...)
Thanx in advance
We are approaching the "go live" date of a big application upgrade, so while the application is being upgraded i was asked to lock out all users and backup all databases and make sure the data isnt changed during a period of time
It turns out, as one of my databases is being replicated to another server, i cant put that database in read_only mode, or restricted_mode, without removing the replication.
Removing the replication means i have to set it up again and that means another 3 hours added to an already cram-packed weekend without sleep!
How I can lock out the users to make sure there are no changes to a database other then read_only/restricted_mode/single_user ?
I really don't want to touch that replication...
I have two SSIS packages in a project, one calling the other. The parent package works fine in my local mechine. After they are deployed to the production, I schedeul jobs to run the packages in the SqlServer. The child package works fine if I run it alone, but the parent package could not find its child package if I run the parent package . As I checked, all xml config files and the connection string pointing to the child package were set correctly. It seems the parent package did not use the xml config file. Can someone help me? Thanks in advance.
View 9 Replies View Related
I have successfully created a SSIS package which execute a DTS 2000 package and with no problem to execute the task. But I failed to schedule this package. I was not success in setting the logging. When running the package in command line:
dtexec file "C:Documents and SettingslyangMy DocumentsVisual Studio 2005ProjectsTraingDTSTraingDTSDTSTraining.dtsx"
Error: 2008-03-24 08:03:24.36
Code: 0xC0012024
Source: Execute DTS 2000 Package Task
Description: The task "Execute DTS 2000 Package Task" cannot run on this edit
ion of Integration Services. It requires a higher level edition.
End Error
Warning: 2008-03-24 08:03:24.38
Code: 0x80019002
Source: DTSTraining
Description: The Execution method succeeded, but the number of errors raised
(2) reached the maximum allowed (1); resulting in failure. This occurs when the
number of errors reaches the number specified in MaximumErrorCount. Change the M
aximumErrorCount or fix the errors.
End Warning
DTExec: The package execution returned DTSER_FAILURE (1).
Any help will be greatly appreciated.
(32 bit machine, standartd edition of SQL 2005)