I am working on a configuration database in SSIS. One of the modules in the package is giving me endless amounts of grief. The module is for some reason set up to add data onto the end of any data already stored in the table. Instead, I would like it to replace this data. I have tried an Execute SQL Task that should delete all of the rows in the table, but this isn't working. Is there a more efficient way to do this?
-Kyle
Hello,
I created a SSIS project with some SSIS packages within my local machine. Once all development and testing stuff was finished I imported the same to SSIS package store within Integration services. Then I created another test folder within my local machine and copied all the packages along with the project .sln file to that test folder.
Now the problem, If I make any changes to the package within test folder it automatically saves the changes to my other folder. Does anybody have a reason why it is doing so.
Thank You
Jatin
We have an SSIS package that was created to migrate data in from a few production databases. The steps for the package are as follows...
backup databases on server 1 (prod database server) restore database to SSIS server (server 2) .
truncate worker tables in SSIS server's (server 2) Main DB database.
copy data from restored db tables to working db tables ( database to database)
Start Multiple threads (15 ) and run steps from here in parrallel
Combination of Data flow tasks and SQL scripts and Stored procedures used to flatten data out and combine data for reporting purposes.
The average run time is 8 hours.
the issue we are seeing is this, the package will fluctuate in run times from 4 hours to over 11 with no change in the data or the underlying SSIS package. We have looked for any changes or things that would effect this but have not found anything that changed...
Also, certain steps are running shorter while others double in time. there doesnt seem to be any rhyme or reason to this behaviour. The server is x64 12GB of RAM 2 dual core 3.2Ghz.
Please let me know if you need any more information or specifics...
the only thing I have seen so far that looks out of place is Tempdb has one of its files that is 20+GB.
Thanks,
Chris
Hello,
I am fairly new to SSIS and do not have a wealth of 'in-depth' knowledge of it and I am running into some problems.
I am part of a team that is using Agile techniques (Scrum based) to implement a reasonably complicated system. This system has the requirement for numerous file feeds and multiple transactional database instaces whos base data is driven from a master seed database.
We are getting to a point where we are starting to hook up alot of the infrastructure and I am running into problems. Especially with the continous evolution of the database. Every iteration, the SSIS packes break because of DB changes. This is a little scary to me because I am starting to see SSIS as adding multiple points of failure to the system. Agile is based on refactoring often and that includes the database for this project. Every database change either breaks or requires a change to the SSIS packages that moves data from file feeds and staging databases to the master seed database. This problem is even more pronounced when considering the processes that move data from the seed database to the transactional databases. It discourages changing the database which is a really bad thing because that can lead to code archtecture smells and adds fragility.
Anyone have any input on this subject?
[VS 2005]
I have a mixed solution with a C# project and an SSIS project. Now matter how many times I set the C# project as the startup project, it seems that as soon as I make a change to the SSIS package, it decides that it's so unbelievably important that it HAS to be the startup project. Is there any way to disable this?
Hello.
I have packages that must generate log errors dynamically including time of execution and the name of the task.
I make it changing the properties of the connection inserting two expressions.
1.-I alter the File Usage Type to 1 to generate this files.
2.- I alter the connection string as: @[User::myvariable] +"constant_description"+ time description+ +".txt"
The time description is :
(DT_STR,40,1252) DAY (GETDATE())+"-"+(DT_STR,40,1252) MONTH( GETDATE())+"-"+(DT_STR,40,1252) YEAR( GETDATE())+" + REPLACE( (DT_STR,10,1252) (DT_DBTIME) GETDATE(),":","_")
But it is not the problem
In those packages I have one connector for all the tasks and in execution time it creates one file for each of the tasks.
The problem is the way I insert in the filename the task name.
I have a pre-execute event handler in each task that modifies a string variable( myvariable) appending the task name. When I execute de package it works great but when I only execute a task, the program do not enter in the event and do not put the task’s name.
How can I put that name without using that handler? There is another handler can I use to do it that happens before the system generates the new file name and after pre-execute? Anyone knows another way to do this kind of things?
I would like to replace data of some tables from STG to DEV database daily using SSIS package. Should I use "Transfer SQL Server Objects Task" to do that? Thanks.
View 3 Replies View RelatedWe require to convert a list of SPs in to SSIS packages. Most of the SPs do the below steps:
mainly our store procedure r to have compare the present date to past date , and comparing emp id between the files and also some joins. updating table r take place.
I would like to replace data of some tables from STG to DEV database daily using SSIS package. Should I use "Transfer SQL Server Objects Task" to do that? Thanks.
View 5 Replies View RelatedI would like to standardize SSIS development so that developers all start with the same basic template. I have set it up so it is an available template ( http://support.microsoft.com/kb/908018 ) but I would like it to be the default when a new project or package is created. Is this an option?
View 4 Replies View RelatedHello, I have a table of data with the following columns:ParticipantResult (win or lose)The table has other data in, but I want to make a league to sort by the amounts of wins. For example: Participant Won PlayedA 5 10B 4 7C 3 3 I want this to show up on web page. I was hoping to do this through the source code on the web page, but I am struggling to do it.Is there a better way to do this, or a place where I can see an example of the code. Many Thanks, Oliver
View 1 Replies View Relatedwith SQL desktop engine installed where do I even begin working with it?
View 1 Replies View RelatedI hope I can explain this somewhat coherantly. Here goes..
I have a products database. I want to select product details for items having distinct style codes. IE if there are 20 products that different from one another in some way, and 3 different style numbers throughout them, I want to end up with 3 products selected. I don't care which ones, as long as i have a sample product for each style.
I've attemped to use GROUP BY to do this, but it's not giving me what I want. I managed to get right number of products, but the query seems to have mixed and matched the fields together in an undesireable way. I guess this kinda makes sense, since providing aggregate functions for each of the extra fields is necessary.
SO THE QUESTION IS: How do I select (number of distinct styles) products, and still keep the records in tact as they should be?
Here is a sample...
Code:
SELECT DISTINCT Min(team) as team, min(pict) as pict, min(colors) as colors, min(league) as league, min(product_description) as product_description, min(category) as category, min(subcategory) as subcategory, min(style) as style
FROM prods
WHERE (style='style1' or style='style2' or style='style3')
AND league='leaguex'
AND soldout=''
GROUP by style
ORDER by style
Thanks very much for any help!! I'm sure I'm going about this all wrong.
Hi,
What is the best method/model for having real time replication set up and also needing the ability to drop and recreate SP's/Tables/UDF's etc? I keep getting the error of the SP is being used in replication so it cannot be dropped...
Thanks in advance.
Adam
Hi alllI have these tables below with 3 fields each. I want to get the recordin table 1 whose field number value is same in table 2 but field number2 on both tables are different. I mean i want the record1500 800 2. Insight: Table 1 is modules ordered and table 2 is modulesdelivered. I want to get 1500 800 2.beacuse module 800 was ordered butin table 2 module 503 was delivered. can some one help me with a joinnice weekeendTable 110 5012 101600502100 502 31500800 2Table 21500503 114004000100502 10100600100502 3
View 3 Replies View RelatedPlease see the detail of tables with fields in SQL server 2000.·TableStudent .Table Good Qualities( It is a list of Students) (It is a list of Good Qualities)1-Student Id (P.Key)1- Serial No. (P. Key)2-Student Name 2- Good Quality· Table Bad Qualities .Table Future Plans(It is a list of Bad Qualities) (It is a list of Plans)1-Serial No. (P. Key) 1- Serial No. (P. Key)2-Bad Quality 2- Future Plans·Table-Personality1-Student Name/ID2-Good Qualities3-Bad Qualities4-Future PlansData will store in Personality table like thatStudent IDG.Q.No B.Q.NoF.P.No101 3 2 1101 4 5 NUll101 8 Null Null202 4 8 9202Null 3 1Required ResultNow I have to generate a report of One student andsometimes of many students comprising of their Name, G.qualities, BadQualities and Future Plan. To generate report User input will be onlystudent id.I Need help in making relationship of these tables as well as inMaking a Query. Any suggestion to get the required result. Speciallyplease look at the Personality Table because I do not understandPrimary key for it. Should I just include the Serial No. as Primarykey.Please make necessary changes to get the required result.I shall be really thankful to you for your kind favor.
View 3 Replies View RelatedHello Everyone,
I have a dimension that has the following members:
Directors
Senior Managers
Managers
Senior Associates
Associates
But they way they need to be displayed includes subtotals. Senior Managers and Managers are placed into Management and Staff is a total of everyone except Directors. They should appear on the report in something like this:
Directors 10
Senior Managers 15
Managers 9
Management 24
Senior Associates 17
Associates 40
Staff 81
Total 91
What is the best way to do this? Right now I am using a matrix because I am also showing headcounts by Period. Is a table a better way to go? Is it something that should be done in my dimension?
I can do the Total part. I am just not sure how to make the subtotals inbetween the other lines.
Thank you for the help.
-Gumbatman
how can i create a report for my database? how can i create bar graphs, etc.? im using msde and sql express. pls help. thanks.
View 3 Replies View RelatedI have been reading up on how replicating schema changes work, but I must be missing something. When I configured a database for replication, it added a rowguid column to each table. This I understand.
I want to test making a schema change and replicating that change. For example, I tried to add a column to an existing table. When I tried to save the table i get the following error:
"Error validating the default for column rowguid".
"- Unable to modify table.
It is invalid to drop the default constraint on the rowguid column that is used by merge replication.
The DDL operation failed inside merge DDL replication manipulation.
The transaction ended in the trigger. The batch has been aborted."
So, what am I missing? Do i need to drop the rowguid column, add the new column, and then re-add the rowguid column?
Many thanks...
Hi,
I have the following flow in which i am looping through a folder containing textfiles
Foreach loop
[
ScriptTask -> DataFlowTask -> Execute SQL
]
In my ScriptTask, i am checking the file validity. Now if the file is found incorrect, I do not want the DataFlowTask and other to execute. Instead i want the file to be moved to Error and the loop to proceed with the next file. If the file validity is correct, DFT and ExecuteSQL execute. How do i do this?
Some explanation supported by examples would help.
Lalit
hi
I am not able to create connection string through vwd express for sql express database file n getting a error:
An error has occurred while establishing a connection to the server. When connecting to SQL Server 2005, this failure may be caused by the fact that under the default settings SQL Server does not allow remote connections. (provider: SQL Network Interfaces, error: 26 - Error Locating Server/Instance Specified)
plz tell me where is the problem n how can i resolve it
Thanks n with regardsCN
Hi Guys, I came here again to ask something about my application, Is there a way to make 2 columns unique with each other even if they don't have a primary key? If there's a way how? I wanna make a rule so, my application wont replicate its record. Graciaz,Nhoyti
View 3 Replies View RelatedHi
I have no idea but want to learn it how to make random names with sql server...
I have a table, called table1, for colums Firstname and Last name
I want it to make random names, so much it is possible it can get, in table2 where the colums is named Names
PLEASE HELP :)
I have a datagrid in my file along with an Update Query.
My Update Query basically adds the numerical values in two columns
together when the page is loaded. This means whenever the page is
Refreshed the Update query is fired.
This is my Update Query (which is in an Stored Procedure):
UPDATE Rental
SET TotalFee = ExtraFee + TotalFee
WHERE DaysOverdue >= 0
I have declared my query in the 'Page_Load' part of the coding, because
I want the query to run automatically. Not manually by a button.
My main question is that how can I get the query to run only once a day, no matter how many times the page is loaded.
Playing with DTS for converting text data to SQL Server table and sending email messages. Package is to be scheduled hourly.
1. Is there any easy way to force DTS to choose Success or Failure based on a SQL Task? Wish to check contents of a table for new values and branch accordingly in DTS.
2. When new records are brought in I wish to build a custom email message rather than attaching a text file. Any Idea how to build a custom message?
I am attempting to auto scan an error log on a Unix based system and email myself when errors occur. This is an exercise to test scheduling of jobs, bridging between technologies and sending out meaningful messages based on the process via email.
Thanks,
Mike Hoyt
Can anyone help me retrieving that result.
take an example
EXEC-ID JOB-CODE
0001 20
0001 63
0001 03
i want result in the following format:
EXEC-ID JOB-CODE1 JOB-CODE2 JOB-CODE3
0001 20 63 03
Sorry if this is a total n00bie question but...
I have table A and table B
I want a field in table B to be equl to the primary key in table A, and i'm not sure how to do that.
Thanks
Hello Experts,
Can anyone tell me, how can I make SQL server database empty.I mean i need to keep only restore the table structur, no table data.
Is there any possible way to make a row delete itself at a certain date and time? I am tring to make an "On_Sale" table and perticular items(rows) must expire at a certain Date and time. I have an idea to delete a row when it expires but it will cause an extra burden on my application. So is it possible to have SQL responsible for deleting a row at a certain time and date?
Also, I have another question:
when I asign a PK Identity to a certain column is there a way I can enforce consecutive order of PK values? For example if I delete a row and the PK value was 5 it will reoder the whole table so all PK values are in consecutive order:
PK-----------Item----------price
1------------XYZ1----------$$$$
2------------XYZ2----------$$$$
3------------XYZ3----------$$$$
when I delete PK 2 this is what happens
PK-----------Item----------price
1------------XYZ1----------$$$$
3------------XYZ3----------$$$$
the PKs are not in consecutive order
What I want is this to happen: when PK 2 is deleted I need it to show as follows
PK-----------Item----------price
1------------XYZ1----------$$$$
2------------XYZ3----------$$$$
The PKs stay in consecutive order.
See how the PK are still in consecutive order?
hi. i'm trying to make a report in asp that sums up a whole bunch of statistics. the report is quite lengthy and takes about 2 minutes to load. i thought i would solve that problem by making a table to dump the statistics into instead of calculating the statistics every time the report is viewed. this way, the page would just read the table with the statistics already put in it. these statistics would be updated every day or two so i don't need anything up to the minute.however, i have run into a small problem which is actually updating the information in the table. my new table is called OfficeReport and it looks like this:UserID Stat1 Stat2 Stat3 ...... Stat3223 0 0 0 056 0 0 0 072 0 0 0 0this is a query for one of the statistics i am currently using:SELECT DefendantCase.UserID, COUNT(DefendantCase.UserID) AS CountOfUserID FROM DefendantCase LEFT JOIN UserDescription ON DefendantCase.UserID=UserDescription.UserID WHERE UserDescription.Status=1 AND UserDescription.UserType=1GROUP BY DefendantCase.UserIDif i run this query, this is what i get:UserID CountOfUserID54 2160 1052 294 427 22how can i modify this query so its output updates the same UserID column in the OfficeReport table? Thanks!
View 3 Replies View RelatedI have the following SQL statement that works out how many days are between todays date and the first of April 2007.
SELECTDATEDIFF(dd,'2007-04-01 00:00:00.000',GetDate()) AS 'Days Left'
This works fine, however: After the 1st April 2007 I want to start counting down the days till 1st April 2008 - and so on and so forth.
How can I do this? Hopefully you understand my question - if not, ask me any questions needed!
Cheers - GeorgeV
I have a table to which data is being imported via bcp. When I get this data there are some fields nvarchar that has some decimal, for instance 1098.000000 is always 6 zeros..What I want to achieve is to make the above number only 1098
So what I did was this:
LEFT (AMOUNT, LEN(AMOUNT)-7) AS AMOUNT
But i get an error saying invalid lenght parameter passed to the left or substring
i m having 2 Database files for the same DB and 2 Logfiles ,i want it to make a single file..... is it possible with DTS or any other thing.....
View 1 Replies View Related