Project Real TestHarness.dtsx - Synchronous Or Asynchronous Process ????
Jan 8, 2008
Hi,
Can someone explain how the PR testharness is supposed to function? When I run it f with and without debugging and by using dtexec other processes can start and stop before the awaiting response from the console date and number of days to process. If you just look at the testharness and the loadgroupfulldaily the increment date and SQL Audits will finish prematurely before the console data is even entered. Is there a property that I am missing?
Thanks,
Larry
View 1 Replies
ADVERTISEMENT
Jan 3, 2008
Can someone please clarify:
If you have a data file, and you only want CERTAIN rows to pass to the destination, ie) a table
and you are using a script task to accomplish this,
is this a synchronous or asynchronous transformation?
Q. And how do you assign the values to the output? Do you have to create output columns, or not?
I am very very confused right now. I can't seem to find a decent answer to what is a very basic question either in my SSIS book or in the documenation. Perhaps it is so basic, that the question doesn't seem valid? I don't know. But I just don't understand this at all.
Thank you
View 9 Replies
View Related
Mar 19, 2008
Porting an existing SQL 2k DTS job over to a SQL 2k5 SQL Server running SSIS.
Background:
The job loads data into an empty work table and performs some work before clearing out the work table.
This job runs every minute.
Question:
If the job happens to take longer than a minute, does SSIS create a second instance of the job?
Or perhaps it does what DTS did and reschedules the job for the next iteration?
Concern:
I need to know because there would be key contraint violations if another instance of the job started before the working table was cleared out.
Thanks in advance
View 1 Replies
View Related
Jan 15, 2008
I'm creating a script component that reads from an OLEDB source and writes to an OLEDB destination.
For every input row, I need to output several rows.I tried using the Row.DirectRowToOutput0() method inside a loop in the
Input0_ProcessInputRow routine but that's not working. Should I be using Addrow() instead? If I use Addrow() does this mean it needs to be an asynchronous transformation?
I remember seeing a blog entry (Jamie's?) that did almost exactly what I wanted, but I can't find it now.
Any pointers appreciated
View 13 Replies
View Related
Nov 10, 2014
We are seeing high number of hadr_sync_wait types on our server after setting up AOAG during peak times. We have setup sync type as synchronous commit and failover automatic. Can we change these settings to async and manual failover whenever we need and change them back to sync commit during off peak timings. Any drawbacks because of these changes ?
View 4 Replies
View Related
Apr 20, 2015
I've got an SSIS solution file with project deployment model in VS 2013 and would like to deploy that to SSISDB on different environments.All these days I followed the regular way to create a project in SSISDB and deploy it to that. Now want to find out if i can automate this process and so got some questions
1. Can we automate the process of creating a project on SSISDB based on our SSIS project name? This will be like when we do a deployment it should check if the project exists or not on SSISDB based on our SSIS project name, if the project exists we just deploy the packages in the project and if the project does not exists in SSISDB it will create that project and deploy the packages.
2. Can we also automate the process of creating environments? In traditional way we manually create the environment variables under environment tab of SSISDB, but can we make that also as part of deployment? Like when we are releasing to Dev server we look if that particular Dev variable exists on that server, if it exists we just update the existing stuff and if it does not exists we just create it.
View 2 Replies
View Related
Nov 14, 2007
Hi Experts,
When you create a SSIS Project in Visual Studio and start adding many, many SSIS and several people do it, they are not being sorted when they get loaded into Visual Studio environment, unlike other objects such as web forms in ASP.NET or classes. as you can imagine, if you have about 100 of SSIS, it makes it very hard for anyone to find one SSIS to alter it.
all these packages are being stored in some xml file, right? so, my boss wants me to write a tool that will open this file and order SSIS by names to make it easier for anyone in a team. i was wondering if anyone has done something like this already, or started. any leads, comments will be appreciated.
thanks a bunch!
View 3 Replies
View Related
Jun 29, 2007
i have sql server 2005 dev and bids installed. i want to create a dtsx package, but when i go to File>new>project the template to create an interogation services package is not there. the template for Olap projects is missing also. it used to be present before i upgraded my system to Vista. Could this be a vista issue or do i just need to install a patch or something simple like that?
View 8 Replies
View Related
Mar 19, 2007
Most of the installation went well. I got to the point of attaching the
source database, but had problems attching the warehouse database:
Database 'REAL_Warehouse_Sample_V6' cannot be started in this edition
of SQL Server because it contains a partition function 'pf_Range_Fact'.
Only Enterprise edition of SQL Server supports partitioning.
I am running standard edition at mey development workstation, the only
one we have running. I do not necessarily need to see partitioning
working. But sounds like I will need to install Enterprise edition at
my development computer to go down this path of discovery? Anyone been
able to do otherwise?
Is there any way to get in and look at the stored procedures at least, or schemas?
Even after going back, it was not clear to me from the prerequisit list that Enterprise edition was required for this.
View 3 Replies
View Related
Dec 9, 2007
Hello,
I am reading Microsoft REAL project ETL part. I find that it uses neither transaction nor checkpoints throughtout the packages. I am wondering if something goes wrong with the process and the process aborts in the middle,how are we going to handle this. Any particular reson not using transaction? Is it best practice without using transaction for performance consideration and log volumn? I am new to SSIS. I would like to follow the best practice. Thanks for advice..
Regards
Jerry
View 6 Replies
View Related
Feb 23, 2007
Hi, I have a package that I am running concurrently with different values. and sometime I'm getting following message for one of the running instances and it fails while the other ones are running.
Execution failed with the following error: "ERROR : errorCode=0 description=Failed to save package file "C:Documents and SettingssmillerLocal SettingsApplication DataMicrosoftSQL ServerSmoInnerPackage.dtsx" with error 0x80070020 "The process cannot access the file because it is being used by another process.". helpFile=TableTransferGenerator Task helpContext=0 idofInterfaceWithError={8BDFE893-E9D8-4D23-9739-DA807BCDC2AC}".
Any ideas what is the reason I have search everywhere for keyword InnerPackage.dtsx as this is not my package name and it is getting created by SSIS enginee
Thanks
View 16 Replies
View Related
Feb 14, 2006
Hi,
We have a DTSX in which parallel threads are processing ETL. There is a Final step in the DTS to do cleanup tasks. we are getting the following error after ETL completes. This is not consistently reproduced.
Description: An OLE DB error has occurred. Error code: 0x80004005. An OLE DB record is available. Source: "Microsoft SQL Native Client" Hresult: 0x80004005 Description: "Login timeout expired". An OLE DB record is available. Source: "Microsoft SQL Native Client" Hresult: 0x80004005 Description: "Unable to complete login process due to delay in prelogin response". An OLE DB record is available. Source: "Microsoft SQL Native Client" Hresult: 0x80004005 Description: "Shared Memory Provider: Timeout error [258].
Connection manager is set as Localhost.
Can somebody help us out here?
thanks
View 1 Replies
View Related
Feb 29, 2008
I am trying to load Project Real. I get warnings when opening the RecurringETL package. However I created the two environment variables it's looking for and have confirmed these variables are functional.
I am unsure how to troubleshoot this and am afraid to do anything given these warnings. Where is the configurations collection??????:
Warning 1 Warning loading TestHarness.dtsx: The configuration environment variable was not found. The environment variable was: "REAL_Configuration". This occurs when a package specifies an environment variable for a configuration setting but it cannot be found. Check the configurations collection in the package and verify that the specified environment variable is available and valid. C:Microsoft Project REALETLTestHarness.dtsx 1 1
Warning 2 Warning loading TestHarness.dtsx: The configuration environment variable was not found. The environment variable was: "REAL_Root_Dir". This occurs when a package specifies an environment variable for a configuration setting but it cannot be found. Check the configurations collection in the package and verify that the specified environment variable is available and valid. C:Microsoft Project REALETLTestHarness.dtsx 1 1
Warning 3 Warning loading TestHarness.dtsx: Failed to load at least one of the configuration entries for the package. Check configurations entries and previous warnings to see descriptions of which configuration failed. C:Microsoft Project REALETLTestHarness.dtsx 1 1
Thanks
View 8 Replies
View Related
Jun 5, 2006
Hi,
If you have two synchronous transformation components and the input of the second is connected to the output of the first, does the first transformation process (loop through) all rows in the buffer before outputting these rows to the second transformation? Or does the first transformation output each individual row to the second transormation as soon as it has finished processing it?
Thanks in advance,
Lawrie.
View 5 Replies
View Related
Mar 25, 2008
Hi List
Im trying to set up an implementation of Project Real --it works like this-
Create two system environment variables called REAL_Root_Dir and
REAL_Configuration with the values given below. Click on
Start -> Control Panel -> System. Go to the Advanced Panel, click Environment Variables button, then New in the System variables box.
If the Project REAL files were installed at C:Microsoft Project REAL, then the variable values will be:
Variable Name: REAL_Root_Dir
Variable Value: C:Microsoft Project REALETL
Variable Name: REAL_Configuration
Variable Value: %REAL_Root_Dir%REAL_Config.dtsconfig
The package OLEDB connections work like this
First read enviroment variable to get location of config file
Next read Config File to get connection string for Config Database
<?xml version="1.0"?>
<DTSConfiguration>
<Configuration ConfiguredType="Property" Path="Package.Connections[SQL - Configuration].Properties[ConnectionString]" ValueType="String">
<ConfiguredValue>Data Source=(local);Initial Catalog=DataWarehouseABC;Provider=SQLNCLI.1;Integrated Security=SSPI;</ConfiguredValue>
</Configuration>
</DTSConfiguration>
Next read Config database to get connection strings for Source and Destination databases
Destination database is called "DataWarehouseABC"
Source database is called "SnapshotABC"
the Source database OLEDB connection works 100%
however the destination OLDB connection we get this error below
PS--Both source and destination databases are on the same development machine , however both databases are restored bak files from another production machine
Error 1 Error loading LoadGroup_Daily.dtsx: SSIS Error Code DTS_E_OLEDBERROR. An OLE DB error has occurred. Error code: 0x80004005. An OLE DB record is available. Source: "Microsoft SQL Native Client" Hresult: 0x80004005 Description: "Login failed for user 'xxxxxx'.". An OLE DB record is available. Source: "Microsoft SQL Native Client" Hresult: 0x80004005 Description: "Cannot open database "DataWarehouseABC" requested by the login. The login failed.".
Any ideas on how one OLEDB Connection in this package can get this corruption
thanks in advance
Dave
View 5 Replies
View Related
May 12, 2008
Executed as user: xxxxxx-sql. Microsoft (R) SQL Server Execute Package Utility Version 9.00.3042.00 for 32-bit Copyright (C) Microsoft Corp 1984-2005. All rights reserved. Started: 4:14:05 p.m. Error: 2008-05-12 16:14:07.84 Code: 0xC00220DE Source: EPT Dimensions - x Description: Error 0x80070003 while loading package file "E:ETLLoadGroup_Dimensions_Daily.dtsx". The system cannot find the path specified. . End Error DTExec: The package execution returned DTSER_FAILURE (1). Started: 4:14:05 p.m. Finished: 4:14:07 p.m. Elapsed: 2.25 seconds. The package execution failed. The step failed.
Hi Im using project real framework and have BIDS installed on same server as SQL Server 2005, when I run from BIDs all
good when I run from SQL Server agent via File system --the first package that master package calls fails
I have double checked the environment variables in Project Real and all good because otherwise BIDs would have failed
and of course in each package the expression data uses @[User::RootDir] to point to the package file path
@[User::RootDir] + \LoadGroup_Dimensions_Daily.dtsx
the issue is the package under SQL Agent is ignoring @[User::RootDir] + \ and finding an old package path somewhere in Bids SLN metadata
Any clues how can refresh the metadata of the package for deployment
thanks in advance
Dave
View 3 Replies
View Related
Apr 1, 2008
dear list can anyone figure out a workaround as to why OLEDB Provider MSDORA cannot store passwords
I have all the info stored in a table (Project Real Best Practise)
The user id I have stored in string ConfiguredValue gets transfered to OLEDB Provider MSDORA named (SQL_REAL_Source_myoradb) but not the password
To workaround this bug only with MSDORA can aynonne sugest a setting I should use in package security
ie the default is EncryptSensitiveWithUserKey?
thanks Dave
CREATE TABLE [admin].[Configuration](
[ConfigurationFilter] [nvarchar](255) NOT NULL,
[ConfiguredValue] [nvarchar](255) NULL,
[PackagePath] [nvarchar](255) NOT NULL,
[ConfiguredValueType] [nvarchar](20) NOT NULL
) ON [PRIMARY]
ConfiguredValue = Provider=MSDAORA.1;Password=abc;User ID=myuserid;Data Source=myoradb;Persist Security Info=True
PackagePath = Package.Connections[SQL_REAL_Source_myoradb].ConnectionS
View 5 Replies
View Related
May 5, 2006
I can create an external text file from within the SQLCLR project, but I cannot run an external executable. Just in case you are asking, I need to do this to push data into a legacy application using a different DB format. I have found it best to simply use my old language (Clipper) for data validation, etc. - and especially since I require multiple indices to be open. So, if you could just take my word on this.
The following code:
Process newProcess = new Process();
string path = @"C:TEST.BAT";
newProcess.StartInfo.FileName = path;
newProcess.Start();
Executes without error, but does not actually run the external.
Now, I can have a DOS (Clipper) application poll a directory for text files, but I am trying to get away from all these "mini" data transformation applications. If an exception is caused in the DOS app, a remote user on the other side of the country has no idea it is broke and his data (coming from a SQL Mobile Device) never gets to our legacy database structure.
So, am I out of luck?
View 6 Replies
View Related
Jan 11, 2008
Hello,
At First sorry for my English.
I have a Problem with a SSIS Project with a Process Task.
If I start the Project in Visual Studio then it work with no problems. But when I start it in a Order with
the Sql Server-Agent, its run infinitely.
There is no error message, the Server Agent says that the job work, I can see the .exe file in the task manager.
but it will no really executed.
What can I do?
View 13 Replies
View Related
May 29, 2008
For some reason in a Team Foundation Team Project that has multiple project types (SSRS, SSIS, WebSite, C# Business DLL...), the SSIS project makes itself the startup project to the team project. If I explicitly set another project as the startup project to the team project and then select an SSIS package in the SSIS project in the team project, the SSIS project becomes the startup project automatically.
I am using Visual Studio 2005 SP1.
View 3 Replies
View Related
Jun 13, 2007
I have a dtsx package that is calling another dtsx package, however, if the called upon dtsx package fails with errors or what not, then the calling package does not continue as well. Is there any way to override this such that if the called upon package fails, the downstream actions in that package can stop, but the calling packages downstream actions to continue?
View 3 Replies
View Related
Jan 5, 2008
The following statement is from Microsoft documentation:
If you use the ExclusionGroup property to specify that rows should only go to one or another of a group of outputs, as in the Conditional Split transformation, you must call the DirectRow method to select the appropriate destination for each row. When you have an error output, you must call DirectErrorRow to send rows with problems to the error output instead of the default output.
I have a question about this because I have never used the "ExclusionGroup" property. For example, I have a script component where I specify 4 separate outputs, because I am sending different groups of rows to each output. I accomplish this programmatically using a lot of conditionals and it works fine.
I did not have to use the "ExclusionGroup" property to do this. So I'm not sure why I would ever need this, or to use DirectRow? I'm trying to understand this better, because maybe I feel like I'm not understanding the DirectRow, or how/when to use it.
Thanks
View 1 Replies
View Related
Jun 24, 2007
Hi,
I have been working on some SSIS packages for a while now and today while i was working i was trying to create a new connection and in the process there was an error and it said the BIDS has to be closed and i closed it but later when i open BIDS and try to open my project(.sln) from the file menu to work on the half done package it pops up an error which shows the path to my project location on the first line and next statement on the pop up error box says:
"Make Sure the application for the project type (.dtproj) is installed."
I tried to check some forums in which some suggested to try installing SP1 which i tried but ..i dont know why but the SP1 fails to install (i dont know if its causing problem becoz i already installed SP2 as i had some other problem before for which the cure was to install SP2).
Did anyone here face such a problem before ?
I'd really appreciate if the experts here can tell a cure for this problem.
thanks,
Ravi
View 4 Replies
View Related
Feb 23, 2007
I have a Visual Studio 2005 solution which contains a Sql Server Integration Services (SSIS) project.
In this solution I have explicitly set a Web application project as startup project, but whenever I edit a DTS package within the SSIS project, VS automatically sets the SSIS project as startup project and the package I edit as startup object.
Needless to say, this may cause some really unwanted incidents when running the solution - thinking that you're about to run the Web application project (that was explicitly set as startup project), but instead, you run the edited package in the SSIS project.
Is there any way to avoid having the SSIS project automatically setting itself as startup project, any workaround here at all? :)
Thanks.
View 4 Replies
View Related
Jun 16, 2015
in order to maintain a deployed project into an Integration Services Catalog I'd like to know if it is possible to import it into a new project inside SSDT.
View 5 Replies
View Related
May 6, 2007
Hi,
I want to update value of a custom field for a perticular project in Project Server 2007 using PSI.
I have created 5 enterprise custom fields(A,B,C,D,E) through PWA/Server Settings.
I want to search all Projects on Server. If any project is having value for custom field A then I want to update rest of the custom fields(B,C,D,E) for that perticular project.
I dont know how to do it please help.
Thanks in Advance
Madhukar
View 5 Replies
View Related
Feb 3, 2007
I have a very small project written in VB.Net 2005 using the SQL Server 2005 SSiS DTSx package.
I took a SQL Server 2000 dts package and using the SQL Server 3005 legacy tools migrated it so I could still use the package withing SQL 2005 until I can build one using BI/SSIS.
Anyway,I added the reference Microsoft.SqlServer.ManagedDTS so I could then use the Microsoft.SqlServer.Dts.Runtime so I can execute the commands:
Dim oApp As New Application
Dim oPkg As New Package
oPkg = oApp.LoadPackage(g_DTSx_Directory & "AOC copy Generic1 CSV to AOC_verify_file_1.dtsx", Nothing)
Dim oResults As DTSExecResult
oResults = oPkg.Execute
Ok. That works fine. Executes without a hitch. So now I try and create a setup project for this and I use the setup wizard.
During the creation of the setup project I get a message that states:
The following files may have dependencies that cannot be determined automatically. Please confirm that all dependencies have been added to the project.
C:windowssystem32msxml6.dll
OK. The dll is part of the reference I mentioned above and I have no idea what other dependencies it may have.
How do I find this out?
Has anyone else created a project like this and experenced the same?
I am on a clean build running WinXP Pro with SP2 - VS2005 with SP1 and the SQL Server 2005 tools.
View 1 Replies
View Related
Mar 31, 2008
I have visual studio 2005 and sql server 2005 with integration service installed on my machine. Couple of days ago, I installed visual studio 2008 professional. When I go to create SSIS project I dont see it in visual studio 2008. What do I have to do to make it appear in visual studio 2008 so that I can create SSIS projects.
Thanks.
View 7 Replies
View Related
Oct 1, 2007
I'm new to SSB, so please bear with me. Our application requirements are:
1) Web app gathers user input from a web UI.
2) Web app calls a stored procedure, passing in the user input gathered in step (1).
3) Procedure issues queries to multiple data sources (SQL Server 2005 db's) derived from the user input.
4) Procedure waits for replies from these multiple data sources, governed by a timeout in case a data source is unavailable or slow to respond.
5) Procedure aggregates the data returned in step (4) from multiple data sources into an XML document.
6) Procedure returns the XML document as an output parameter.
This is different than the usual SSB asynchronous application paradigm. In particular, I'm wondering:
How can I setup a synchronous dialog, where the procedure that issues the SEND waits for a reply from the target service? I don't want the initiator procedure to end after SENDing, but rather wait for the replies to those messages so it can aggregate the data from the reply message bodies.
Thanks - Dana Reed
View 4 Replies
View Related
Sep 25, 2006
HI, I need to know whether I am on the last row or not in my script component. If this is the case, I would alter a column in the row that indicates me that I am processing the last row. Is there a way to do it? I tried with process input but when the EndOfRowSet() indicates me that the last row is porcessed, I cannot alter the row in the buffer.
Thank you,
Ccote
View 4 Replies
View Related
Jul 23, 2007
I fail to use project professional 2003 to access to the project server 2003 using MSDE 2000 in local area network, following message was shown,
Connection failed:
SQLState: '01000' SQL Server Error 1326 [Microsoft][ODBC SQL Server Driver][DBNETLIB]ConnectionOpen (Connect())
Connection failed:
SQLState '08001'
SQL Server Error: 17
[Microsoft][ODBC SQL Server Driver][DBNETLIB]SQL Server does not exist or access denied.
I have seen these pages with similiar cases but can't help.
http://support.microsoft.com/kb/818047/
http://support.microsoft.com/kb/837653/en-us
Can anyone help me?? Thanks a lot!
View 1 Replies
View Related
May 24, 2007
Hi guys,in my db i have these three tables1.Stores 2.Products3.Partstheir structure is something like :Stores ----Products ----PartsStores----------------StoreId, StoreNameProducts----------------ProductId, StoreId, ProductNameParts----------------PartId, ProductId, PartNamenow, in my application i wanna to implement a bulk-copy operation souser can copy products from one store to another one and when aproduct copied to new store;all of it's parts should copy too.in fact i need a method to insert a Product item in Products table andsynchronously copy it's parts into Parts table and repeat this stepsuntil all of proucts copied.how can i do that without cursors or loops ?Thanks
View 19 Replies
View Related
May 13, 2006
I have synchronous mirroring. Some times I loose connection to witness and mirror servers. These times primary server is down. Is there any way I can change mirroring to asynchronous when primary server is down due to communication break down between witness and mirrored servers? I can break mirroring but to re-establish mirroring, I have to backup and restore on the other side. So if I can change mirroring to asynchronous when primary server is down due to connection breakdown between witness and mirrored server, then when witness and mirror servers come back, I don't need to restore the entire database. Ofcourse I could use asynchronous always but that does not failover automatically. I am thankful to all answers and suggestions. Thanks.
View 3 Replies
View Related
