Handling Encrypted Data In SSIS
Mar 11, 2008
Hi, I have source data in encrypted format. How should i handle it in SSIS?
I have found no information for such situation.
Anybody have any idea about it.
Bhakti
ADVERTISEMENT
SSIS - Handling Recursive XML Elements In Data Flow Task
May 22, 2008Hi All,
I have a requirement here to import data from XML file to SQL Database. The XML schema contains of various elements and one of the element is recursive ie. Parameter node contains parameter node within it and it can have n number of iterations. I have given the sample schema below:
<xs:element minOccurs="0" name="Parameter">
<xs:complexType>
<xs: sequence>
<xs:element minOccurs="0" name="ID" type="xs: string" />
<xs:element minOccurs="0" name="Description" type="xs: string" />
<xs:element minOccurs="0" name="Type" type="xs: string" />
<xs:element minOccurs="0" name="Parameter">
<xs:complexType>
<xs: sequence>
<xs:element minOccurs="0" name="ID" type="xs: string" />
<xs:element minOccurs="0" name="Description" type="xs: string" />
<xs:element minOccurs="0" name="Type" type="xs: string" />
<xs:element minOccurs="0" name="Parameter">
<xs:complexType>
<xs: sequence>
<xs:element minOccurs="0" name="ID" type="xs: string" />
<xs:element minOccurs="0" name="Description" type="xs: string" />
<xs:element minOccurs="0" name="Type" type="xs: string" />
</xs: sequence>
</xs:complexType>
</xs:element>
<xs:element minOccurs="0" name="Parameter">
...............
</xs: sequence>
</xs:complexType>
</xs:element>
</xs: sequence>
</xs:complexType>
</xs:element>
But all the nodes contain the data which has to be imported to a single table dbo.Parameters. I cannot use Union ALL since i dont know how many iterations I will have in the file. Is there any way to do this operation in Data Flow Task using XML Source? Can anyone help me on this?
Thanks,
Dhileep
Integration Services :: Handling Empty Text File Load Into Table Through SSIS Data Flow?
Jun 16, 2015We have created SSIS package to load a text file into a table. Source system shares 10 text files and recently they stopped generating data for one of the text file (comping empty), after few months they will start generating the data for the empty file batch processing.
The Issue here is Data Flow task is getting failed while loading empty text file into table. How to handle this empty file load issue in SSIS package.
Question About - Encrypted SQL Server SSIS Configurations
Feb 22, 2008
I tried to follow this link last night (http://curionorg.blogspot.com/) but when i try to configure my package configuration to use sqlserver i recieved the following error message:
The Table "[dbo].[ssisencryptedconfiguration]" cannot be used as a source for SQL Server configurations. The column "ConfiguredVaue" does not have the expected data type.
consult books online for the schema requirements
I followed the blog line by line. i have closed down my project and created a new test package, new project, still the same error. Its there a property somewhere that needs to be configured to allow package configuration to use encryption with sqlserver? I'm running sql 2005 w/sp2. Its there a hotfix that i'm missing. i have not been able to google an answer.
I have been able to successfully use sqlserver as my package configuration. I wanted to see if i could use the encryption code posted on the blog above. i have seen folks reference it or point other folks to it to use for encypting sqlserver package configurations, i just wondered if they have had the same problem.
thanks
Integration Services :: How To Use Encrypted Password For Server In FTP Task In SSIS
Oct 16, 2015I have created a FTP task in SSIS to upload file to the server. Due to some policy issues we do not have the actual password with us. we have been provided with a encrypted password. Hence I am not able to connect to the server through the FTP task connection manager. Is there a way to connect to the server and send file using the encrypted password, so that we don't require the actual password to run the process successfully?
View 3 Replies View RelatedIs Data Sent Via DTS Encrypted?
Jan 27, 2005Hi all,
I may have a requirement to send data from a SQL Server at site A to an Oracle server at site B. These sites have no network connection between them, and the current suggestion is to use ftp, but the transfer (or username and password) will not be encrypted.
If I create a DTS package transferring data from site A, will that transfer be encrypted?
If not, is there an option with SQL Server DTS to ensure that the data is sent in an encrypted form?
Thanks in advance.
Encrypted Data
Feb 23, 2004Hi,
How can I save encryped data in SQL2000? Also, how can I retrieve it?
Need Help --&&> Connecting To Oracle Throught SSIS Package(Failed To Decrypt An Encrypted XML Node Because The Password Was Not S)
May 14, 2007Hi,
In BI Tool SSIS Packages run fine and get data From Oracle and Save it in SQL Server.
Package Protection Level is EncryptSensitivewithPassword.
In BI tool when i open the package it ask password and then run fine.
If i change the Protection Level to Dont save Sensitive,
It does not run fine in even BI tool.
It is fine if i use EncryptSensitivewithPassword.in BI Tool and run it.
Now the problem is that i need to run this package through SQL Job.
so Job give error
"Failed to decrypt an encrypted XML node because the password was not specified or not correct. Package load will attempt to continue without the encrypted information."
Please i need help ?
Thanks
Searching Encrypted Data; Using MAC Secret Data
Aug 10, 2006I just finished reading an article on how to search encrypted data efficiently and they suggested creating a new column with a Message Auhtentication Code. To be honest, reading the aritcle makes my head hurt. I can hardly understand what they were doing myself and I can't begin to explain it to a developer.
Are there any easier ways to search encrypted columns for a speciifc match? If not, does any have some stored procs that implement this messy MAC stuff?
TIA,
Barkingdog
Error Handling In SSIS
Feb 22, 2008All, I am new to SSIS package. I need to know how to handle the errors in SSIS work flow. Can you please provide any specific procedure or method to handle the error. Basically, i wanted to trap the failed records and alert through email... I would be pleased if any one could answer for this..
View 4 Replies View RelatedSSIS Error Handling..
Apr 24, 2008Hi,
I have a package which stores the data from Excel to SQL database. Excel file contains 2 records in which one is good record and other one is bad record. I would like to transfer the bad record to sql 2005 error table.
How should i accomplish this and what are the steps to be followed for successful package implementation.
Its Urgent....
ThanksRamesh.R
Handling Errors In SSIS
Sep 20, 2006When I have an alternet Data Flow in an event handler, caused by a record failing to be inserted due to a unique-key constraint violation, does this increment the number of errors, counting towards the MaximumErrorCount? How can I NOT count it as an error?
The thing is, I need to insert 300,000+ records each day, and some may be duplicates from data already in the table. So I set a unique key constraint on the table, and if during the load, it fails, it will trigger an alernate data flow to load the error records into another table. But if someone tries to load a file that already has been loaded, for example, all the records would be duplicates, which would be equivelant to 300,000+ errors, and I don't want to keep setting the MaximumErrorCount property higher and higher.
Is there any way to treat the error as "being handled" in the dataflow, so therefore doesn't treat it as an error? Or conversely, can I set the MaximumErrorCount property to 0 or -1 to accept all errors, no matter how many?
Error Handling In SSIS
Nov 2, 2007I have an SSIS package in which i use the OLEDB command to InsertUpdate the table (i.e., the command calls an Stored Procedure which has the logic to insertupdate in the table). In the SP we are having Try.. Catch blocks which logs the error into the table, Now my problem is if i give ignore failure then the error is not logged anywhere, if i give fail component obviously the package fails, which i dont want, however when i give Redirect Row then the errors are logeed in the error table via the SP but the problem is it is showing a warning because i have given no destination to the error output. Am i doing anything wrong wat should be the actual mechanism if i want to get the error logged into the table from the SP and not thro. the SSIS package. Thanks in advance.
View 5 Replies View RelatedError Handling In SSIS
Feb 20, 2006Hi!
I guess this is an easy one, but I haven't found a (nice) way to handle "error"-rows.
In my application I load a lot of data, in earlier version I used the Transformation task, which did deliver log and error-files. How do I make the same in SSIS? In the best of worlds I would like to log when a package/task i starting, ending and how many rows are OK and how many rows are errors. For the errors I want a Description what is wrong (FK, PK, NULL value ....) and the complete row in a file so it will be easy to reolad it.
I've tried the logging-feature but when I redirects the error-rows to a text-file, it also eliminates the error text in my log file. So it seems that I have to choose to have the description or the row-records?!
Best, Jocke
Error Handling In SSIS
Apr 18, 2007How do I handle error login login and output using SSIS? Is there any step-by-step tutorial on this error hanlding and notification topic?
DB Design :: How To Use Encrypted Data In Oracle DB
Jun 26, 2015I am using a sql server 2012 and having a database "Test".In this database , having a table "Employee". I have encrypted the employe table data with the password. How can is use these employee data in oracle server. ie;How can I show the employee data as the encrypted and decrypted form of data in Oracle server( The actual encryption happened only in SQL DB).
View 8 Replies View RelatedSQL 2005, Replicating Encrypted Data?
Jun 1, 2006Hello,
I've been searching for information on this for awhile unsuccessully. I am using SQL Server 2005 Enterprise Edititon and transactional replication with separate publisher, distributor and subscribers. I need to encrypt data on our publisher and then I need to replicate that data out (i.e., encrypted data, not encrypted transmission). Since the keys are server specific, I imaging there might be some difficulty in doing this, but I figure I can't be the only one with this problem. Does anyone have a resource or know what must be done to get this accomplished?
Thanks,
Aaron Lowe
Backup And Resore Encrypted Data
Mar 27, 2007I have two databases, one production and other a production copy.
One column of the production table is encrypted by the following syntax.
CREATE MASTER KEY ENCRYPTION BY PASSWORD = 'password!'
CREATE CERTIFICATE cert_demo
WITH SUBJECT = 'encryption demo'
CREATE SYMMETRIC KEY symkey_demo
WITH ALGORITHM = TRIPLE_DES
ENCRYPTION BY CERTIFICATE cert_demo
-----------------------------------
I have stored procedure to add records to a production table like this.
ALTER Procedure AddRecord
@id int,
@var1 varchar(100)
AS
BEGIN
DECLARE @var2 varbinary(256)
OPEN SYMMETRIC KEY symkey_demo
DECRYPTION BY CERTIFICATE cert_demo
SET @var2 = EncryptByKey(
key_guid( 'symkey_demo' ),
@var1 )
insert into dbo.Test(id,Data,EData)
values(@id,@var1,@var2)
CLOSE SYMMETRIC KEY symkey_demo
END
---------------------------------
Now I need to decript the data in the production copy database. So I had export production data to the production copy database and also generates the same key and cerificate. When i tried to decrypt the data, no results has returned.
Create procedure ViewData
@id int
AS
BEGIN
DECLARE @var1 varbinary(256)
DECLARE @var2 varchar(100)
OPEN SYMMETRIC KEY symkey_demo
DECRYPTION BY CERTIFICATE cert_demo
select @var1=EData from Test where id=@id
Set @var2 = convert( varchar(100), DecryptByKey( @var1))
print @var2
END
It doesn't print the enctypted data. Please suggest what goes wrong.
How To Protect Encrypted Data When DB Is Stolen
Jun 25, 2007I got a problem concerning encryption. The thing is I have decided to use symmetric key protected by certificate to encrypt certain information. Certificates are protected by database masterkey and by service key.
But I also want to be sure that if someone steals my database with all its data he wont be able to decrypt it with his own SQL Server Management Studio where he has all the permissions.
Also after some time I will need to take my database and set it up on another PC.
Has anyone ideas how to solve this??
P.S. As far as I know if symmetric key is protected by certificate which is protected by DB master key and service master key then you cant decrypt data if database is moved to another workstation and opened with another Management Studio. Please can anyone explain how this works( if its true). And if this is true then how can i move my DB without loosing access to encrypted data???
Null Value Handling In SSIS Package
May 24, 2007
Hi All
I am facing this problem while loading data from text file into Table.
Scenario is -
There are chances of having spaces for null values in text file.
when i m trying to rum my SSIS package this is getting failed.
How can i avoid this problem? i want null values to be inserted if ther is spaces for that field in text file.
Thanks,
Anshu
SSIS Error Handling Issues
Mar 29, 2006I'm currently rewriting a package that was developed in SQL 2000 DTS to SSIS SQL 2005. When I right click on my DTS package and I go to the Logging tab at the bottom there is an "Error Handling" section. I have the path to the text file where I want the errors saved off. I also have a check in "Fail Package on First Error". My question is how can I do this in SSIS? I can't find any way to do this.
Thank You for any help
Jerry
SSIS Error Handling Not Redirecting
Mar 26, 2008Hi,
I am developing a package which takes data from SQL Server 2005 and hits the JDE files.
In this package I need to send error mail when anyrow returns failure on OLE DB Command component which calls JDE Stored Procedure. So I created a script component which takes error output (red arrow) from OLE DB Command component. But whenever error is triggered, this is not redirecting the rows to script component even though I configured Error output as "Redirect Row". Can anyone please tell me how to redirect error to a script component?
Lookup Error Handling In SSIS
Aug 21, 2006Hi,
I am new to using SSIS. I need to know how can I retrieve the records in a Lookup component that cause an error to use them in a Data Transfer task. I created the error event handler but I don't know how to retrieve the records causing the error to use them in the Data Transfer task.
Thanks in advance for help!
Thanks,
Aref
Handling Custom Validation In SSIS
Mar 19, 2008Hi,
Please suggest me the approach for handling the below custom validation in SSIS
Source: A Table in the database
Destination: One or More Tables in the database
Transformation:For each record in the source table, I have to do custom validation and redirect to one of the destination tables
Custom validation for each record include
1. Checking a set of fields are not matching with main record data in database - If so move to a duplicatelist table
2. Checking for set of fields have got any new values - If so make an entry in a newvaluesfound table
3. If no new/duplicate found move to success table
Approach 1
An OLEdb Source Task to point the source table
An OLEdb Transformation to call an SP for each record in the source
Concerns
The Transformation is implemented in SP(Custom validation & loading to destination)
Approach 2
An Script task from the control flow items - To call an SP in the database
Concerns
The Source, Transformation, Destination - everything managed in the SP
Suggestions needed
Is there any better approach of handling this type of requirement in SSIS
If I write all the code in SP, I may need to use cursors for looping through each record..
Thanks & Regards,
Kalyan
Error Handling For SSIS Packages In .NET
Oct 15, 2007Hi guys,
i'm running my SSIS package via a web application,
Microsoft.SqlServer.Dts.Runtime.Application app = new Microsoft.SqlServer.Dts.Runtime.Application();
Package pkg = app.LoadFromSqlServer("\send mail test", "localhost", null, null, null);
Variables vars = pkg.Variables;
vars["username"].Value = "C# user Name ";
vars["message"].Value = "C# body text";
DTSExecResult result = pkg.Execute();
//MessageBox.Show(result.ToString(), "result from dts"); --app mode
now sometimes the "result" variable return Failure, but i don't know what's wrong, when i run the package in debug mode (in VS) it works fine. DTSExecResult only returns "Success" or "Failure", is there a way to find out more information about the result. like why did the package fail?
the ssis package is built with event handlers but they don't get triggered. it's just when i execute from the web it just returns failed. i checked all the SQL connections as well, they are all fine, as they use SQL authentication, and connection strings are hardcoded as expressions.
any help?
Understanding Error Handling In SSIS
Apr 26, 2007I have a flat file component in a package that is reading a comma separated file.
It expects two columns of int type.
I have an on error event for the entire package. The On error event has one script task which just logs to the windows event log.
if I corrupt the input values in the file, for example instead of a number I put a character
When I debug the package, I see that the script task in the on error event blinks 7 times and I get 7 errors in the windows event log.
Now If I put another script task next to the flat file source and attach it to the flat file source as a error output (with row redirect).
I get only one error message.
Why are the two behaviours different? Both are error handlers one is at the component level the other is at the package level.
So why is it that the package level is invoked 7 times if the input is corrupt but the component level is invoked only once.
regards,
Abhishek.
Searching For Encrypted Fields In Data Columns
Jul 20, 2005I am new to database programming and was curious how others solve theproblem of storing encrypted in data in db table columns and thensubsequently searching for these records.The particular problem that I am facing is in dealing with (privacy)critical information like credit-card #s and SSNs or business criticalinformation like sales opportunity size or revenue in the database. Therequirement is that this data be stored encrypted (and not in theclear). Just limiting access to tables with this data isn't sufficient.Does any database provide native facilities to store specific columns asencrypted data ? The other option I have is to use something like RC4 toencrypt the data before storing them in the database.However, the subsequent problem is how do I search/sort on these columns? Its not a big deal if I have a few hundred records; I couldpotentially retrieve all the records, decrypt the specific fields andthen do in process searches/sorts. But what happens when I have (say) amillion records - I really don't want to suck in all that data and workon it but instead use the native db search/sort capabilities.Any suggestions and past experiences would be greatly appreciated.much thanks,~s
View 10 Replies View RelatedMAC Used As Search Index For Encrypted Data: How Secure?
Aug 17, 2006I have read recommendations about searching encrypted data. Typically, they involve creating a MAC (message authentication code) table. One of the elements of that table is a HASH of the encrypted data (plus a Mac key) that is used as an index for searching. Is that HASH as secure as the encrypted data itself, or is this approach less secure? If it is less secure, then may I assume that approach is the only feasible way to search data encrypted by nondeterministic algorithms?
TIA,
Barkingdog
Backup Database With Encrypted Column Data
Jan 25, 2007Hi,
I have database with encrypted column data in my customer server. Sometime, I might need to backup their database back to office for troubleshooting.
How could I backup/restore master key, symmetric and asymmetric key created for my database?
Thank you
Date And Time Handling Abd Validation In SSIS
Nov 22, 2007I have two columns in Informix data base One has Data Type of date and another column of data type string.
Time is stored in string format. I have to Validate wether both are correct, not null, greater than 1753 and concate to get one datetime field to transfer to SQL Server.
Right now I am doing it in script component, as I need to log error if any thing is wrong
Is there any better way to do it,(derived column or any other component) so that I can log the error also.
Thanks
Dharmbir
Reporting Services :: Error While Deleting Encrypted Data
Sep 15, 2011I am using SQL Server 2008 R2.I have opened the Reporting Service Configuration Manager to delete the encrypted data.In the encryption key section, when I click on the 'Delete' button to delete the encrypted data, it gives me below error. Microsoft.Reporting Services. WmiProvider. WMIProvider Exception: An error occurred when attempting to connect to the report server remote procedure call (RPC) end point. Verify that the Report Server Windows service is running, and then retry the operation.
---> System.Runtime.InteropServices.COMException (0x800706B3): The RPC server is not listening. (Exception from HRESULT: 0x800706B3)
--- End of inner exception stack trace ---
at Microsoft.ReportingServices.WmiProvider.RSWmiAdmin.ThrowOnError(ManagementBaseObject mo)
at Microsoft.ReportingServices.WmiProvider.RSWmiAdmin.DeleteEncryptedInformation()
at ReportServicesConfigUI.WMIProvider.RSReportServerAdmin.DeleteEncryptedInformation()
Howto Reinstall With Local Encrypted Data, No Domain?
Aug 17, 2005This is a question more about EFS, I think, than SQL itself, so apologies for that.
View 1 Replies View RelatedIntegration Services :: Handling Duplicate Records In SSIS?
Oct 14, 2011I have one ssis package moving the data from staging to destination. In stating table we have the duplicate data. But in destination table 4 columns have primary key. How to handle the duplicate records in oldedb source.
View 8 Replies View Related