Problem Restoring Database With Encrypted Columns To Different Database Or Server With Encrypted Columns.
Jan 23, 2006
I need to start encrypting several fields in a database and have been doing some testing with a test database first. I've run into problems when attempting to restore the database on either the same server (but different database) or to a separate server.
First, here's how i created the symmetric key and encrypted data in the original database:
create master key
encryption by password = 'testAppleA3';
create certificate test
with subject = 'test certificate',
EXPIRY_DATE = '1/1/2010';
create symmetric key sk_Test
with algorithm = triple_des
encryption by certificate test;
open symmetric key sk_Test decryption by certificate test;
insert into employees values (101,'Jane Doe',encryptbykey(key_guid('sk_Test'),'$200000'));
insert into employees values(102,'Bob Jones',encryptbykey(key_guid('sk_Test'),'$500000'));
select * from employees
--delete from employees
select id,name,cast(decryptbykey(salary) as varchar(10)) as salary from employees
close all symmetric keys
Next I backup up this test database and restore it to a new database on a different server (same issue if restore to different database but on same server).
Then if i attempt to open the key in the new database and decrypt:
open symmetric key sk_Test decryption by certificate test;
I get the error: An error occurred during decryption.
Ok, well not unexpected, so reading the forums, i try doing the below first in the new database:
ALTER MASTER KEY ADD ENCRYPTION BY SERVICE MASTER KEY
Then I try opening the key again and get the error again:
An error occurred during decryption.
So then it occurs to me, maybe i need to drop and recreate it so i do
drop symmetric key sk_test
then
create symmetric key sk_Test
with algorithm = triple_des
encryption by certificate test;
and then try to open it.
Same error!
So then i decide, let's drop everything, the master key, the certificate and then symmetric key:
drop symmetric key sk_test
drop certificate test
drop master key
Then recreate the master key:
create master key
encryption by password = 'testAppleA3';
Restore the certificate from a backup i had made to a file:
CREATE CERTIFICATE test
FROM FILE = 'c:storedcertsencryptiontestcert'
Recreate the symmetric key again:
create symmetric key sk_Test
with algorithm = triple_des
encryption by certificate test;
And now open the key only to get the error:
Cannot decrypt or encrypt using the specified certificate, either because it has no private key or because the password provided for the private key is incorrect.
So what am I doing wrong here? In this scenario I would appear to have lost all access to decrypt the data in the database despite restoring from a backup which restored the symmetric key and certificate and i obviously know the password for the master key.
I also tried running the command
ALTER MASTER KEY ADD ENCRYPTION BY SERVICE MASTER KEY
I've a SQL server 2014 running on one of our server. We're in the process of implementing security steps for our databases. I've encrypted a column in one of the table in the database on the server. The issue is when I restore the backup on my local SQL server and run a query to decrypt the column data it gives me null values. On the other end when I decrypt the column data on the main server it works fine. I found a thread on this forum which states to do the following when restoring the encrypted database on different server.
USE [master]; GO OPEN MASTER KEY DECRYPTION BY PASSWORD = 'StrongPassword'; ALTER MASTER KEY ADD ENCRYPTION BY SERVICE MASTER KEY; GO
select File_Name , CONVERT(nvarchar,DECRYPTBYKEY(File_Name)) from [test].[dbo].[Orders_Customer]
I have idea on SMK, DMK and symmetric and asymmetric keys. I have also idea on TDE. But Is there any way to encrypt all the records of all the columns of a table in a database? actually I need to encrypt the database. Someone .... thinks that when someone will write select query he will get the encrypted records. As per as I am concerned it is not possible. I can encrypt the specific column using symmetric or other keys...
Is there any software or any tool which will provide encrypted records of database?
I read a previous post that Raul responded to on the format of the data prefixing Symmetric key encrypted columns, is it possible to reproduce these from a client? Given I know the key name, i can pull back the first 16 for the GUID, for now 01000000 will work for the version, the IV can be created or read, but the last 8 bytes were marked as 'header', can I get an explanation on what this is or if it is required?
The purpose is in being able to do SQL Server compatible encryption on the client side, given a shared certificate for the public password. I certainly can do this with a CLR function and use my own encryption, but comparably it is dog slow, the built in SQL functions will encrypt/decrypt 100k rows in about 20 seconds or less on my test box, where as the CLR function takes 5.5 minutes. This performance difference is too huge to ignore.
I would like to be able to generate a SQL Server compatible prefix for a Symmetric keyed column or find a way to improve the CLR function performance of an AES_256 (rijndael) up to something at least remotely close to the built in functions.
I am new to database programming and was curious how others solve theproblem of storing encrypted in data in db table columns and thensubsequently searching for these records.The particular problem that I am facing is in dealing with (privacy)critical information like credit-card #s and SSNs or business criticalinformation like sales opportunity size or revenue in the database. Therequirement is that this data be stored encrypted (and not in theclear). Just limiting access to tables with this data isn't sufficient.Does any database provide native facilities to store specific columns asencrypted data ? The other option I have is to use something like RC4 toencrypt the data before storing them in the database.However, the subsequent problem is how do I search/sort on these columns? Its not a big deal if I have a few hundred records; I couldpotentially retrieve all the records, decrypt the specific fields andthen do in process searches/sorts. But what happens when I have (say) amillion records - I really don't want to suck in all that data and workon it but instead use the native db search/sort capabilities.Any suggestions and past experiences would be greatly appreciated.much thanks,~s
For those implementing encrypted columns, what is the recommended approach when allowing users to also do partial searches on encrypted data? (ie email or creditcard info where the tables contain millions of rows). I understand one cannot have the encryption without performance impact, but the searches can be 10 to 20 times as long as when the info is stored in normal char(20) columns. Just looking as a way to try and lessen the impact.
I am developing an application for a big office which uses SQL Server 2000. Apart from my database, on that server, there are two databases by other companies. The administrator also has access to server but the client only wants him to backup the database.
I have two questions:
1) First of all (if it is possible) I would like to protect my own database from the other companies.
I don't want them to:
see the data in the tables (around 20 tables) make changes to the stored procedures (more than 100 stored procedures) be able to backup the database
2) The client will save sensitive data to the database (mainly currency amounts, salaries etc) which he wants to keep hidden. I am using float type for these fields and I would like to make the data encrypted. I could do it for nvarchar fields but changing these float to nvarchar would be time consuming.
I'm working with two sql servers instances, ServerA and ServerB, which run under two different service accounts on different machines. They both have a database, DatabaseA, that has some encrypted fields.
If I take a backup of DatabaseA on ServerA and restore it on ServerB, I need to re-encrypt the Database Master Key (DMK) with the Service Master Key (SMK) as follows:
OPEN MASTER KEY DECRYPTION BY PASSWORD = 'pwd used to encrypt DMK' ALTER MASTER KEY ADD ENCRYPTION BY SERVICE MASTER KEY
However, if I again take a backup on ServerB and then restore it on ServerA, I can use the DMK on ServerA without re-encrypting it with the SMK.
Shouldn't I have to re-encrypt the DMK with the SMK everytime I restore from a backup that was generated from a different server?
I Forgot for my longtime used home expense update application password which has backend sql expressedition database.
i was used the application before 3 years, unfortunately i stopped updating my home expendature to the software.and now i require to login the application but i dont how reset the password in db, i have open database include tables of users profile. and password, but its encrypted.
I have run into problems trying to use backed-up encrypted databases. I have an ASP.Net/SqlServer 2005 website for which I have production, test, and development environments, each on its own box. I have set up encryption using Master Keys, Certificates, and Symmetric Keys using the following commands:
CREATE MASTER KEY ENCRYPTION BY PASSWORD = 'password'
CREATE CERTIFICATE CERTIFICATE_NAME WITH SUBJECT = 'My Website'
CREATE SYMMETRIC KEY KEY_NAME WITH ALGORITHM = TRIPLE_DES ENCRYPTION BY CERTIFICATE CERTIFICATE_NAME.
All databases have the same certificate and symmetric key names. The Production database has a different password than the development and test databases.
I am running into problems when I back up the development database and try to move it to the test environment. When I do the move, and then try to access the encrypted data, I get an error "An error occurred during decryption", even though they are the "same" database (I originally detached and copied the development database to the test box when I set up the test environment) with the same password.
I then try the following commands, which I used when I brought the development database over to the test box the first time:
OPEN MASTER KEY DECRYPTION BY PASSWORD = 'password'
ALTER MASTER KEY REGENERATE WITH ENCRYPTION BY PASSWORD = 'password'
This caused the following error: "An error occurred while decrypting certificate 'CERTIFICATE_NAME' that was encrypted by the old master key. The FORCE option can be used to ignore this error and continue the operation, but data that cannot be decrypted by the old master key will become unavailable."
So, how do you set up encrypted databases so that you can backup from one database server and restore on another? This problem will be very important when I set up a backup production database server.
Hi everyone, I am currently reading ASP.NET unleashed and practising few examples. The following code converts a user's text into a symmetric encryption: 'nd: define keys Const DESKey As String = "ABCDEFGH" Const DESIV As String = "HGFEDCBA" 'nd: convert string to byte array Function convert2ByteArray(ByVal strInput As String) As Byte() Dim intCounter As Integer Dim arrChar As Char() arrChar = strInput.ToCharArray Dim arrByte(arrChar.Length - 1) As Byte For intCounter = 0 To arrByte.Length - 1 arrByte(intCounter) = Convert.ToByte(arrChar(intCounter)) Next Return arrByte End Function
Private Sub btnGo_Click(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles btnGo.Click Dim arrDESKey As Byte() Dim arrDESIV As Byte() Dim arrInput As Byte() Dim objFileStream As FileStream Dim objDES As DESCryptoServiceProvider Dim objEncryptor As ICryptoTransform Dim objCryptoStream As CryptoStream 'convert string to bytes arrDESKey = convert2ByteArray(DESKey) arrDESIV = convert2ByteArray(DESIV) arrInput = convert2ByteArray(txtInput.Text) objDES = New DESCryptoServiceProvider 'pass keys objEncryptor = objDES.CreateEncryptor(arrDESKey, arrDESIV) 'create to file to save password objFileStream = New FileStream(MapPath("secret.txt"), FileMode.Create, FileAccess.Write) 'pass in file and keys objCryptoStream = New CryptoStream(objFileStream, objEncryptor, CryptoStreamMode.Write) 'pass in text objCryptoStream.Write(arrInput, 0, arrInput.Length) objCryptoStream.Close() lblDone.Text = "Done!" End Sub
It works fine. But, how to i save this encrypted password into a database field instead of a writing it to a file? Also, could some please tell me how to paste code into this forum? I tried <code></code> tags but it did not work. Many thanks, Kevin
I have an encrypted database in server A. The reporting service is running on server B. I deployed all my reports to Server B. When I run the reports, I got the following message:
The report server cannot decrypt the symmetric key that is used to access sensitive or encrypted data in a report server database.
I googled the problem, some said to backup the key, but when I opened the reporting service configuration tool to backup the encrypted, it said I needed to restore the key first. Some said I need to delete the key, but that defeated the purpose of an encrypted database.
I have encrypted some columns of a table in a database. Following is the method which i applied for encryption.
I created a master key with a password and it is also encrypted by service master key. Now i created a certificate without password, so it is only encrypted by master key of the database. Now i created a symmetric key encrypted by the above certificate. The data is encrypted by this symmetric key.
To decrypt data i use DecryptByKeyAutoCert.
On my server this encryption & decryption is working perfectly.
But when i take this database to another server, it is not working.
What is the solution for this, should i drop service master key to encrypt master key or is there any soln.
I am executing a stored procedure in one database (Database1) that pulls data from another database (Database2) that is the back end for a third party application. Some of the fields in that other database are now encrypted. I need to decrypt those fields but since the query is running in a database other than where the data lives (which is also where the symmetric key + cert lives), I am getting the following error: "Cannot find the symmetric key" Below is an example of what I am running in the stored procedure:
OPEN SYMMETRIC KEY [XXXXKey] DECRYPTION BY CERTIFICATE [XXXX_CERT]; select CONVERT(Varchar(50), DECRYPTBYKEY( <ENCRYPTED FIELD> )) FROM Database2.dbo.TABLE1 CLOSE SYMMETRIC KEY [XXXXKey];
What do I need to add to Database1 so the stored procedure can decrypt the data it pulls from Database2?
Hi, I have database with encrypted column data in my customer server. Sometime, I might need to backup their database back to office for troubleshooting.
How could I backup/restore master key, symmetric and asymmetric key created for my database?
I have inherited a database and looking to upgrade it from 2008 to 2014. I have imported the database into db projects but it has flagged warning s straight away because one of the previous contractors has created a bunch of stored procs with encryption.
To do this successfully do I need to backup the Service master, Database master, and database itself from the the Source server, then restore all three of them on the destination server?
(I'm concerned that restoring the source Service Master key to a new target server with an existing sql 2005 install will screw things up big time.)
I'm very new new at this. I'm try to deploy a report model and got this message. I have no idea what its going on about.
Can anyone help me?
Aku
------------------------------
System.Web.Services.Protocols.SoapException: The report server cannot decrypt the symmetric key used to access sensitive or encrypted data in a report server database. You must either restore a backup key or delete all encrypted content. Check the documentation for more information. (rsReportServerDisabled) ---> Microsoft.ReportingServices.Diagnostics.Utilities.RPCException: The report server cannot decrypt the symmetric key used to access sensitive or encrypted data in a report server database. You must either restore a backup key or delete all encrypted content. Check the documentation for more information. (rsReportServerDisabled) ---> System.Exception: The report server cannot decrypt the symmetric key used to access sensitive or encrypted data in a report server database. You must either restore a backup key or delete all encrypted content. Check the documentation for more information. (rsReportServerDisabled) ---> System.Exception: The report server cannot decrypt the symmetric key used to access sensitive or encrypted data in a report server database. You must either restore a backup key or delete all encrypted content. Check the documentation for more information. (rsReportServerDisabled) ---> System.Exception: The report server cannot decrypt the symmetric key used to access sensitive or encrypted data in a report server database. You must either restore a backup key or delete all encrypted content. Check the documentation for more information. (rsReportServerDisabled) ---> System.Exception: The report server cannot decrypt the symmetric key used to access sensitive or encrypted data in a report server database. You must either restore a backup key or delete all encrypted content. Check the documentation for more information. (rsReportServerDisabled) ---> System.Exception: Bad Data. (Exception from HRESULT: 0x80090005) --- End of inner exception stack trace --- --- End of inner exception stack trace --- --- End of inner exception stack trace --- --- End of inner exception stack trace --- --- End of inner exception stack trace --- at Microsoft.ReportingServices.Diagnostics.DataProtection.ProtectData(Byte[] unprotectedData, String tag) at Microsoft.ReportingServices.Library.ConnectionManager.ConnectStorage() at Microsoft.ReportingServices.Library.Storage.NewStandardSqlCommand(String storedProcedureName) at Microsoft.ReportingServices.Library.DBInterface.GetAllConfigurationInfo() at Microsoft.ReportingServices.Library.RSService.GetSystemProperties(Property[] requestedProperties) at Microsoft.ReportingServices.WebServer.ReportingService.GetSystemProperties(Property[] Properties, Property[]& Values) --- End of inner exception stack trace --- at Microsoft.ReportingServices.WebServer.ReportingService.GetSystemProperties(Property[] Properties, Property[]& Values) (System.Web.Services)
The passwords in table 'users', column 'passwordencrypted' are encrypted. Someone enters a password and I'd like to compare if it is correct. The syntax below seems ok but nothing is returned. Why not?
OPEN SYMMETRIC KEY mykey DECRYPTION BY CERTIFICATE mycert; DECLARE @mypw varchar(300); SET @mypw = 'test'; SELECT * FROM users WHERE passwordencrypted = EncryptByKey(Key_GUID('mykey'), @mypw); CLOSE SYMMETRIC KEY mykey;
I tried to follow this link last night (http://curionorg.blogspot.com/) but when i try to configure my package configuration to use sqlserver i recieved the following error message:
The Table "[dbo].[ssisencryptedconfiguration]" cannot be used as a source for SQL Server configurations. The column "ConfiguredVaue" does not have the expected data type.
consult books online for the schema requirements
I followed the blog line by line. i have closed down my project and created a new test package, new project, still the same error. Its there a property somewhere that needs to be configured to allow package configuration to use encryption with sqlserver? I'm running sql 2005 w/sp2. Its there a hotfix that i'm missing. i have not been able to google an answer.
I have been able to successfully use sqlserver as my package configuration. I wanted to see if i could use the encryption code posted on the blog above. i have seen folks reference it or point other folks to it to use for encypting sqlserver package configurations, i just wondered if they have had the same problem.
I am trying to run a test migration of our application from V8 to V9. To do this I am creating an empty database by running the Tasks -> Script database
Once that is done, I restore it to our dev server and run SQLCompare against it to generate the change scripts
The problem I am facing is that there are a few tables that have encrypted triggers on them and I cannot include them in the generate scripts. I want to take the table structure, including indexes, but I am not interested in the triggers. There are only 5 tables so I could script them by hand, but this means that I cannot automate the process.
As a result of the missing tables SQLCompare will try to generate a CREATE TABLE rather than an ALTER TABLE DDL
The live migration won't be a problem because this will be a database restore and migration - my problem is the CREATE TABLE DDLs in the script.
I have created a FTP task in SSIS to upload file to the server. Due to some policy issues we do not have the actual password with us. we have been provided with a encrypted password. Hence I am not able to connect to the server through the FTP task connection manager. Is there a way to connect to the server and send file using the encrypted password, so that we don't require the actual password to run the process successfully?
I have a strange situation when I try to execute the same Stored Procedure on servers with different processors. Both servers are running the SQL SERVER 2008 R2 version with all updates.
All updates bios, disk controller, firmware, were applied on the new server.
New Server: (considerable difference in processing time) Stored Procedure without encryption, runs at about 02:16hs Stored Procedure with encryption, runs at about 08:00hs
Server Processor:
Intel (R) Xeon (R) CPU @ 2.60GHz E5-2697 v3
Old Server: (There Are No difference in processing time) Stored Procedure without encryption, runs at about 01:00hs Stored Procedure with encryption, runs at about 01:00hs
Server Processor: Intel (R) Xeon (R) CPU @ 2.7GHz E5-2697 V2
In terms of configuration, the server 2, have lower technology, lower bus, lower number of processors.
I believe that combination of Hardware/SO/Sql Server has a potential performance loss when running encrypted SP.
I'm trying to capture Column Statistics Profile as if I was using SSIS data profiling task. I do not have this option and would like to see how I could go about capturing the min max and avg of all numeric columns within a database.
I'm using Visual Studio 2005 and SQL Server 2005. I am creating a report to put on the Internet using C# and other .NET features. I'm building several charts using database information extracted from Microsoft Excel. I can make charts based on the data from columns using the categories, series, and values on the Data tab of chart properties. However, when I create a query that grabs a record ( a row) from a database, it forces me to use the field values even though each field contains only one value.
Basically, I want to take a row from a database and make it my x-axis data values and another row (record) from a database and make it my y-axis data values.
I'd prefer not to change the format in the database, if possible. Thanks in advance.
Case: Exporting Report to PDF/Printing/TIFF Report: Contains 1 table with 19 Columns. 1 column is static, the other 18 are visible at the users descretion. Report when printed/exported to pdf spans 2 pages naturally, 16 on the first page, 3 on the second, and the column widths have been adjusted to provide a perfect page span .
User A elects to hide two of the columns, and show the rest. The report complies and the viewable version is perfect, the excel export is perfect.. the PDF export on the first page causes every fith column, starting with the last column that was hidden to be expanded to take up additional width. On the spanned page, it renders the first column on that page correctly, then there is a white space gap equal to the width of the hidden columns and then the rest of the cells show with the last column expanded to take up the same width that the original 2 columns were going to take up, plus its width.
We have tried several different settings to see if it helps this issue or makes it worse. So far cangrow/canshrink/keep together have made no impact. It is not possible to increase the page size due to limited page size selection availablility for the client. There are far too many combinations of what the user can elect to show or hide to put together different tables to show and hide on the same report to remove this effect.
Any help or suggestion on this issue would be appreciated
I have an issue to restore an encrypted backup via GUI.I can restore an encrypted backup on another instance using t-sql command, but cannot do the same using the GUI of sql server 2014. The message error is: No backupset selected to be restored.I have restored the master key from the instance where the encrypted backup was created on the instance where I want to restore the encrypted backup, then restored the certificate. I managed to open the master key. I have the following versions:
On the instance where the encrypted backup is taken: Microsoft SQL Server 2014 - 12.0.2000.8 (X64) Feb 20 2014 20:04:26 Copyright (c) Microsoft Corporation Standard Edition (64-bit) on Windows NT 6.2 <X64> (Build 9200: )
On the instance where I'm trying to restore the bakcup via GUI: Microsoft SQL Server 2014 - 12.0.2000.8 (X64) Feb 20 2014 20:04:26 Copyright (c) Microsoft Corporation Standard Edition (64-bit) on Windows NT 6.3 <X64> (Build 9600: )
have a Prod Server A having TDE enabled on 2 of those databases. I have a Prod Server B having TDE enabled on 3 of those databases. Now I have to create a single Dev server Server C for all the above 5 databases residing on the two servers. So how can I restore all the 5 database backup files on server C.
Does it mean that I need to copy the certificates and Keys from both the Prod server to this Dev Box and then restore the backup files. Once done, I can enable the encryption ON on those 5 database on Dev box or is there any different approach.Also how will tempdb behave in this scenario.
when we built the login control by built in login control of vs2005 the passwords are saved in the database which is automatically created.how can i get the unencrypted form of these passwords from database.