I am new to database programming and was curious how others solve the
problem of storing encrypted in data in db table columns and then
subsequently searching for these records.
The particular problem that I am facing is in dealing with (privacy)
critical information like credit-card #s and SSNs or business critical
information like sales opportunity size or revenue in the database. The
requirement is that this data be stored encrypted (and not in the
clear). Just limiting access to tables with this data isn't sufficient.
Does any database provide native facilities to store specific columns as
encrypted data ? The other option I have is to use something like RC4 to
encrypt the data before storing them in the database.
However, the subsequent problem is how do I search/sort on these columns
? Its not a big deal if I have a few hundred records; I could
potentially retrieve all the records, decrypt the specific fields and
then do in process searches/sorts. But what happens when I have (say) a
million records - I really don't want to suck in all that data and work
on it but instead use the native db search/sort capabilities.
Any suggestions and past experiences would be greatly appreciated.
I just finished reading an article on how to search encrypted data efficiently and they suggested creating a new column with a Message Auhtentication Code. To be honest, reading the aritcle makes my head hurt. I can hardly understand what they were doing myself and I can't begin to explain it to a developer.
Are there any easier ways to search encrypted columns for a speciifc match? If not, does any have some stored procs that implement this messy MAC stuff?
I need to start encrypting several fields in a database and have been doing some testing with a test database first. I've run into problems when attempting to restore the database on either the same server (but different database) or to a separate server.
First, here's how i created the symmetric key and encrypted data in the original database:
create master key encryption by password = 'testAppleA3';
create certificate test with subject = 'test certificate', EXPIRY_DATE = '1/1/2010';
create symmetric key sk_Test with algorithm = triple_des encryption by certificate test;
open symmetric key sk_Test decryption by certificate test;
insert into employees values (101,'Jane Doe',encryptbykey(key_guid('sk_Test'),'$200000')); insert into employees values(102,'Bob Jones',encryptbykey(key_guid('sk_Test'),'$500000'));
select * from employees --delete from employees select id,name,cast(decryptbykey(salary) as varchar(10)) as salary from employees
close all symmetric keys
Next I backup up this test database and restore it to a new database on a different server (same issue if restore to different database but on same server).
Then if i attempt to open the key in the new database and decrypt:
open symmetric key sk_Test decryption by certificate test;
I get the error: An error occurred during decryption.
Ok, well not unexpected, so reading the forums, i try doing the below first in the new database:
ALTER MASTER KEY ADD ENCRYPTION BY SERVICE MASTER KEY
Then I try opening the key again and get the error again:
An error occurred during decryption.
So then it occurs to me, maybe i need to drop and recreate it so i do
drop symmetric key sk_test
then
create symmetric key sk_Test with algorithm = triple_des encryption by certificate test;
and then try to open it.
Same error!
So then i decide, let's drop everything, the master key, the certificate and then symmetric key:
drop symmetric key sk_test drop certificate test drop master key
Then recreate the master key:
create master key encryption by password = 'testAppleA3';
Restore the certificate from a backup i had made to a file:
CREATE CERTIFICATE test FROM FILE = 'c:storedcertsencryptiontestcert'
Recreate the symmetric key again:
create symmetric key sk_Test with algorithm = triple_des encryption by certificate test;
And now open the key only to get the error:
Cannot decrypt or encrypt using the specified certificate, either because it has no private key or because the password provided for the private key is incorrect.
So what am I doing wrong here? In this scenario I would appear to have lost all access to decrypt the data in the database despite restoring from a backup which restored the symmetric key and certificate and i obviously know the password for the master key.
I also tried running the command
ALTER MASTER KEY ADD ENCRYPTION BY SERVICE MASTER KEY
I am executing a stored procedure in one database (Database1) that pulls data from another database (Database2) that is the back end for a third party application. Some of the fields in that other database are now encrypted. I need to decrypt those fields but since the query is running in a database other than where the data lives (which is also where the symmetric key + cert lives), I am getting the following error: "Cannot find the symmetric key" Below is an example of what I am running in the stored procedure:
OPEN SYMMETRIC KEY [XXXXKey] DECRYPTION BY CERTIFICATE [XXXX_CERT]; select CONVERT(Varchar(50), DECRYPTBYKEY( <ENCRYPTED FIELD> )) FROM Database2.dbo.TABLE1 CLOSE SYMMETRIC KEY [XXXXKey];
What do I need to add to Database1 so the stored procedure can decrypt the data it pulls from Database2?
I am developing an application for a big office which uses SQL Server 2000. Apart from my database, on that server, there are two databases by other companies. The administrator also has access to server but the client only wants him to backup the database.
I have two questions:
1) First of all (if it is possible) I would like to protect my own database from the other companies.
I don't want them to:
see the data in the tables (around 20 tables) make changes to the stored procedures (more than 100 stored procedures) be able to backup the database
2) The client will save sensitive data to the database (mainly currency amounts, salaries etc) which he wants to keep hidden. I am using float type for these fields and I would like to make the data encrypted. I could do it for nvarchar fields but changing these float to nvarchar would be time consuming.
I need a query to pull the data from Sql server. my requirement is i need to pull the data from multiple columns, in that there are three email fields are there like email1, email2, email3. i need query to retreive the data from table first it search for email in the above 3 fields if any one of the fields contains the record the it display as Main mail id.
I read a previous post that Raul responded to on the format of the data prefixing Symmetric key encrypted columns, is it possible to reproduce these from a client? Given I know the key name, i can pull back the first 16 for the GUID, for now 01000000 will work for the version, the IV can be created or read, but the last 8 bytes were marked as 'header', can I get an explanation on what this is or if it is required?
The purpose is in being able to do SQL Server compatible encryption on the client side, given a shared certificate for the public password. I certainly can do this with a CLR function and use my own encryption, but comparably it is dog slow, the built in SQL functions will encrypt/decrypt 100k rows in about 20 seconds or less on my test box, where as the CLR function takes 5.5 minutes. This performance difference is too huge to ignore.
I would like to be able to generate a SQL Server compatible prefix for a Symmetric keyed column or find a way to improve the CLR function performance of an AES_256 (rijndael) up to something at least remotely close to the built in functions.
hi ; I am abut searching in a database that html data stored in it, I want to search something like this: search me And it stored like this: search <b> me </b> I don’t know how to remove html tags on stored procedure,
It will be great if you direct me to a search stored procedure that handle this it rather make the sorting and paging stuff too .
I have been trying to write a query that will allow me to search through text fields. This is a problem because SQL doesn't let me use any functions on TEXT datatypes.
When I was using access I did it like this:
where upper(searchtext) like ('%SEARCHSTRING%')
However UPPER doesn't work on text fields (I want the search to be case insensitive)
I tried this: where patindex(searchtext, '%SEARCHSTRING%') <> 0
but that is not case-insensitive... Help me SQL gurus, you are my only hope
I have idea on SMK, DMK and symmetric and asymmetric keys. I have also idea on TDE. But Is there any way to encrypt all the records of all the columns of a table in a database? actually I need to encrypt the database. Someone .... thinks that when someone will write select query he will get the encrypted records. As per as I am concerned it is not possible. I can encrypt the specific column using symmetric or other keys...
Is there any software or any tool which will provide encrypted records of database?
For those implementing encrypted columns, what is the recommended approach when allowing users to also do partial searches on encrypted data? (ie email or creditcard info where the tables contain millions of rows). I understand one cannot have the encryption without performance impact, but the searches can be 10 to 20 times as long as when the info is stored in normal char(20) columns. Just looking as a way to try and lessen the impact.
Hi, I am developing a search engine for my application and I need to grab the entire row from the table if I found the search field in any of the columns and the search is not defined to one table. Please let me know how to search all columns in table without using column names indivdually.
I've a SQL server 2014 running on one of our server. We're in the process of implementing security steps for our databases. I've encrypted a column in one of the table in the database on the server. The issue is when I restore the backup on my local SQL server and run a query to decrypt the column data it gives me null values. On the other end when I decrypt the column data on the main server it works fine. I found a thread on this forum which states to do the following when restoring the encrypted database on different server.
USE [master]; GO OPEN MASTER KEY DECRYPTION BY PASSWORD = 'StrongPassword'; ALTER MASTER KEY ADD ENCRYPTION BY SERVICE MASTER KEY; GO
select File_Name , CONVERT(nvarchar,DECRYPTBYKEY(File_Name)) from [test].[dbo].[Orders_Customer]
Dear GroupThe scaenario is as follows:FirstName and LastName are separate columns in the contact table and Iwant to be able to search e.g. for the FirstName and part of theLastName at the same time e.g. 'John A' should return 'John Adams'.Would be grateful if you can give me some hint as I don't seem to getit work.SELECT FirstName, Lastname FROM i2b_contact WHERE (SELECT Firstname +Lastname AS CName) LIKE 'John A%'Thanks very much for your help and efforts!Martin
I admit I am a novice to queries and stored procedures but understand stored procedures are more secure, so I am choosing to challenge myself with a stored procedure and have not been successful at all.
What I am looking for is a stored procedure which would query the single table named Hospital, which would search from a single textbox the end user would enter the text they are searching for on a Windows VB application but is dependant on a few end user selected items.
1. The end user can select a radio button either "Starts with" or "Contains" 2. The end user can select from 6 checkboxes (Hospitals, Address, City, State, Zip, Type) to limit or expand their search from the columns in the table.
The table has 17columns (CO, PARENTCO, ADDR, POBox, CITY, State, ZIP, TEL, FAX, WEBSITE, HOSP_TYPE, OWNERSHIP, BEDS, CONTACT, TITLE, Email) but I am only allowing the end user to search by the above 6 columns and need to display the above 6 columns plus the phone, fax and website columns.
After the user clicks the search button it displays the results in a listbox and the user will then select the one they want and it displays the information in another set of textboxes.
Hopefully someone can get me pointed in the right direction! Thanks for your help!
I may have a requirement to send data from a SQL Server at site A to an Oracle server at site B. These sites have no network connection between them, and the current suggestion is to use ftp, but the transfer (or username and password) will not be encrypted.
If I create a DTS package transferring data from site A, will that transfer be encrypted?
If not, is there an option with SQL Server DTS to ensure that the data is sent in an encrypted form?
I have just started using SQL Server reporting services and am stuck with creating subreports.
I have a added a sub report to the main report. When I right click on the sub report, go to properties -> Parameters, and click on the dropdown for Parameter Value, I see all Sum and Count fields but not the data fields.
For example, In the dropdownlist for the Parameter value, I see Sum(Fields!TASK_ID.Value, "AppTest"), Count(Fields!TASK_NAME.Value, "CammpTest") but not Fields!TASK_NAME.Value, Fields!TASK_ID.Value which are the fields retrieved from the dataset assigned to the subreport.
When I manually change the parameter value to Fields!TASK_ID.Value, and try to preview the report, I get Error: Subreport could not be shown. I have no idea what the underlying issue is but am guessing that it's because the field - Fields!TASK_ID.Value is not in the dropdown but am trying to link the main report and sub report with this field.
Am I missing something here? Any help is appreciated.
I am using a sql server 2012 and having a database "Test".In this database , having a table "Employee". I have encrypted the employe table data with the password. How can is use these employee data in oracle server. ie;How can I show the employee data as the encrypted and decrypted form of data in Oracle server( The actual encryption happened only in SQL DB).
I've been searching for information on this for awhile unsuccessully. I am using SQL Server 2005 Enterprise Edititon and transactional replication with separate publisher, distributor and subscribers. I need to encrypt data on our publisher and then I need to replicate that data out (i.e., encrypted data, not encrypted transmission). Since the keys are server specific, I imaging there might be some difficulty in doing this, but I figure I can't be the only one with this problem. Does anyone have a resource or know what must be done to get this accomplished?
I have two databases, one production and other a production copy.
One column of the production table is encrypted by the following syntax.
CREATE MASTER KEY ENCRYPTION BY PASSWORD = 'password!'
CREATE CERTIFICATE cert_demo
WITH SUBJECT = 'encryption demo'
CREATE SYMMETRIC KEY symkey_demo
WITH ALGORITHM = TRIPLE_DES
ENCRYPTION BY CERTIFICATE cert_demo
-----------------------------------
I have stored procedure to add records to a production table like this.
ALTER Procedure AddRecord
@id int,
@var1 varchar(100)
AS
BEGIN
DECLARE @var2 varbinary(256)
OPEN SYMMETRIC KEY symkey_demo
DECRYPTION BY CERTIFICATE cert_demo
SET @var2 = EncryptByKey(
key_guid( 'symkey_demo' ),
@var1 )
insert into dbo.Test(id,Data,EData)
values(@id,@var1,@var2)
CLOSE SYMMETRIC KEY symkey_demo
END
---------------------------------
Now I need to decript the data in the production copy database. So I had export production data to the production copy database and also generates the same key and cerificate. When i tried to decrypt the data, no results has returned.
Create procedure ViewData
@id int
AS
BEGIN
DECLARE @var1 varbinary(256)
DECLARE @var2 varchar(100)
OPEN SYMMETRIC KEY symkey_demo
DECRYPTION BY CERTIFICATE cert_demo
select @var1=EData from Test where id=@id
Set @var2 = convert( varchar(100), DecryptByKey( @var1))
print @var2
END
It doesn't print the enctypted data. Please suggest what goes wrong.
I got a problem concerning encryption. The thing is I have decided to use symmetric key protected by certificate to encrypt certain information. Certificates are protected by database masterkey and by service key.
But I also want to be sure that if someone steals my database with all its data he wont be able to decrypt it with his own SQL Server Management Studio where he has all the permissions.
Also after some time I will need to take my database and set it up on another PC.
Has anyone ideas how to solve this??
P.S. As far as I know if symmetric key is protected by certificate which is protected by DB master key and service master key then you cant decrypt data if database is moved to another workstation and opened with another Management Studio. Please can anyone explain how this works( if its true). And if this is true then how can i move my DB without loosing access to encrypted data???
I have read recommendations about searching encrypted data. Typically, they involve creating a MAC (message authentication code) table. One of the elements of that table is a HASH of the encrypted data (plus a Mac key) that is used as an index for searching. Is that HASH as secure as the encrypted data itself, or is this approach less secure? If it is less secure, then may I assume that approach is the only feasible way to search data encrypted by nondeterministic algorithms?
Hi, I have database with encrypted column data in my customer server. Sometime, I might need to backup their database back to office for troubleshooting.
How could I backup/restore master key, symmetric and asymmetric key created for my database?
I am using SQL Server 2008 R2.I have opened the Reporting Service Configuration Manager to delete the encrypted data.In the encryption key section, when I click on the 'Delete' button to delete the encrypted data, it gives me below error. Microsoft.Reporting Services. WmiProvider. WMIProvider Exception: An error occurred when attempting to connect to the report server remote procedure call (RPC) end point. Verify that the Report Server Windows service is running, and then retry the operation.
---> System.Runtime.InteropServices.COMException (0x800706B3): The RPC server is not listening. (Exception from HRESULT: 0x800706B3) --- End of inner exception stack trace --- at Microsoft.ReportingServices.WmiProvider.RSWmiAdmin.ThrowOnError(ManagementBaseObject mo) at Microsoft.ReportingServices.WmiProvider.RSWmiAdmin.DeleteEncryptedInformation() at ReportServicesConfigUI.WMIProvider.RSReportServerAdmin.DeleteEncryptedInformation()
I want to retrieve SQL 2000 Encrypted Column Data From SQL 2005 strored proc. My Stored Procedure was on SQL 2000 and it works fine....Then I restore Database From SQL 2000 to SQL 2005. The Following Statement is on my store proce.
select user_id , Encrypt(user_pass) from OpenRowset('SQLOLEDB','myserver';'sa';'mypass',databasename.dbo.users) as a
The Following Error I get When I execute the above statement.
Msg 195, Level 15, State 10, Line 1
'Encrypt' is not a recognized built-in function name.
I have one table namely consumer with approx 50 columns.
I have created one same table with audit prefix including 2 more column for action n timestamp fields.
My question is that if user change only 10 column data at a time: i want to add only that particular column data rather to add entire row. Currently I am adding entire row in audit table but now scenario is change to update only updated column data.
hi need help from the wonderful people here i am working on project and convert a OLD DOS program to SQL server on the web this is the resalt i need to show
I've called a resultset from SQL Server using an SQL Selection. I need to iterate over that entire result set (200+ columns/fields) and all I need are the random numbers contained in any of the rows/columns. I don't want to have to name each field/column and then use an if > 0 statement.Isn't there some way to generically loop through the column's by index or something instead of their field name so I can just use an integer loop to walk the dataset? I know there is I've done it about 5 years ago. The question is how do you do it in C#?SqlConnection thisConn = new SqlConnection(ConfigurationManager.ConnectionStrings["SQLQuery"].ConnectionString); SqlCommand thisCmd = new SqlCommand("Command String", thisConn); thisCmd.CommandText = "Select * from SelectionsByCountry where [" + DropDownList1.SelectedItem.ToString() + "] > '0'"; thisConn.Open(); SqlDataReader thisReader = null; thisReader = thisCmd.ExecuteReader(CommandBehavior.CloseConnection); while (thisReader.Read()) { DropDownList2.Items.Add(thisReader["System"].ToString().Trim());/*** There are 200+ columns left I want to walk over using a loop structure of some sort. How do I do that?*/ }- Rex
