T-SQL (SS2K8) :: Detect / Determine Data Stored In Varbinary Field
Oct 21, 2014
I have several tables a varbinary column in a database. They have names like CSB_BLOB or OBJECT_BLOB. Now I am having intermittent success with getting the data out.
For example this query returns readable text from this data.
0x46726F6D3A20226465616E6E6167726.....etc --data as stored in the column
SELECT CAST(CSB_BLOB AS VARCHAR(MAX)) AS 'Message' FROM OBJECT_BLOB
However this column has the following query results.
0x0001000000FFFFFFFF01000000000000000C....etc. --data as stored in column
--this query returns empty result
SELECT (CSB_BLOB AS VARCHAR(MAX)) AS 'Message' FROM CSB_STATUS_LOG
--this query returns no change???
SELECT CONVERT(VARCHAR(MAX), CONVERT(VARBINARY(MAX), CSB_BLOB, 2), 2) FROM CSB_STATUS_LOG
0001000000FFFFFFFF01000000000000000C....etc
Obviously there is a difference between the two but I am not educated enough to interpret this difference. What do I need to learn / read so I can look at the data in one of these BLOB columns and know how to convert it to something meaningful?
Something like:
1. Try to cast as varchar to see if it is text.
2. Turn into a byte array and see if it is a jpg
3. Turn into a byte array and see if it is a pdf
4. Convert it to hex and then cast as varchar
5. etc....
Consider a situation where a stored procedure taking a varbinary(max) (BLOB) input parameter then calls a nested stored procedure and passes along that varbinary(max) as an input parameter to the nested stored procedure.
Is a copy of the BLOB provided to the nested stored procedure (passed by value) OR is the BLOB passed by reference.
My interest is in understanding the potential memory hit when handling large BLOBs in this environment.
For example, if the BLOB is 200MB, will SQL server need to allocate memory for a new copy each time it's passed to another stored procedure?
Looks like table type parameters are passed by reference, but I haven't been able to find any info on BLOBS in this context.
Better method to detect records with one or more 0's, other than hard coding all of the potential values. For example, I need to detect account duplicates where the duplicate contains multiple/all 0's in the "MasterAccount" column:
DECLARE @Account TABLE(AccountNo varchar(3), MasterAccount varchar(10) ) INSERT INTO @Account SELECT 'ABC','0000000ABC' UNION SELECT 'DEF','0000000DEF' UNION SELECT 'ABC','0000000000' UNION ---- unwanted duplicate SELECT 'DEF','00000' UNION ---- unwanted duplicate
I need to run a replace on a varbinary field but I am unsure how. I have tried mulitple different methods and to no avail. I am stumped.
I have attached a zip folder that contains a spreadsheet that shows what I am trying to do. In the first tab it shows the SQL command I use to call the field I want to change. In this case it's the Content field and I am calling it using the content_key. As you can tell the content is stored as varbinary. In the second sheet, It shows this varbinary being converted to varchar(max) and it shows what the actual text says.
The table that this information is stored is is called digitalassetcontent
That is in front of the actual product description.
Somehow I need to run this command:
Code: Update digitalassetcontent set content = replace(CAST(content as varchar(max)), '<STYLE>H1{font-weight:bold}H1{font-size: 14pt}OL</STYLE><H1>Product Description</H1>','')
Where CAST(content as varchar(max)) like '<STYLE>H1{font-weight:bold}H1{font-size: 14pt}OL</STYLE><H1>Product Description</H1>%' AND content_key = 'desc214974236480438500781058983745755010'
This is the error message I get when running this code: Implicit conversion from data type varchar(max) to varbinary(max) is not allowed. Use the CONVERT function to run this query.
I'm trying to insert a string expression into a varbinary field. I've tried it several ways, but the data does not seem to get inserted.
I map DT_STR field ("T") with varbinary field in destination table and the package executes properly, but when I see the data that it has been loaded I only see empty values (0x).
I have also tried other approaches, like converting to DT_BYTES during SSIS flow, but I always get the same result.
Can a bitmap be added to a Varbinary field using the INSERT statement. If not what other method can I use to add a photo image (bmp or gif) to my Sql database?
OK, I use asp.net membership and have created a new table that links together related users by use of the uniqueidentifier field as a foreign key. However, it is possible to link to a not-yet existent user (by their email) so that when that user creates an account and gets a guid, it will be updated into my table of related users and finalise the link. Trouble is, I'm trying to stop people adding the same "pending user" multiple times, so in my BLL logic I look for the entered email address for the current user and I want to check on the foreign key (the GUID field) so that I can tell the user that they've already added this user, but they haven't signed up yet (if they haven't got a foreign user key with a guid in it, they haven't signed up yet). However, when I check the foreign key guid value (shows as null in sql enterprise manager) for null values, it never goes down the "true" logic channel... what I mean is that I've debugged it, hovered over the item I've read (it says it is system.dbnull, which it is) and yet when I do "isDBNull(myGUID.Value)" it never equates to true! Why not?
DelimitedSplit8k and PatternSplitLoop seem to have potential, but I'm just plain stuck on some things:
1. DelimitedSplit8k: the delimiter split the folder paths, but the pattern can be within the strings that result. 2. PatternSplitLoop: I would have to cross apply 16 times and have an awful WHERE clause to determine which of the four strings matched first.
Unless I'm missing something. Short example is below.
WITH testctes (string, pattern) AS ( SELECT 'oh_look_at_this.thing.hishers_stuffmine.craftyours_protein', 'his first' UNION ALL SELECT 'i.am.a._thing.hershis_thingsmine.refrigeratoryours_potato', 'hers first' UNION ALL SELECT 'path_like.things_minehers.some_elsehis_garbageyours_sneakers', 'mine first' UNION ALL SELECT 'more_stuff.yoursminehershis_falafel', 'yours first' ) SELECT string, pattern, ca.item, ca.itemnumber FROM testctes CROSS APPLY [dbo].[PatternSplitLoop] (string, '%his%') ca
I'm looking for feedback on a query I've devised to return a numeric value if the 3 most recent executions of a job have failed. The purpose of the query is to server as a custom counter alert for Idera SQL DM, and by definition an SQL Script alert must return a numeric value.
My environment is SQL Server 2008. Note that the job name is hard coded.
;WITH CTE_Restore_JobHistory AS (SELECT h.[job_id] ,j.[name] as JobName ,h.[message] ,h.[run_status]
I have a table with addresses and activity dates. I need to be able to retrieve the past 3 activity dates and see if the first and last occurred within 15 days. If so, I need to flag them.
Using max date gets me the last date but not the previous two. I was trying to use top 3 in desc order and that didnt seem to work either.
I am trying to setup an indicator value for an SSRS report to show green and red values on a report, based on the NRESULT value. The problem I am facing is that I have several different CASE statements that have the same logic, and they are processing just fine. NRESULT is a decimal field, so no conversion should be necessary. I do not know why I am getting the "Arithmetic overflow error converting varchar to data type numeric." error message.
Below is the CASE statement where the error is occurring. It is in the part of the ELSE CASE. The first CASE works just fine when the ELSE CASE is commented out. If I also change the ELSE CASE statement to say "else case when LEFT(NRESULT,1) = '-' then '0'", then it processes fine, too, so it has to be something I am missing something in the check on negative values. I do need the two checks, one for positive and one for negative values, to take place.
case when LEFT(NRESULT,1) <> '-' then --This portion, for checking positive values, of the CASE statement works fine. CASE WHEN LEFT(ROUND(NRESULT,2),4) between 0.00 and 0.49 THEN '2' --Green ELSE CASE WHEN LEFT(ROUND(NRESULT,2),4) > 0.49 THEN '0' --Red ELSE '3' --White END END else case when LEFT(NRESULT,1) = '-' then --This portion, for checking negative values, of the CASE statement is producing the conversion error message.
[code]....
I checked the NRESULT field, and there are not any NULL values in there, either.
HI, Thanks in advance for taking your time to read this post. I am trying to write a SQL query using MS SQL 2005 that will read the value of a field and tell if it is alpha or numeric. I have tried the following but it does not work: select field1 from table1 where left(field1,2)='[0-9]' select field1 from table1 where isnumber(left(field1,2) tried with a =1 at the end and without and =1 at the end the goal is to read through a field and format it so if a field looks like this 12xxx111xx I can change it to look like 12-xxx-111-xx. Any help is greatly apprecaited
I am currently trying to find a way in which I can determine if a column in a Select clause is Japanese. The column currently supports English and Japanese Kanjis and other kanas. Is there a way to determine if this column is not English or if it is Japanese without physically looking at it.?
Hi,This is driving me nuts, I have a table that stores notes regarding anoperation in an IMAGE data type field in MS SQL Server 2000.I can read and write no problem using Access using the StrConv function andI can Update the field correctly in T-SQL using:DECLARE @ptrval varbinary(16)SELECT @ptrval = TEXTPTR(BITS_data)FROM mytable_BINARY WHERE ID = 'RB215'WRITETEXT OPERATION_BINARY.BITS @ptrval 'My notes for this operation'However, I just can not seem to be able to convert back to text theinformation once it is stored using T-SQL.My selects keep returning bin data.How to do this! Thanks for your help.SD
The following Client1 accounts have A1 value and a blank A2 field. Accounts: Ac1,Ac2,Ac3,Ac4, The following Client2 accounts have A1 value and a blank A2 field. Accounts: Ac1,Ac2,Ac3, The following Client3 accounts have A1 value and a blank A2 field. Accounts: Ac1,Ac2,Ac3, The following Client4 accounts have A1 value and a blank A2 field. Accounts:
Desired output (no trailing comma):
The following Client1 accounts have A1 value and a blank A2 field. Accounts: Ac1,Ac2,Ac3,Ac4 The following Client2 accounts have A1 value and a blank A2 field. Accounts: Ac1,Ac2,Ac3 The following Client3 accounts have A1 value and a blank A2 field. Accounts: Ac1,Ac2,Ac3 The following Client4 accounts have A1 value and a blank A2 field. Accounts:
Next, how do I call the stored procedure without doing it RBAR? Is that possible?
I am trying to order by the field and direction as provided by input parameters @COLTOSORTBY and @DIR while using a CTE and assigning Row_Number, but am running into syntax errors.
Say I have a table called myTable with columns col1,col2,col3,
Here's what I'm trying to do
with myCTE AS ( Select col1 ,col2 ,col3 ,row_number() over (order by case when(@DIR = 'ASC') then
case when @COLTOSORTBY='col1' then col1 asc when @COLTOSORTBY='col2' then col2 asc else col3 asc end else
case when @COLTOSORTBY='col1' then col1 desc when @COLTOSORTBY='col2' then col2 desc else col3 desc end end from myTable )
Please let me know what i can do with minimal code repetition and achive my goal of dynamically sorting column and direction. I do not want to use dynamic SQL under any circumstance.
We serialize a custom object into a byte array (byte[1000]) and store it in a SQL Server 2005 table column as varbinary(1000). There are a lot of rows retrieved with each SqlDataReader from C# code: up to 3,456,000 rows at a time (that is, roughly 3.5 million rows).
I am trying to figure out what will be more efficent in terms of CPU usage and processing time. We have come up with quite a few approaches to solve this problem.
In order to try a few of them, I have to know how I can extract certain "pieces" of data from a varbinary value using T-SQL.
For example, out of those 1000 bytes, at any given moment we need only the first 250 bytes and the third 250 bytes. Total: 1000 -> [250-select][250-no-need][250-select][250-no-need]
One approach would be to get everything and parse it in C#: get the 1st and the 3rd chunks of data and discard the unneeded 2nd and 4th. This is WAY TOO BAD.
Another approach would be to delegate the "filtering" job to SQL Server so that SqlDataReader gets only what it needs.
I am going to try a SQL-CLR stored procedure, but when I compared performance of T-SQL vs. SQL-CLR stored procs a few weeks ago, I saw that the same job is done by T-SQL a bit faster AND (more importantly for us) with less CPU consumption than SQL-CLR.
So, my question is: how do I select certain "pieces" of varbinary column data using T-SQL?..
In other words, instead of SELECT MyVarbinary1000 FROM MyTable how do I do this: SELECT <first 250 from MyVarbinary1000>, <third 250 from MyVarbinary1000> FROM MyTable ?
I have a need to insert rows into an Audit type table when valueschange in certain fields in a table. I thought I could do this via atrigger. However, on requirement is to include in the audit both theold and new value.Is there a "simple" way to do this? I know I could query the tablebefore the update and compare to what the new value is and reactaccordingly.Just wondering if there is something nifty in Sql Server that I ammissing that could help me with this.Thanks in advance for your help.Bill
I need to store 256 bit hash (SHA-2 alogrithmn) in one of the table'sprimary key. I would prefer to use numeric data type rather varcharetc.* Decimal datatype range is -10^38 +1 to 10^38 -1. I can split my 256bit hash into two decimal(38, 0) type columns as composite key* I can store the hash as varbinary. I never used it and don't havemuch understanding in terms of query writing complexities and dealingit through ADO (data type etc.)It would be heavy OLTP type of systems with hash based primary keyused in joins for data retrieval as well.Please provide your expert comments on this.RegardsAnil
I need to put .doc data into a varbinary column for full text searching. I have created the db and columns but am unsure as to how to insert the varbinary data. I have found some discussions about inserting images but nothing explicitly on .doc files. Can anyone suggest resources or sample code?
I have a third party application with a ntext field that I need to parse the data out of. The data looks like this: <xmlF><FNumber type="int">2421</FNumber><AttachmentPath type="string" /><RequesterId type="int">232</RequesterId><Requester type="string">John Smith</Requester><RequestDate type="DateTime">3/24/2008 11:23:27 AM</RequestDate</xmlF> The fieldname is Data and the tablename is ProcessData Again, this looks like xml, but the field type is ntext. I would like to create a view displaying the parsed data in fields. How would I go about parsing the data? Thanks.
Hey all, I have a profile table for members of a website that allows them to upload an avatar for use throughout the site. I decided to add the option to just remove the avatar if they want to no longer use one at all and I am stuck. I am running an update statement against thier entire profile incase they change more then just removing thier avatar but of course when I try this statement. myCommand.Parameters.AddWithValue("@avatar", "") I get a conversion error, which makes sence. So what instead of "" do I need for code to just insert empty data to overwirte thier current avatar data. Any help would be great, thanks for you time.
Hey all, Another varbinary question. I am trying to move an image stored in a table varbinary(max) directly from one table to another programmatically. The rest of the data is just nvarchar(50) so I just use a T-SQL select statement in the code behind and feed all of the date into an SqlDataReader, I do this becuse there is some user interaction when the data is moved, so there may be some new values updated when transfering from one table to another, so once the old and possibly new data is stored in seperate variables then I use a T-SQL insert statement to move all of the data into the other table. Problem is I am not really sure how to handle the image data(varbinary(max)) to just do a straight up transfer from the one table to another. I get conversion errors when trying to handle the data as a string which makes sense. Not sure what to put for code examples since I really am stumped with this, but here is what is not working. Dim imageX As String SqlDataReader Code - imageX = reader("imageData") Insert code - myCommand.Parameters.AddWithValue("@imageData", imageX) Thanks in advance,
I have a database that I am using as an archive for emails and am storing them in varbinary(max) data types. The database works fine for inserts and retrieval but I cannot delete large sets of records ( > 30K) without it timing out. I assume this is because SQL server isn't simply releasing the handle to the blob but is instead doing a lot of work reclaiming the space. Is there any flag I can set or approach I can use to resolve this issue?
I was considering moving the blobs into a seperate simple table and putting async triggers on the primary to delete the blobs, will this work?
Any ideas are appreciated. Besides the "store files in the file system" idea. - Christopher.
I'm unable to create a field with the type varbinary(max). When I try doing this with Management Studio, it tells me that the maximum length is 8000 bytes. I've also tried creating the field with DDL as shown below, but that doesn't work either. If I create the varbinary field with a length of 8000 or less, it works fine. Is there some trick to using varbinary(max)?
I have some source tables which contain timestamp fields (that's timestamp data type not datetime). My dimension table holds the maximum timestamp value as a varbinary(8).
I want my package to have a variable that holds that value but I don't know which data type to use for this. The reason for this is because I want to use that variable to then retrieve all records from my source table that have a timestamp value greater than the value stored in the variable.
