Storage And Performance Of NULLs And Empty Strings (was Noobish Question)
Feb 11, 2005
Am I right in assuming that when I have a column where all fields contain NULL, this does not increase the total data storage size if my database? Also, what kind of impact would it have on performance?
And what if I inserted "" in varchar columns? I would think the increase in size would be marginal?
The reason I'm asking is that I want to use an existing table and stored procedures for another purpose, but only need half of the columns. But it would significantly simplify application development.
I have two SSIS packages that import from the same flat file into the same SQL 2005 table. I have one flat file connection (to a comma delimited file) and one OLE DB connection (to a SQL 2005 Database). Both packages use these same two Connection Managers. The SQL table allows NULL values for all fields. The flat file has "empty values" (i.e., ,"", ) for certain columns.
The first package uses the Data Flow Task with the "Keep nulls" property of the OLE DB Destination Editor unchecked. The columns in the source and destination are identically named thus the mapping is automatically assigned and is mapped based on ordinal position (which is equivalent to the mapping using Bulk Insert). When this task is executed no null values are inserted into the SQL table for the "empty values" from the flat file. Empty string values are inserted instead of NULL.
The second package uses the Bulk Insert Task with the "KeepNulls" property for the task (shown in the Properties pane when the task in selected in the Control Flow window) set to "False". When the task is executed NULL values are inserted into the SQL table for the "empty values" from the flat file.
So using the Data Flow Task " " (i.e., blank) is inserted. Using the Bulk Insert Task NULL is inserted (i.e., nothing is inserted, the field is skipped, the value for the record is omitted).
I want to have the exact same behavior on my data in the Bulk Insert Task as I do with the Data Flow Task.
Using the Bulk Insert Task, what must I do to have the Empty String values inserted into the SQL table where there is an "empty value" in the flat file? Why & how does this occur automatically in the Data Flow Task?
From a SQL Profile Trace comparison of the two methods I do not see where the syntax of the insert command nor the statements for the preceeding captured steps has dictated this change in the behavior of the inserted "" value for the recordset. Please help me understand what is going on here and how to accomplish this using the Bulk Insert Task.
Will the fuzzy grouping task match a null value to an empty string (or spaces)? I've got 5 columns I'm matching on, and one of them may be null for certain rows but an empty string for others. Given the 4 other columns may match, will this difference stop similar columns being grouped together?
(Someone's modified my grouped data since it was deduped, which takes a while, and I'm hoping for a quick answer on this).
I am in the process of converting DB2 mainframe data to SQL2005. During the conversion I ran into an issue with the DB2OLEDB provider not handling strings with embedded nulls. With the help of the Microsoft Tech support folks I was able to get a fix for the x64 DB2OLEDB provider to handle strings with embedded nulls.
The problem now however is that it appears that when the Data is copied to the Pipeline buffer it is truncated at the first null regardless of the DT_STR length. I have read where .NET is supposed to handle embedded nulls in strings but I am not sure what I need to do to get SSIS packages to handle this situation.
I know when I preview the query in the OLEDB provider within SSIS the data is correct, but as soon as it is passed to the SQL connector, a scripting component or a data conversion component the string is truncated at the first occurence of the embedded null.
I also tried doing a straight copy from the Data provider to a flat file, but the strings are once again truncated.
Is anyone else experienced any other similiar problems or found any resolutions to this type of problem I am getting down to crunch time on this conversion project and any help would be most appreciated.
What is the T-SQL command to check for NULL or '' in a field in one statement? I would like to change the following code to be more readable (without the OR).
Hello, I have a transformation in which the column of data at the flat file source is nine characters long, and typically contains a string of six or seven zeros with a non-zero number in the last two or three characters. If none of the records in that column were an empty string, I think I could get away with this:
The destination is a SQL Server 2000 table, and the column is of type Integer. What do I do when Col004 is an empty string? I've tried a couple of different IF statements, but they have not worked. Empty strings need to become zero values.
************* Edited by moderator Adec *************** Inserted missing < code></ code> tags. Always include such tags when including code in your postings. Don't force the moderators to do this for you. Many readers disregard postings without the code tags. **************************************************
Well met,
Let's say I have a web form that allows users to insert and update records in a SQL database. Is it better to set empty web controls (textbox, etc.) to DBNull or let it go as an empty string into the database?
I have always (or at least intended to) treat NULL and empty strings separately in my SQL querying history. Now I have run across something that mystifies me (but probably shouldn't) that I would like an explanation for.
Consider this bit o' code:
DECLARE @ORDER CHAR(10) SET @ORDER= ( SELECT NULL ) IF @ORDER <> '' PRINT 'Not an empty string'
IF @ORDER IS NULL PRINT 'It is NULL'
Run this and you will get:
It is NULL
I was expecting:
Not an empty string It is NULL
Why is NULL not passing the 'not an empty string' test? In other words, how does NULL = '' ? Is NULL cast to an empty string for this comparison?
I have a SQL database where I am attempting to perform a complicated query that I cannot seem to figure out. I am using SQL Server.
I have 4 tables (TableA, TableB, TableC, and TableD). TableA and TableB are guaranteed to have a relationship.
TableC and TableD are guaranteed to have a relationship.
The trick is, I need to link between TableA and TableC essentially using a LEFT JOIN. I need to retrieve all of the values from TableA regardless and the information from TableC and TableD if there is a link, if there isn't a link, then the values from TableC and TableD need to be empty strings.
Does anyone know how I can do this? I've been trying for the last 5 hours without any luck. I feel I'm close, but there is something I feel I'm overlooking.
I am having data where there are empty string in the business keys which should be used for Slowly changing dimesnion type 2, how do i over come this as due to empty strings i am getting new rows even though the rows havent really changed.
example of data is name and salary are business keys
name salary age address dev 23 klddldldlk sdfg 24 34 kdlddlkd
when the same is given as input the row dev 23 klddldldlk is coming as anew row where it already exists how do i over come this
If you have your Data, Logs, System and TempDB all in VMDK's and those VMDK's are formatted to 64K and then reside on Clustered Storage that is formatted as 4K, which is then running through a SAN controller that is reading and writing in 2MB chunks, is there value in formatting the SQL drives as 64K? Also, would it be better to format the Clustered Storage as 64K? Is this a situation in which formatting the SQL drives as 64K is becoming a moot point?
In my pocket pc inventory application i keep database .sdf file in My Documents. This gives me good performance. But the problem is that sometime my pocket pc hangs and some users cold boot the device and everything is lost.
I tried with keeping my .sdf file on SD card. This way data is there even after cold reboot, but the preformance is very bad. it takes some seconds just to insert only one record. Also if connection is not closed properly due to any reason, the database file is corrupted.
Can anyone tell me if there is a way to persist my .sdf file even after cold reboot without degrading the performance?
What are the performance or storage implications of using a large value for NVARCHAR? For example, if I specify a NVARCHAR(4000) column just to cope with the rare case there are strings that long, but have a table full of strings 255 characters long, is the performance identical to specifying an NVARCHAR(255) column? Is there a reason then NOT to specify NVARCHAR(4000) on everything? i.e. does the query optimizer use it?
I know how long a row is and how many rows can fit in a page (4096 bytes) affects performance, but my understanding is that NVARCHAR only stores the characters needed so it wouldn't be affected unless there was actually a longer string. I wanted to know if there are any other considerations to using a large value here.
Also, how does NTEXT compare to using NVARCHAR? I noticed the documentation said it used a new page after 256 characters, which sounds like it's different from how a 4000 byte NVARCHAR would be stored?
When creating my database I have modeled some of the tables after the Adventureworks sample database.
There are some fields or entire tables in Adventureworks that I do not see an imediate use for, however; I would hate to ommit them to find out later they would have been benificial. (.eg territory table).
In general terms what would the impact be on size and performance of a database which contains tables or fields that do not contain data.
How to implement distinct storage tiers on SQL Remote BLOB Storage (RBS)?
I want to use this SQL Feature to move files(images, videos, pdf files) from a database to a distinct database dedicated to RBS. Then I want to have several storage tiers, where objects will be saved and moved according access frequency. Old data will be arquived in cheap storage, but it must be always accessible if needed.
Description: - 1st and main tier: new and frequently accessed objects stored in high performance storage; - 2nd tier: automatically move older or less accessed objects to an inexpensive and different storage tier; - in all cases, all objects must be accessible to all users, but accessing to archived objects(2nd tier) will be much slower;
When i do a select on my emplee table for rows with null idCompany i dont get any records
I then try to modify the table to not allow a null idCompany and i get this error message:
'Employee (aMgmt)' table - Unable to modify table. Cannot insert the value NULL into column 'idCompany', table 'D2.aMgmt.Tmp_Employee'; column does not allow nulls. INSERT fails. The statement has been terminated.
I am a Windows developer for the IBM Tivoli Storage Manager Server (TSMS) product. Our product installation is built with InstallShield and uses the Windows Installer.
On a new installation of Windows 2003 x64 Storage Server R2, at a customer's site, the TSMS product fails to install. The install of the OS has version 3.01.400.3959 of the Windows Installer and I see no newer version that installs.
Part of our product is 32 bit (console) and another part is x64 (server). When installing I can see that the install's default is being redirected/reset to C:Program Files (x86)TivoliTSM after it is explicitly set by a custom action to ..Program Files.. . I further observe that our custom actions to write 64 bit registry entries are being refused.
REGSAM samMask = KEY_ALL_ACCESS; if ( regIsWow64Process () ) samMask = samMask | KEY_WOW64_64KEY; lStatus = RegCreateKeyEx( hLocalConnectKeyRoot, szSubkey, 0L, NULL, REG_OPTION_NON_VOLATILE, samMask, NULL, hKey, &dw ) ; The above fails to create the key.
We have tried four versions of our TSMS spanning many changes but the install acts the same. This does not happen on any other Windows OS we test on but we do not test on Windows 2003 Storage Server R2 being that it is an OEM product. We did test on Windows server 2003 R2 x64 and do not see this problem.
Do you have any suggestions on how to tackle this problem? I have full installation traces but can only see that the registry work is being refused. I can't see why.
is there an elegant way to use one equals sign in a where clause that returns true when both arguments are null, and returns true when neither is null but both are equal and returns false when only one is null?
Hello Everyone and thanks for your help in advance. I am developing a document storage application for an intranet that will store various Word, Excel, and PDF documents. Most of the examples I see utilize SQL Server and an image field rather than the FileSystem Object to store documents. My concern with this method is that some of the documents may be several hundred pages (not exactly sure of the actual file size yet, but they must be fairly large). My question is, where does the use of SQL Server become impractical for this type of application? Any insight would be greatly appreciated. Thanks.
Does anyone know the upper limit of data size that one SQL 2K table can hold. I've seen 50GB tables in some warehousing servers, but what is the true limit. Soes the limit vary with the SQL2k version?
I have an MSDE installation on Windows server 2003. It looks like the C: drive is taking the brunt of the data when I load up the database. I would like to specify a different drive for data...Is there a way to do this?
How should i know size of the table in the DB. suppose my DB has 5 tables and the size of the DB is 500 MB. How can I know size of the indivdual table.
I'm migrating a images DB of a system I know the structure of the data tables and all type of data in it How can I learn about the STORAGE of IMAGES? In sql Server Where can I found information about that? I need to know something about that topic usually, whats the way for image’s storage ?
I am currently developing a system thats stores exchange stats in a db. Since our customers are companies with 20 employees up to 5 000 there a a big difference in the volume of data needed to be stored.
We currently thinking of supplying a SQL Server Express DB to the small customers and suggest a SQL Server to the bigger.
But since I would like to use the same structure for both types of customers I wonder how should i design the storeage.
Since the could be from 500 records a day up to 20 000. There are quite simple recordes with only simple datatypes. about 15 fields with no more than 10 chars each, mostly 2.
Should i separate the data in diffrent tables for a week or a day etc. Since I am only going to filter data on 1 or 2 fields the data will be easly indexed.
The reports generated will almost always only use 1-3 months of data, but historical reports have to be possible.
My question are ofcourse: Whats the best solution for me?
In MSQL Server 2000 how can I expand or use multiple transaction logs because the hard disk i am using windows dont have more than 4 GB free and the query i want to run overcomes this space. I have another one HDD with 20-30 GB free space and i want to use this disk so to use a second transaction log or move this log to this disk. Can this be happen and how ????
additional to data, what other type of information can be store in sql databases, i need to store pictures and mp3's that can be done, if not do you know what storage can be used for this purpose?
I have recently designed and built my first database using SQL server 2005 express. I have included an image (BLOB) column in one of the database tables. This is a bad idea according to some experts, and some say it is OK!
I am currently carrying out a trial with just 3 pictures via Visual Basic 2005 express forms, and there is no problem so far as the images are displayed for each record. But I anticipate between 300 - 1000 images for the table, and this could pose real problems for SQL server 2005 express and Visual Basic 2005 express, I guess.
I have just been reading that the cost of storing large images in the database is too high! I have also read it's better to store images (BLOB) into the file system because it is cheaper to store them no matter how many there are.
But the question is how I can reference an image in this path: C:PictureProductGrocery 0052745.jpg in the database table, so that when I select a record Visual Basic 2005 forms the image is displayed accordingly, similar as when stored directly in the database table? Your help very much appreciated.
From what I've read, if a row contains more than 8060 bytes and has varchar(MAX) columns in it, the data in those varchar(MAX) columns will be stored off-row. But what happens if you have two varchar(8000) columns instead and both contain more than 4030 bytes, is the data for both stored off-row? If so, just for that row, or for all rows in that table? And is there ever a good reason to have two varchar(8000) columns in a SQL Server 2005 table, instead of using varchar(MAX)?
