Table Columns Storage
Jul 20, 2005
Hi ,
This is related to datawarehouse , data mining . We are told that data
mining tools such as spss or sas , need a large table which has lots
of columns inside . Based on our project , out final table which will
be used in data mining , has lots of computed values . The final
number of columns of the table is 7800!(lots of calculated values)
First of all , i dont see the reason of this requirement of the data
mining tools. I will be appreciated if someone can clarify this: Why
does a data mining tool need such a large table?
The main issue is if i create table with 7800 columns inside ( in fact
oracle only allows to create table with 1000 columns) , i believe it
cant be queried.
My basic calculation shows that the average row size of this table
will be 160kb. , considering my db_block_size of 16kb. this means 10
blocks for 1 row. (The table will have more than 10.000.000 rows) No
matter how fast my disk subsystem is , i think the queries against
this table will fail. So what can i do? May be I need a different
type of storage technique for instance column based storage ( i heard
that sysbase has this feature , dont know the details / purpose..)
How can i solve such a problem? The database server really does not
matter , it can be oracle,sqlserver , sybase,informix , etc... I will
be appreciated if someone can help me abut the issue.
Kind Regards,
hope
View 1 Replies
ADVERTISEMENT
Aug 17, 2006
Dear all,
In my current database design, there is one table (PState) which has a Primary Key (int) and a few other fields.
During development, a pattern started to arise; for certain rows in PState, I wanted to specify an additional set of columns (over 10 of them with quite large lengths) for each row in PState. However, as these additional columns would only be required in approximately 20% of the rows of PState, there would be plenty of NULL values in PState if I would make this table wider than necessary. So, I decided to create a separate table with those optional columns (PStateWFI). In order to attach these additional columns in PStateWFI to PState in the cases they were needed, I would obviously have to create a Foreign Key constraint on the Primary Key of PStateWFI so that these optional rows would know which row in PState they would belong to.
However, the problem with this approach is that one could define multiple rows in PStateWFI referring to the same row in PState, which would not make sense. Thus, a UNIQUE index constraint added to the constrained ID column in PStateWFI would make sense to ensure that there could only be one set of optional columns added to each row in PState. But now, when adding the UNIQUE index, the FK constraint started to appear as a bidirectional key link in the Diagram; hence, new entries in PState would have to meet a FK constraint based on PStateWFI, which was not intended.
Hence, I had to create a quite awkward design to enforce the constraints:
1. The PState table has a Primary Key (PState.ParticleID, int, Identity Specification: Yes)
2. The PStateWFI table has a Primary Key (PstateWFIID, int, Identity Specification: Yes)
3. PStateWFI has field "PStateID" which has a FK constraint to PState.ParticleID (which is a one-way constraint operating in the correct way and does not constrain insertions in PState)
4. PStateWFI has an additional column ParticleIDIndex which has a UNIQUE Index attached to it.
5. There is a check constraint on PStateWFI enforcing PStateWFI.ParticleIDIndex = PStateWFI.ParticleID.
Although this structure does the job, it makes it necessary to add a redundant column in PStateWFI by duplicating the PStateWFI.ParticleID into PStateWFI.ParticleIDIndex, since I can't create a UNIQUE index on PStateWFI.ParticleID without constraining the PState table as well. So, insertions into this table would have to insert the same value into two columns. Not a big deal, but appears slightly ugly.
Basically I'd hope someone could explain why a bidirectional FK constraint has to be enforced on the primary key table in a relationship when the constrained column in the primary key table has a UNIQUE index attached on it. I have a few other cases where the above approach would benefit from a more clear structure.
Thanks in advance for any advice.
View 6 Replies
View Related
May 23, 2007
Hi,
This question comes without the wish for any philosofical debates concerning NULL
Normally SQL Server 2005 uses 13 bytes to store a NUMERIC(22,7) value. Does it use the full 13 bytes when the column contains NULL as well? And what about DATETIME and (BIG)INT? Most of the info on storage of NULL values is about varchar/char/nvarchar/varchar...
Thnx, Jeroen.
View 3 Replies
View Related
Oct 27, 2014
How to implement distinct storage tiers on SQL Remote BLOB Storage (RBS)?
I want to use this SQL Feature to move files(images, videos, pdf files) from a database to a distinct database dedicated to RBS. Then I want to have several storage tiers, where objects will be saved and moved according access frequency. Old data will be arquived in cheap storage, but it must be always accessible if needed.
Description:
- 1st and main tier: new and frequently accessed objects stored in high performance storage;
- 2nd tier: automatically move older or less accessed objects to an inexpensive and different storage tier;
- in all cases, all objects must be accessible to all users, but accessing to archived objects(2nd tier) will be much slower;
View 0 Replies
View Related
Feb 20, 2002
Hello,
Does anyone know the upper limit of data size that one SQL 2K table can hold. I've seen 50GB tables in some warehousing servers, but what is the true limit. Soes the limit vary with the SQL2k version?
Thanks!
View 1 Replies
View Related
Feb 21, 2005
How should i know size of the table in the DB. suppose my DB has 5 tables and the size of the DB is 500 MB. How can I know size of the indivdual table.
Thanks.
View 6 Replies
View Related
Aug 3, 2007
hello forum friends,
i need to know where the database table stored in,
where to find the data table without distrubing the
sql server editor.whtr it is possible to get it from
"C:Program FilesMicrosoft SQL ServerMSSQLData"
or some where else.
Regards
prathap
View 13 Replies
View Related
Jul 22, 2015
So I have been trying to get mySQL query to work for a large database that I have. I have (lets say) two tables Table_One and Table_Two. Table_One has three columns: Type, Animal and TestID and Table_Two has 2 columns Test_Name and Test_ID. Example with values is below:
**TABLE_ONE**
Type Animal TestID
-----------------------------------------
Mammal Goat 1
Fish Cod 1
Bird Chicken 1
Reptile Snake 1
Bird Crow 2
Mammal Cow 2
Bird Ostrich 3
**Table_Two**
Test_name TestID
-------------------------
Test_1 1
Test_1 1
Test_1 1
Test_1 1
Test_2 2
Test_2 2
Test_3 3
In Table_One all types come under one column and the values of all Types (Mammal, Fish, Bird, Reptile) come under another column (Animals). Table_One and Two can be linked by Test_ID
I am trying to create a table such as shown below:
Test_Name Bird Reptile Mammal Fish
-----------------------------------------------------------------
Test_1 Chicken Snake Goat Cod
Test_2 Crow Cow
Test_3 Ostrich
This should be my final table. The approach I am currently using is to make multiple instances of Table_One and using joins to form this final table. So the column Bird, Reptile, Mammal and Fish all come from a different copy of Table_one.
For e.g
Select
Test_Name AS 'Test_Name',
Table_Bird.Animal AS 'Birds',
Table_Mammal.Animal AS 'Mammal',
Table_Reptile.Animal AS 'Reptile,
Table_Fish.Animal AS 'Fish'
From Table_One
[Code] .....
The problem with this query is it only works when all entries for Birds, Mammals, Reptiles and Fish have some value. If one field is empty as for Test_Two or Test_Three, it doesn't return that record. I used Or instead of And in the WHERE clause but that didn't work as well.
View 4 Replies
View Related
May 31, 2007
Hello,
I am building partitiong tables, partitioning on different file groups:
the question is:
Partitioned table referred to old data that are not frequent accessed for reporting can be stored on separate location(External storage, tape and so on) or to make partitioning functioning must all file groups must be presents?
If not, how can I separate old data from current ones (still using partitioning) to reduce the size of DB?
What it is the best for storage data and easy to access it when needs arise (eg reporting): Tape, external storage, others?
Thank
View 1 Replies
View Related
Jan 14, 2008
I am a Windows developer for the IBM Tivoli Storage Manager Server (TSMS) product.
Our product installation is built with InstallShield and uses the Windows Installer.
On a new installation of Windows 2003 x64 Storage Server R2, at a customer's site, the TSMS product fails to install.
The install of the OS has version 3.01.400.3959 of the Windows Installer and I see no newer version that installs.
Part of our product is 32 bit (console) and another part is x64 (server).
When installing I can see that the install's default is being redirected/reset to C:Program Files (x86)TivoliTSM after it is explicitly set by a custom action to ..Program Files.. . I further observe that our custom actions to write 64 bit registry entries are being refused.
REGSAM samMask = KEY_ALL_ACCESS;
if ( regIsWow64Process () ) samMask = samMask | KEY_WOW64_64KEY;
lStatus = RegCreateKeyEx( hLocalConnectKeyRoot,
szSubkey,
0L,
NULL,
REG_OPTION_NON_VOLATILE,
samMask,
NULL,
hKey,
&dw ) ;
The above fails to create the key.
We have tried four versions of our TSMS spanning many changes but the install acts the same.
This does not happen on any other Windows OS we test on but we do not test on Windows 2003 Storage Server R2 being that it is an OEM product. We did test on Windows server 2003 R2 x64 and do not see this problem.
Do you have any suggestions on how to tackle this problem?
I have full installation traces but can only see that the registry work is being refused. I can't see why.
View 1 Replies
View Related
Aug 22, 2007
I need to generate a report of DTS package results, i. e. succeed, fail, error, etc. What tables is information of this type stored for SQL 2000?
winniemax
View 3 Replies
View Related
Apr 23, 2015
I am using sql server 2008 r2 on my end. I have created a database named testDB. I have a lot of tables with some log tables in this. some tables have contain lack of records in log table.
So my purpose is that I want to fix the table size of those tables(log tables) and want to move records in other database table placed on another location. So my database has no problem.
is there any way to make such above steps which I want for my database?
Is there already built any such functionality in sql server?
View 2 Replies
View Related
Sep 19, 2007
Hi,
I have two tables
Table Source
{
category varchar(20),
LastUpdate datetime
}
Table Destination
{
SubjectId varchar(20),
SubjectDate datetime,
category varchar(20),
LastUpdate datetime
}
Please note that the number columns are different in each table.
I wanted to dump the data of Source table to Destination table. I meant to say that the rows of 2 columns in Source table to last 2 rows of Destination table.
And also my oreder of the columns in Destination table will vary. So i need to a way to dynamically insert the data in bulk. but i will know the column names for sure before inserting.
Is there anyway to bulk insert into these columns.
Your quick response will be appreciated
~Mohan Babu
View 2 Replies
View Related
Apr 29, 2015
I have a business need to create a report by query data from a MS SQL 2008 database and display the result to the users on a web page. The report initially has 6 columns of data and 2 out of 6 have JSON data so the users request to have those 2 JSON columns parse into 15 additional columns (first JSON column has 8 key/value pairs and the second JSON column has 7 key/value pairs). Here what I have done so far:
I found a table value function (fnSplitJson2) from this link [URL]. Using this function I can parse a column of JSON data into a table. So when I use the function above against the first column (with JSON data) in my query (with CROSS APPLY) I got the right data back the but I got 8 additional rows of each of the row in my table. The reason for this side effect is because the function returned a table of 8 row (8 key/value pairs) for each json string data that it parsed.
1. First question: How do I modify my current query (see below) so that for each row in my table i got back one row with 19 columns.
SELECT A.ITEM1,A.ITEM2,A.ITEM3,A.ITEM4, B.*
FROM PRODUCT A
CROSS APPLY fnSplitJson2(A.ITEM5,NULL) B
If updated my query (see below) and call the function twice within the CROSS APPLY clause I got this error: "The multi-part identifier "A.ITEM6" could be be bound.
2. My second question: How to i get around this error?
SELECT A.ITEM1,A.ITEM2,A.ITEM3,A.ITEM4, B.*, C.*
FROM PRODUCT A
CROSS APPLY fnSplitJson2(A.ITEM5,NULL) B, fnSplitJson2(A.ITEM6,NULL) C
I am using Microsoft SQL Server 2008 R2 version. Windows 7 desktop.
View 14 Replies
View Related
May 26, 2015
Here is My requirement, I'm not sure if this is possible. Creating table called master like col1, col2 col3, col4 , col5 ...Where Col1, col2 are updatable - this can be done easily
Col3, col4 are columns in another table but these can be just a read only ?? Is this possible ? this is possible with View but not friendly with share point CRUD...Col 5 is a computed column of col 2 and col5 ? if above step can be done then sure this can be done I guess.
View 4 Replies
View Related
Jul 20, 2005
Hello,Using SQL Server 2000, I'm trying to put together a query that willtell me the following information about a view:The View NameThe names of the View's columnsThe names of the source tables used in the viewThe names of the columns that are used from the source tablesBorrowing code from the VIEW_COLUMN_USAGE view, I've got the codebelow, which gives me the View Name, Source Table Name, and SourceColumn Name. And I can easily enough get the View columns from thesyscolumns table. The problem is that I haven't figured out how tolink a source column name to a view column name. Any help would beappreciated.Garyselectv_obj.name as ViewName,t_obj.name as SourceTable,t_col.name as SourceColumnfromsysobjects t_obj,sysobjects v_obj,sysdepends dep,syscolumns t_colwherev_obj.xtype = 'V'and dep.id = v_obj.idand dep.depid = t_obj.idand t_obj.id = t_col.idand dep.depnumber = t_col.colidorder byv_obj.name,t_obj.name,t_col.name
View 2 Replies
View Related
Apr 4, 2008
I'm in the process of converting legacy DTS packages to SSIS. I need to populate a table that has more fields than the source file. In DTS I did this with an ActiveX script. How do I go about doing this within SSIS.
In the ActiveX script most of the fields were defaulted with either spaces or zeroes.
One of the Destination fields needs to be incremented by 1 for each new record inserted.
Any help would be appreciated.
Thanks,
Jeff
View 6 Replies
View Related
Dec 23, 2013
I just created a new table with over 100 Columns and I need to populated just the first 2 columns.
The first columns to populate is an identify column that is the primary key. The second column is a foreign_key to an other column and I am trying to populate this columns with all the values from the foreign_key value. This is what I am trying to do.
column1 = ID
column2= P_CLIENT_D
SET IDENTITY_INSERT PIM1 ON
INSERT INTO PIM1 (P_CLIENT_ID)
SELECT
Client.ID
FROMP_Client
So I am trying to insert both an identity values and a value from an other table while leaving the other columns blank. How do I go about doing this.
View 1 Replies
View Related
May 4, 2006
Hello,I have a query that I need help with.there are two tables...Product- ProductId- Property1- Property2- Property3PropertyType- PropertyTypeId- PropertyTypeThere many columns in (Product) that reverence 1 lookup table (PropertyType)In the table Product, the columns Property1, Property2, Property3 all contain a numerical value that references PropertyType.PropertyTypeIdHow do I select a Product so I get all rows from Product and also the PropertyType that corresponds to the Product.Property1, Product.Property2, and Product.Property3ProductId | Property1 | Property2 | Property3 | PropertyType1 | PropertyType2 | PropertyType3 PropertyType(1) = PropertyType for Property1PropertyType(2) = PropertyType for Property2PropertyType(3) = PropertyType for Property3I hope this makes sence.Thanks in advance.
View 3 Replies
View Related
Jan 20, 2014
I have two tables with the common column-name/ The first one include such columns as the name of football team, its country and budget. The second one include the team name, victories (quantity of it), lost games, and season. So I have the task to select such column as name, victories, lost games in 2nd table but only that ones that has budget over 1 mln? How to build select statement SELECT name, victories, lost g/ from TABLE2 WHERE budget >1000000 from TABLE1? Or should I equates the table1.name=table2.name??
View 6 Replies
View Related
Feb 12, 2007
mahesh writes "HI,
I am new to sql server.
can anybody help me
I have a table named tblqualificationmaster.
can i know the foreignkeys and the table related to this
tblqualificationmaster having foeign keys using stored procedure."
View 1 Replies
View Related
Dec 13, 2007
Hello:
I am running into an issue with RS2k PDF export.
Case: Exporting Report to PDF/Printing/TIFF
Report: Contains 1 table with 19 Columns. 1 column is static, the other 18 are visible at the users descretion. Report when printed/exported to pdf spans 2 pages naturally, 16 on the first page, 3 on the second, and the column widths have been adjusted to provide a perfect page span .
User A elects to hide two of the columns, and show the rest. The report complies and the viewable version is perfect, the excel export is perfect.. the PDF export on the first page causes every fith column, starting with the last column that was hidden to be expanded to take up additional width. On the spanned page, it renders the first column on that page correctly, then there is a white space gap equal to the width of the hidden columns and then the rest of the cells show with the last column expanded to take up the same width that the original 2 columns were going to take up, plus its width.
We have tried several different settings to see if it helps this issue or makes it worse. So far cangrow/canshrink/keep together have made no impact. It is not possible to increase the page size due to limited page size selection availablility for the client. There are far too many combinations of what the user can elect to show or hide to put together different tables to show and hide on the same report to remove this effect.
Any help or suggestion on this issue would be appreciated
View 1 Replies
View Related
Mar 20, 2006
I have a table A, with two ID columns. In a report both ID colums shouldbe shown with the actual value stored in a second table, BThe problem is, both IDs need to be looked up in B, but are not in thesame row.How do I do this in an efficient way? A sub select?Thanks,--John MexIT: http://johnbokma.com/mexit/personal page: http://johnbokma.com/Experienced programmer available: http://castleamber.com/Happy Customers: http://castleamber.com/testimonials.html
View 5 Replies
View Related
Feb 5, 2007
I have two tables:
Orders, with OrderID as primary key, a code for the client, and a code for the place of delivery/receipant.
Both the client and place of delivery should be linked to the table:
Relations, where each relation has it's own PrimaryID which is a auto-numbered ID. Now I want to substract my orders, with both the clientcode, and the place of delivery code linked to the relations table, so that for both the name and adress is shown.
I can link one of them by:
InnerJoin On Orders.ClientID = Relations.ClientID, but it's not possible to also link to the ReceipantsID. Is there a way to solve this?
View 7 Replies
View Related
Oct 26, 2015
Say you have a fact table with a few columns that all reference the same key column in a dimension table, you want to write a view to return the information for those keys?
USE MyTestDB;
GO
SET NOCOUNT ON;
IF OBJECT_ID ('dbo.FactTemp' ,'U') IS NOT NULL
DROP TABLE dbo.FactTemp;
[Code] ....
I'm using very small data at the moment, and the query plan and statistics don't really say which way.
View 2 Replies
View Related
Jul 19, 1999
What is the easiest way to determine the number of columns in a table (in a particular database)? Also is there any way to get the column id associated with each column (is there such thing)??
NOTE, this needs to be done with only knowing the name of the table and database and nothing else.
All help is greatly appreciated.
Thanks in advance.
ziggy
View 1 Replies
View Related
Sep 13, 1999
I need to find out the number of columns per table for ALL of my tables in
my database. This way I can compare my old copy of the database to the new and make sure that any new columns are there.
Thank you,
Tracy Young
View 2 Replies
View Related
May 12, 2008
Hi,
I have a table (that i get as a result of pivots and joins in a stored procedure) that has two columns with the same name, (each time the stored procedure is called, different columns will be in this situation). For every row, only one of these columns has a value and the other has NULL.
Does anyone know how to do prevent this situation during the JOIN that creates this table? OR of a way to merge these columns once they are there ?
Thanks !!
View 2 Replies
View Related
Oct 15, 2007
i have a table with 6 million rowwhat is the affect of increasing a varchar(255) columnto varchar(500)tiaMJ
View 1 Replies
View Related
Nov 27, 2007
HI,
I have a procedure when i need to get the columns of a table and store them in a temp table and process further.
For this i am planning to use sys.columns table as follows:
SELECT OBJECT_name(c.object_id) TableName, c.name ColumnName
FROM sys.columns AS c
Can I use this table in a procedure? Do I need to have any permissions to use this table?
If so, is there any alternative to get the columns of a table so that I wont use sys.columns table.
Please advice on this
Thanks in advance
View 4 Replies
View Related
Feb 18, 2007
Hi, I have 3 tables which are part of my query and I need to total one column from each of them, but cannot get it to work. I had this working within Access using the nz function (below), but when trying to use this within VS to produce my ASP.net code did not work, saying NZ was not a valid function. nz(HW.HGF)+nz(HD.HGF)+nz(HL.HGF) AS FWhere HW,HD,HL are the table names and HGF is the same column name in each which I want to total.Any help would be grately appreciated.Thanks
View 3 Replies
View Related
Apr 8, 2008
I have a 2 columns ID and DriverID.I need ID and DriverID column's in such a way that
what are the settings should be set?
ID DriverID
1 0001
2 0002
3 0003
4 0004
....
999 000999
View 3 Replies
View Related
Nov 15, 2005
I want to build a page that will handle all my list table... how can i get the names of the colums in specific table if i got the table name?
View 8 Replies
View Related
