Data Model Design For Query Performance
Apr 22, 2008
I have an opportunity to rebuild a database model with the express purpose of improving query performance. So given the following I have a few questions.
Table A (~500M records)
Primary Key Field (int)
Field 1 (varchar)
Field 2 (varchar)
Field 3 (varchar)
Field 4 (varchar)
Field 5 (varchar)
Table B (1B+ records)
Primary Key Field (int)
Foreign Key Field (int)
Field 1 (varchar)
Field 2 (varchar)
Field 3 (varchar)
Field 4 (varchar)
Field 5 (varchar)
* Assumed: Tables are inner joined on all queries. The database is readonly.
-- Most of my lookups are based on querying Field 1 of Table A. The data content of Field 1, Table A is 90% unique.
1) Would it be more beneficial to put the clustered index on Field 1 instead of the PK field in Table A?
2) Can an Identity column be non-clustered?
3) Alternatively, would it be beneficial to build a separate lookup table with just the PK & Field 1 of Table A, with a clustered index on the lookup table Field 1 which I join on Table A? (did that make sense?)
-- I have a secondary lookup that performs queries on Fields 1, 2, 3, 4 & 5 of Table B
1) Would it be more beneficial to create an additional indexed lookup column of the concantenated values of Fields 1-5 of Table B versus a covering index of all 5 columns?
2) Does a clustered index have to be unique?
3) Would a clustered index be more beneficial over Fields 1-5 or the special lookup column versus the PK or FK fields?
4) Would creating a special lookup table with just the requisite fields be more beneficial?
An extra question. The existing data model uses the CHAR datatype for all columns less than 9 characters wide and the columns are set to allow nulls. This requires every select statement to COALESCE() and RTRIM() all these columns. I intend to make all (affected) columns VARCHAR, NOT NULL with a default value of a 0-length string.
Will this enhance query performance?
Thanks in advance for any insight.
View 7 Replies
ADVERTISEMENT
Jul 27, 2015
I never created table on the basis of Data model diagram . I have to create the 3 table on the basis of given Data model diagram. There are 3 tables
1.md_geographyleveltype
2.md_geographylevel
3.md_geographylevelxref
I have tried to create 2 table but unable to create 3rd table.
CREATE TABLEÂ [dbo].[md_geographylevel](
[type_key] [int] NOT NULL,
[geog_key] [int] NOT NULL,
[Type_description] [nvarchar](50) NULL,
[Store_flag] [nvarchar](10) NULL,
[Type_short_desciption] [nvarchar](50) NULL,
[Code] ....
I am getting difficulties to create table script for table
md_geographylevelxref
How to create the table create script for
md_geographylevelxref
View 2 Replies
View Related
Aug 1, 2006
How to create a stored procedure to query the data mining model?
Where can I find some sample code?
Thanks.
Joe.
View 3 Replies
View Related
Nov 12, 2015
I have created 60 queries and added them to my data model in Power Pivot/Excel 2016. I created some calculated fields for one of my queries but I would LOVE to know how to just copy these across all queries in the data model instead of having to create them in the data model for one query at a time. That will take forever.Â
View 3 Replies
View Related
May 14, 2015
I am using sql2012 se and we want to use couchbase to store some data as documents. So the database will be Hybrid(partly SQLServer and Partly couchbase). However the database is still in the design phase. What are the things that we should keep in mind when designing this database from a design perspective? Our database which was completely SQLServer based(RDBMS) was using GUIDs everywhere based off NEWID() and prime goal is to get rid of GUIDs for the most part.
View 6 Replies
View Related
Jun 16, 2008
Hi there
We are thinking to create a small scale of the "Software On-Demand" model for Support Ticket Management System which means that the client can login to do a support tick and shared the application using the same URL.
Couple options though:
1) We design by table such has a SUPPORTS table with has a foreign key of the ClientID from CLIENTS table. This will cover that client can only can a specific data within the SUPPORTS table.
2) Alternatively, create a dynamic table which is created specifically for "Client B" for instance for SUPPORT table and it will be called such as ClientBSupport table. Something that can be common it will be shared such as like look-up table.
Couple consideration though:
1) Considered client wanting to backup this table. Obviosly it has too querying based on the client id to get the right data.
2) Security withing that table. Is the data needs to be incrypted ... I guess this is expensive excercise.
3) Possibly have a model that access to this table in querying directly etc etc.
4) If the Support table is getting larger and larger and it's not fair for small client accessing the small information which has equal to the one that has big records? Cause it's sharing table.
Any inputs I am really appreciated.
Thanks
View 3 Replies
View Related
Dec 11, 2007
Hi,Facts:I created a database to support an application that tracks events ondifferent objects. The two main tables are tbl_Object andtbl_EventLog. Each table has unique ID and on the tbl_EventLog thereis FK for a record in the tbl_Object. The events are inserted all thetime for the same or different objects from the tbl_Object. There areabout 600,000 objects in the tbl_Object and 1,500,000 (and growing)events in tbl_EventLog.Question:The user often wants to know what the last event was for a specificobject.What is the best way of retrieving the last event?Should I simply do a max(eventdatetime) on a specific object? orShould I add a LastEventID column to tbl_Object and update it everytime a new event is inserted? or any other way to implement it?I chose the second method because I didn't think it made sense searchthe event table everytime the user wants to know the last event, but Iwanted to know what the experts thought.Please let me know what you think.Thank you,Oran Levin
View 5 Replies
View Related
Oct 15, 2015
I have one database which is multi tenant, every table has tenant id  and every tenant is mapped with separate file group. Now can I achieve below  :?
1. Can I take the backup of only one tenant and restored in separate database?
2. Can I takeout the data of one tenant quickly?
3. Can I undo the data of a particular tenant  with 2 days older data without disturbing the other tenant ?
View 3 Replies
View Related
Jan 31, 2008
In order to setup my forecasting mining model I have created a special view that runs against my fact table and creates time series on the level I need.
Code Snippet
Select DFUKEY, DATE, QTY from Dim_FACT where DFKEY like '020%'
So I get the following input fr my model:
time series key (e.g. DFUKEY)
date (time key)
QTY (to be predicted)
For testing purposes I created a small view (similar to AdventureWorks) that only contained a few time series. The model was created and processed in ~2 minutes or less. The viewer came up almost immediately and I was able to see results.
Now my real view has about 25000 time series I need a forecast for and that I also like to review in the viewer. If I create a mining model against that bigger view the processing takes ~15m or so and the viewer is likely to time out.
The worst part thought is when I try to get the forecast for a time series (see query below) it takes minutes before the answers come back.
Code Snippet
SELECT FLATTENED [DFUKEY], PredictTimeSeries([QTY], 12)
FROM [TimeSeries]
WHERE [DFUKEY] = '02078147AG'
I would understand if without the WHERE clause it would take long but as is?
I am pretty sure I created the model the right way so I wonder if somebody can give me recommendations as to how forecast larger numbers.
Finally I intend to call the foarcast for a particular series using an Action from within a cube.
Thanks in advance.
Dirk
View 6 Replies
View Related
Nov 29, 2006
I am using BI Dev Studio for SS2005 in a research (as opposed to a production) environment. Often I want to compare the results of multiple models using the same attributes. If I switch to a different model, the Design view completely resets. Is there any way to retain the same field names with different models in the Design view?
My current workaround is to give my models similar names with AR, DT, CL, LOG, NN suffixes and make global changes in the DMX.
I have consulted the following without finding an answer:
http://msdn2.microsoft.com/en-us/library/ms178445.aspx
http://msdn2.microsoft.com/en-us/library/ms175642.aspx
http://msdn2.microsoft.com/en-us/library/ms175678.aspx
http://msdn2.microsoft.com/en-us/library/ms175637.aspx
Thanks for your help,
Sam
View 3 Replies
View Related
Jan 14, 2006
I develop my database with SQL Server 2005 Express.I have a member profile, stored in tblMember.In this profile I have a field "Religion" and a field "Country".I COULD store the value of these fields just in those fields, eg Christian in the Religion field and "Sweden" in the Country field.ORI could make a seperate table for each, eg for tblReligion:ReligionID (PK)ReligionName (string)and then in the tblMember make a FK to the PK of this table.but I have many of these kind of fields (where a FK might be needed) in my tblMember.So if I want to search in my members, I will first have to obtain the PK of the other tables.My questions:is this good DB design?Will I get performance problems when I do it like this?Are there better solutions?Thanks ahead!!
View 3 Replies
View Related
Jul 25, 2005
I have created an internal “e-mail� type system in our business application. The idea was pretty basic have a message table which would hold the data such as To, FROM, blah blah. Then there would be an Inbox table which would have the User Name and the Message.ID from the message table. By joining these two you would have the person’s inbox. Each table has an ID column which is the clustered index, The Inbox table has an additional Index on the User Name. These tables are very busy and large, here are the stats.
Messages – 860,000 rows est. Total
Inbox – 2,584,000 rows est. Total
For June there were 144,000 Messages of which 780 went to multiple recipients causing 276,000 Inbox references.
Everything works except getting a persons Inbox can get very slow. To address this we are currently Index Defragging and Re-Indexing. Now these action appear to be correcting the problem but only for a short time which is why we are doing it several times a day.
Solution?
I think the problem is that my most used Index (User Name in the Inbox) is not clustered? And I should have a larger Fill Factor than 90%?
Any advice greatly appreciated, the transaction logs are killing me.
View 4 Replies
View Related
Aug 10, 2005
Hi,Can anyone put a name to the following type of design...I need to be able to modify the amount/names of data "fields" storedfor an entity so instead of representing the data stored for an entityin fields associated with that entity you would relate it to an anotherentity which would contain a datafield type field and its associatedvalue, so different rows in the parent table could possibly havedifferent datafields associated with it.E.gThe parent table is "person" and it holds common data amongst allpeople - it has primary key of personID.There is another table assoicated with person called "personDetails".This would have fields "personID", "dataType" and "dataValue". Itmight have records like the following:personID, dataType, dataValue1,phone,1234561,email,t...@test.com1,state,NSW2,phone,9874563212,state,VIC3,phone,789456123Of course this is just an example - the parent entity could beanything.I've seen a few XML files implement this type of design. e.g<job><jobNumber>XYZ12345</jobNumber*><jobCreated>12-03-2005</jobCre*ated><jobdeatils><jobdetail type="WEIGHT">167</jobdetail><jobdetail type="WEIGHTMEASURE">KGS</jobd*etail><jobdetail type="CUBIC">45</jobdetail><jobdetail type="CUBICMEASURE">M3</jobdet*ail><jobdetail type="NUMPIECES">345</jobdetai*l></jobdetails></job>Hope someone can put a name to this sort of design. I want to researchinto this to see how people search effectively in this design and itsperformance compared to traditional methods etc etc.I would be implementing this design in MS SQL and it would be accessedby an 400-500 users via ASP.NET.Any help would be appreciated. Thanks Adam.
View 2 Replies
View Related
Jul 28, 2006
Hi
I have a database that stores all the data in one table that is only the data we are required to store hence it didnt require any other tables
the table has five colunms
Orderid, ordercost, orderdate, ordersystem, orderref
it is used in an web application that seraches for all the rest of the information using an orderid and displays the rest of the details the size of the table currently is about 123000 records that is increasing by 20000 every week.
i have an archive of the same data that has 7,666,000 records that are also going to be placed into the same database.
my question is is that a good idea of doing this?
are there any performance issues that i need to be aware of as currently the application runs quite quick?
View 15 Replies
View Related
Dec 5, 2005
Running 2005 Beta 3 Refresh. When I first deploy, it works fine. Subsequent deployments yield the following error:
View 9 Replies
View Related
Feb 13, 2002
Ok, I'm doing a football database for fixtures and stuff. The problem I am having is that in a fixture, there is both a home, and an away team. The tables as a result are something like this:
-------
Fixture
-------
fix_id
fix_date
fix_played
----
Team
----
tem_id
tem_name
-----------
TeamFixture
-----------
fix_id
tem_id
homeorawayteam
goals
It's not exactly like that, but you get the point. The question is, can I do a fixture query which results in one record per fixture, showing both teams details. The first in a hometeam field and the second in an away team field.
Fixture contains the details about the fixture like date and fixture id and has it been played
Team contains team info like team id, name, associated graphic
TeamFixture is the table which links the fixture to it's home and away team.
TeamFixture exists to prevent a many to many type relationship.
Make sense? Sorry if this turns out to be really easy, just can't get my head around it at the mo!
View 2 Replies
View Related
Jun 23, 2006
Hello Everyone,I have a very complex performance issue with our production database.Here's the scenario. We have a production webserver server and adevelopment web server. Both are running SQL Server 2000.I encounted various performance issues with the production server with aparticular query. It would take approximately 22 seconds to return 100rows, thats about 0.22 seconds per row. Note: I ran the query in singleuser mode. So I tested the query on the Development server by taking abackup (.dmp) of the database and moving it onto the dev server. I ranthe same query and found that it ran in less than a second.I took a look at the query execution plan and I found that they we'rethe exact same in both cases.Then I took a look at the various index's, and again I found nodifferences in the table indices.If both databases are identical, I'm assumeing that the issue is relatedto some external hardware issue like: disk space, memory etc. Or couldit be OS software related issues, like service packs, SQL Serverconfiguations etc.Here's what I've done to rule out some obvious hardware issues on theprod server:1. Moved all extraneous files to a secondary harddrive to free up spaceon the primary harddrive. There is 55gb's of free space on the disk.2. Applied SQL Server SP4 service packs3. Defragmented the primary harddrive4. Applied all Windows Server 2003 updatesHere is the prod servers system specs:2x Intel Xeon 2.67GHZTotal Physical Memory 2GB, Available Physical Memory 815MBWindows Server 2003 SE /w SP1Here is the dev serers system specs:2x Intel Xeon 2.80GHz2GB DDR2-SDRAMWindows Server 2003 SE /w SP1I'm not sure what else to do, the query performance is an order ofmagnitude difference and I can't explain it. To me its is a hardware oroperating system related issue.Any Ideas would help me greatly!Thanks,Brian T*** Sent via Developersdex http://www.developersdex.com ***
View 2 Replies
View Related
Oct 12, 2015
I've question about how to handle structural datamodel changes in a datasource of PowerPivot. Suppose I'm developing a starmodel in SQL Server and sometimes a datatype changes or a name of a field changes in a table. It seems to me that PowerPivot handle this not gracefully as Analysis MD does (mostly). I received an error because of a wrong fieldname or even no error when a dattype changes in PowerPivot. Is this common or do I something wrong here. Does this mean that every time the datamodel changes the PowerPivot should be recreated? Or am I missing the clue here?
View 6 Replies
View Related
Apr 22, 2015
I am getting ErrorCode 8 while loading the data from stage to model. I have checked my error view it states that "Member Code is Inactive".
Initially I have loaded same set of data in Model from MDS Stage table but then deleted with ImportType = 5 which removed all the data from the MDM model.
Now i want to load it back but its giving the Error Code 8 .. Before loading the same data i have changed the stage table Importtype to 2 and Importstatusid to 0.
View 6 Replies
View Related
Sep 10, 2013
I don't know if the question has been nailed down. Aside from deleting tables, can we delete the *content* of data within the tables. It doesn't seem crazy that, if you can pull in data from a feed then you should be able to remove the content out again (without also destroying the user's meta-data work ). Reasons for this include:
- Security (a user may not have rights to see *my* data and should go refresh their own)
- Size (workbook doesn't need to have GB's of irrelevant data saved to disk in a workbook if it was just useful during development phase to a pre-production data feed)
- Bad data (pre-production data feed is not good data)
- User-friendliness (data feed was refreshed 2 years ago and workbook was saved to file server. Users shouldn't be presented with irrelevant data, but should get empty pivot tables until they go do their refresh)
Obviously Excel internally knows how to clear out PowerPivot data, given the prompt shown here: [URL] ....
But how does a user initiate this on their own (corruption aside)?
Previous time this question was asked, without a real resolution: [URL] ....
View 8 Replies
View Related
Mar 16, 2007
I'm trying to write some T-SQL code for adding some fills and thrillsto backups but ran into a snag. Is there a way to query what backupmodel a database is using? I can see it if I right click a database,go to properties, select the options tab, and then can see the currentmodel (full, bulk-logged, and simple) but I would need to hard codeeach database. In the spirit of automation, I'd rather query for thetype and then work around that instead. But I've been all over themaster database and can't seem to find any field that will indicatethe backup type.Any guru's out there that can point me in the right direction?Thanks,Utah
View 2 Replies
View Related
Jan 31, 2008
Hi, it's my first time for using report model as the datasource of reporting service. But I don't know how to build the query string to get data from the report model.
Can anyone help me?
Thanks
View 3 Replies
View Related
May 17, 2007
Hi,
I am finding nice examples data model of CMS. Do you know any link to data model phorum or any link to data model examples(nice)?
Thanks
View 3 Replies
View Related
Jun 16, 2008
Why would we need to know the data model to write queries?
Sorry but its been a long time since I've worked with databases and I don't remember!
Could anybody lead me in the right direction please?
View 1 Replies
View Related
Jun 16, 2008
Why would we need a data model to write queries?
Sorry but its been a loong time since I've worked with databases and I barely remember.
Could anybody lead me in the right direction please?
Thanks.
View 1 Replies
View Related
Dec 5, 2006
Hi guys,
I added a Named Query on my existing data source view and I recreate the report model, however, I am unable to see the Named Query on the Report Model, I can see only the tables.
Does anybody has any idea?
Sincerely,
View 1 Replies
View Related
Dec 12, 2006
I have a question about writing a prediction query against a clustering model that has the same column added more than once.
Per Jamie, I can accomplish some crude weighting by adding a column to my model multiple times. See this post for an explnation... Now that I have that worked out, I was wondering how my DM query would look? If I have Input_A1, Input_A2 , & Input_A3 all being source from the same column in my structure do I have to reference all three when writing my prediction query?
View 1 Replies
View Related
Oct 19, 2015
I need to develop a language specific dwh, meaning that descriptions of products are available from a SAP system in multiple languages. English is the most important language and that is the standard. But, there are also requirements of countries that wants productdescriptions in their language.Â
Productnr Productdesc Language
1       product    EN
1       produkt    DE
One option is to column the descriptions, but that is not very elegantly. I was thinking of using bridge tables to model this but you have to always select a language in a filter (I think)..
I'm thinking of a technical solution, such that when a user logs on, the language is determined and a view determines whether to pick a certain product table specific for a certain language. But then I don't have the opportunity to interchange the different language specific fields in a report (or in my case PowerPivot).
View 2 Replies
View Related
Apr 25, 2008
We have the following scenario,
We have our Production server having database on which Few DTS packages execute every night. Most of them have Bulk Insert stored procedures running.
SO we have to set Recovery Model of the database to simple for that period of time, otherwise it will blow up our logs.
Is there any way we can set up log shipping between our production and standby server, but pause it for some time, set recovery model of primary db to simple, execute DTS Bulk Insert Jobs, Bring it Back to Full recovery Model AND finally bring back Log SHipping.
It it possible, if yes how can we achieve this.
If not what could be another DR solution in this scenario.
Thanks Much
Tejinder
View 6 Replies
View Related
Aug 2, 2007
Hi,
I have been given the assignment to come up with an Entity Relationship diagram of an existing database. The database wasn't designed by me. I was wondering if there is a standard scientific way of displaying the relationships between the various tables at a functional level on Visio. Since, I didn't design the database in the first place, I don't know how to take a step backward and recreate the ERD from the given database.
Any help will be appreciated.
thanks
View 4 Replies
View Related
Sep 28, 2007
Hi,
I would like to know best way to design the database for the following requirement. I have a collection of tree nodes. each node has a type and set of attributes and a parent node (except for the node which has no parent). node type refers to the level of the node in the tree. child node inherits the attributes from the parent node (similar to object oriented programming where derived class inherits properties of the base class). user can add/update/delete nodes from the tree and user can choose to override the attributes of the parent node in child node.
what is best way to store this type of data? should there be a separate table for each node type (level in the tree). but the problem with this approach is that we need to duplicate the columns of the parent node, because user can overwrite the parent node attributes in the child node. there can be more than one at the same level and all of them share same set of attributes. this concept is exactly like inheritance in object oriented programming. as far as the data is concerned, there are around 15 levels, around 30K nodes and 30 attributes spread across different node levels.
thanks,
RK
View 1 Replies
View Related
Jan 24, 2007
This may be a noob question... but I am not able to create a data model and its frustrating me. :)
I made the Data Source... I made the Data Source View (just grabbed a table) and made the report model... but the model is empty >< When it autogenerated the model, it did yell at me about the table not having a primary key... do all the tables have to have primary keys???
Please help I'm a bit lost.
View 2 Replies
View Related
Jul 31, 2007
I am trying to do the examples using SQL Server 2005 Reporting Services (2006) by Brian Larson.
The example says to use GDS as the data source yet all I see is Adventure Works.
The tables for AW are not the same as the tables for GDS.
Is there something I need to do or does the DBA need to install something else.
View 1 Replies
View Related