Better Practices Wanted For Cascading Inserts Of Hierarchical Data From Staging Tables

Aug 28, 2007

I apologize if this has been asked, but I can't find a complete answer.

We have a situation with parent/child tables which have an identity column as their PK. We need to be able to insert into the live tables from staging tables. The data in the staging tables are related via a surrogate key.

I have found the OUTPUT clause, but that can only refer to columns of the actual table (since there is no FROM clause in an INSERT). Our current best solution to this problem involves adding bogus "staging" columns to the destination tables, and removing them after we've inserted everything from staging. This is an unattractive solution to say the least.

I'll give an example that mirrors our actual solution, and ask if anyone has a better solution?
----------




Code Snippet
CREATE TABLE [dbo].[TABLE_A](
[ID] [int] IDENTITY(1,1) NOT NULL,
[DATA] [nchar](10) NOT NULL,
[STAGING_COLUMN] [bigint] NULL,
CONSTRAINT [PK_TABLE_A] PRIMARY KEY ([ID] ASC)
)
GO
CREATE TABLE [dbo].[TABLE_B](
[ID] [int] IDENTITY(1,1) NOT NULL,
[A_ID] [int] NOT NULL,
[DATA] [nchar](10) NOT NULL,
[STAGING_COLUMN] [bigint] NULL,
CONSTRAINT [PK_TABLE_B] PRIMARY KEY ([ID] ASC)
)
GO
ALTER TABLE [dbo].[TABLE_B]
ADD CONSTRAINT [FK_TABLE_A_TABLE_B] FOREIGN KEY([A_ID]) REFERENCES [dbo].[TABLE_A] ([ID])
GO
CREATE TABLE [dbo].[STAGE_TABLE_A](
[A_Key] [bigint] NOT NULL,
[DATA] [nchar](10) NOT NULL
)
GO
CREATE TABLE [dbo].[STAGE_TABLE_B](
[B_Key] [bigint] NOT NULL,
[DATA] [nchar](10) NOT NULL,
[A_Key] [bigint] NOT NULL
)
GO


The STAGING_COLUMN columns are the ones that will be added before, and dropped after.






Code Snippet
DECLARE @TABLE_A_MAP TABLE (
A_ID INT,
A_Key BIGINT
)
INSERT INTO TABLE_A (DATA, STAGING_COLUMN)
OUTPUT INSERTED.ID, INSERTED.STAGING_COLUMN INTO @TABLE_A_MAP
SELECT DATA, A_Key FROM STAGE_TABLE_A
INSERT INTO TABLE_B (A_ID, DATA)
SELECT TAM.A_ID, STB.DATA
FROM STAGE_TABLE_B STB INNER JOIN @TABLE_A_MAP TAM ON TAM.A_Key = STB.A_Key






This seems to work, but I'd really like another alternative. Even though this is happening when nobody else is using the database, I cringe at the thought of adding and removing columns just to make this work.

Here are a few of my constraints:



The above is a simplification of the actual problem. The actual problem goes about five levels deep (hence the B_Key in STAGE_TABLE_B). At the top level, our larger customer will have 100,000 rows to insert. Each level will average 3 times as many rows as the next higher level, so we're talking about real volumes here.

This has to finish over the course of a weekend.

This has to be delivered to QA this Friday
Thanks for any help or insight.

View 3 Replies


ADVERTISEMENT

Bulk Inserts To Data Warehouse - Best Practices?

Jul 20, 2005

Hello all,I just started a new job this week and they complain about the length oftime it takes to load data into their data warehouse,which they do once a month.From what I can gather, they rebuild the indexes before the insert with an80% Fillfactor, then insert the data (with theindexes enabled), then rebuild the indexes with a 100% Fillfactor.Most of my RDBMS experience is with a different product. We would havedisabled the indexes and Foreign Keys, loaded the data, thenre-enabled them, moving any records that violated the constraints into anappropriate audit table to be checked after.Can someone share with me what the accepted "best practices" are for loadingdata efficiently into a data warehouse?Any thoughts would be deeply appreciated.Steve

View 2 Replies View Related

Data Warehousing :: Copy Data From Staging Tables To Other Instance Master Tables?

Aug 14, 2015

I need to copy data from warehouse tables to master tables of different SQL instances. Refresh need to done once in an hour. What is the best way to do this? SQL agent jobs or SSIS packages?

View 3 Replies View Related

Staging Tables

Dec 7, 2005

I've found staging tables to still be necessary (for example, when you have a large dimension table that you can't load into memory.  The memory restricted lookup is too slow as it sends a query for each row).  I have a few questions:

View 7 Replies View Related

Hierarchical Query From Two Tables

Nov 17, 2014

I have created a store procedure as below:

WITH TextType AS
(
SELECT AppTxtTypeId,AppTxtTypeCode, AppTxtTypeParentCode, Name,Description,Active,SortOrder ,0 as TypeLevel,AppTxtTypeId as parentId
FROM [ApplicationTextTypes]
WHERE AppTxtTypeParentCode IS NULL

[Code] ....

From this i am able to get data in the below format:

Parent
--Child1
--Child2
---Subchild1-Child1
---Subchild2-Child1
---Subchild1-Child2
---Subchild2-Child2

Actually my requirement is :

Parent
--Child1
---Subchild1-Child1
---Subchild2-Child1

--Child2
---Subchild1-Child2
---Subchild2-Child2

View 1 Replies View Related

Guidance Needed: Loading Hierarchical XML Into Relational Tables

Aug 8, 2007

I've got a lot of XML like this (simplified):




Code Snippet





... 8 MORE


... 9 MORE TIMES






I need to get this into three existing SQL Server 2005 tables, each with identity columns for their primary keys:




Code Snippet
CREATE TABLE ELEMENT1 (

[ID] INT IDENTITY
)

CREATE TABLE ELEMENT2 (

[ID] INT IDENTITY,
[ELEMENT1_ID] INT
)

CREATE TABLE ELEMENT3 (

[ID] INT IDENTITY,
[ELEMENT2_ID] INT
)





With primary and foreign keys as you'd expect, and, of course, many more columns!

How would I get this into tables through SSIS, preferably in a high-performance manner (there may be several gigabytes of XML to load).

The issue, of course, is that in order to insert an ELEMENT2 row, I need the ID from the coresponding ELEMENT1, etc.

Any ideas or pointers to articles would be welcome.

View 14 Replies View Related

SQL Server 2012 :: Index View Won't Work With Hierarchical Tables?

Apr 4, 2015

Consider following code:

SELECT e1.EntityIdentity as CompanyID
FROM dbo.Entitye1 --company
JOIN dbo.EntityAssociationea
ON e1.EntityID = ea.EntityID1
JOIN dbo.Entitye2 --user
ON ea.EntityID2 = e2.EntityID

This query occurs as a sub-query in many stored procedures where exists a WHERE clause that includes CompanyID IN (above query).

Since dbo.Entity and dbo.EntityAssociation change infrequently I thought that an indexed view would really improve performance. But I've found one of the seemingly undocumented Microsoft features when trying to create the clustered index and get the following error msg:

Msg 1947, Level 16, State 1, Line 1
Cannot create index on view "ROICore.dbo.vEntityEntityAssociation_CompanyUser". The view contains a self join on "ROICore.dbo.Entity".

I really need to improve performance on this subquery. Entity currently has over 20m rows and EntityAssociation over 35m rows and both are growing.

How to improve performance? Indexes on both tables for the most part give index seeks, but I thought my saviour might be the index view. Obviously this will not work.

View 3 Replies View Related

Sliding Window Scenario For Shifting Partitions...do All Indexes Need To Be Built On The Staging Tables?

Oct 31, 2007

Kimberly Tripp describes a recipe for switching partitions in and out, thru the use of staging tables, when it comes time to "slide the window" on a partitioned table. She says that the clustered index (on staging) must be the same as that chosen for the partitioned table itself but she doesnt discuss whether or not all of the non clustered indexes need to be the same too once the
ALTER TABLE Orders
SWITCH PARTITION 1
TO OrdersOctober2002
and
ALTER TABLE OrdersOctober2004
SWITCH TO Orders PARTITION 24
run. For the data being switched out, I wouldnt want to do anything extra. For the data being switched in, I'd like to understand if she is implying that all other indexes would be built automatically as a result of the 2nd ALTER statement?

Kimberly's article is at http://www.sqlskills.com/resources/Whitepapers/Partitioning%20in%20SQL%20Server%202005%20Beta%20II.htm#_Toc79339965

View 1 Replies View Related

Hierarchical Table Functions Vs Hierarchical CTE

Jun 9, 2006

Recently I was in need of a hierarchical tree data. I learned about CTE and how they can be used to build hierarchical data with simple syntax. I used CTE and was through with the task. Later during free time, I tried to compare CTE approach with the traditional SQL 2K Table Function approach. It was surprising to see the query costs when I ran both the modes at one go...

Query Cost (relative to batch) : 0.49%
Query Text : Select * From fn_GetTree(8);

Query Cost (relative to batch) : 99.51%
Query Text : with treedata (id, parentid, status, prevStatus, lvl) as (select ...)


What does that indicate? Does it mean that the Table Function approach is much faster than CTE? I am sure that I was not making unwanted Joins in the CTE mode.

Can someone explain why that huge difference is there? And what the scenarios where CTE is better over Table Functions?

View 8 Replies View Related

Cascading Deletes - Which Is Better - A Trigger Or Foreign Key Cascading Delete?

Aug 17, 2005

I need to implement my cascading deletes on a SQL database.  Is it better (performance/reliablility-wise) to use the Foreign Key Cascading Deletes or to just write my own triggers to do the deletes?I was hoping someone had experimented and found which works best.

View 2 Replies View Related

Loading Old Data From Staging

Jan 3, 2007

I'm populating a table (B) in SQL Server from a Staging table (A) using a stored procedure.At any point of time, the Staging table holds 60 months' old data.In the first load of the destination table B, I get 13 months of old data whereas for every subsequent load,I need to load the data for the most recent month and delete data for the 1st(oldest) month. For example, if the load procedure runs on December 02,2006, it should pick data for the month of November,2006 from the Staging table and delete data for the 1st month.

I have a column DATA_MONTH_KEY in table B which maps to the column DATA_MONTH in my staging table A. I get the data for the first 13 months using:

(B.DATA_MONTH_KEY BETWEEN ( DATEADD(month,-13,@startdate)) AND @startdate) where startdate is the current date on which the procedure for populating table B is run. I get the value of startdate from a function.

How do i get data for the most recent months and delete the oldest month in subsequent loads?

Any help appreciated. Thanks.



View 7 Replies View Related

Storing Staging Data As XML In SQL 2000.

Aug 25, 2005

Scenario:
We want to stage all data coming into our database from various
applications.  We have a generic control where all data goes
through to get to the database.

We want have the data staged in a 'staging database'.  My question
is, what would be the best way to store the staging database in the
database.

I'm thinking storing it as XML in a SQL column.  The reason for
this is because the data could change from application to
application.  It would be impossible to create a relational table
because maintenance would be a nightmare.

Has anyone ever done something like this.  Would you save the node names in another table in SQL?

Any help would be appreciated as I'm new to XML.

Thanks,

ScAndal

View 1 Replies View Related

Hierarchical Data Model

Sep 28, 2007

Hi,

I would like to know best way to design the database for the following requirement. I have a collection of tree nodes. each node has a type and set of attributes and a parent node (except for the node which has no parent). node type refers to the level of the node in the tree. child node inherits the attributes from the parent node (similar to object oriented programming where derived class inherits properties of the base class). user can add/update/delete nodes from the tree and user can choose to override the attributes of the parent node in child node.
what is best way to store this type of data? should there be a separate table for each node type (level in the tree). but the problem with this approach is that we need to duplicate the columns of the parent node, because user can overwrite the parent node attributes in the child node. there can be more than one at the same level and all of them share same set of attributes. this concept is exactly like inheritance in object oriented programming. as far as the data is concerned, there are around 15 levels, around 30K nodes and 30 attributes spread across different node levels.

thanks,
RK

View 1 Replies View Related

Hierarchical Data In Result Set

May 1, 2006

How can I create a function that returns hierarchical data from a table with this structure:

- CategoryID
- CategoryName
- CategoryFather

I want to bring the result set like this...

CategoryID | CategoryName | CategoryFather | HierarchicalLevel
1 | Video | 0 | 0
2 | DivX | 1 | 1
3 | WMV | 1 | 1
4 | Programming | 0 | 0
5 | Web | 4 | 1
6 | ASP.Net | 5 | 2
7 | ColdFusion | 5 | 2


How can I do this? Does anybody has a sample code? I need this on SQL Server 2000 and if it's possible (but not too necessary) in SQL Server 2005.

Thanks.

View 5 Replies View Related

Flatten Hierarchical Data

Apr 17, 2008

Hello

I have a problem that I am hoping somebody can help me with!

I have built a hierarchy using the adjacency list model so I have records with an id that maps to the parent record so my hierarchy looks something like this:-

Newspapers

National Newspapers

Daily Express

Express Publications
Express Supplements
Daily Mail

Mail Publications
Mail Supplements
Mirror

So my table would look like below:-

1 Newspapers Null
2 National Newspapers 1
3 Daily Express 2
4 Express Publications 3
5 Express Supplements 3

and so on. What I would like to be able to do is flatten out the hierarchy so I get something like below where each level is in a column.

NewsPapers National Newspapers Daily Express Express Publications
NewsPapers National Newspapers Daily Express Express Supplements

Ive used CTE's for displaying the hierarchy and producing aggregate figures when joing the hierarchy to spend information but am struggling to come up with any code that would produce a flattened hierarchy.

Any help would be greatly received!

Thanks

Rich


View 5 Replies View Related

Import Hierarchical Data

Feb 17, 2006

hi folks,

I have to import hierarchical text files like:
32;country;city;postalcode;street
21;name;firstname;salutation;title;age;nickname
21;name;firstname;salutation;title;age;nickname
...

additionally I have to eleminate doubles. what is the best way for this problem ?
I have set up a flatfilesource with two columns and a conditional split on the first column
so now I have an output with [country;city;postalcode;street] and one with [name;firstname;salutation;title;age;nickname]. How do I split this in columns, put it in a dataset keeping the relations and remove doubles ?

Iam looking forward for any helping idea.

rgrds,
matze

View 11 Replies View Related

Creating Different Staging Table With Data Extensi

Jun 11, 2008

Hi All,

I am extracting source data which is in txt fille to OLE DB destination. But data of each day I want to save in different staging table. For Eg; tblProduct20081206, tblProduct20081207. How can it be done. I have seen lots of posting and script when destination is Txt. I want to use same table for staging but want to create different table for each day with adding date extension.

Please Help

View 2 Replies View Related

Copy Data From Staging Table To Production?

Apr 12, 2015

I am trying to insert data from staging table to production table. In the staging table I only have period or date but no primary key.

This is my staging table

Create stagingtable(
[Period] [char](7) NOT NULL,
[CompanyCode] [varchar](100) NOT NULL,
[total] [int] NULL,
[status] [varchar](50) NULL
)

Create Production(
[Period] [char](7) NOT NULL,
[CompanyCode] [varchar](100) NOT NULL,
[total] [int] NULL,
[status] [varchar](50) NULL
)

I get this every month. What can I do to make sure only unique record are loaded into production table with no duplicate from previous month.

View 5 Replies View Related

Generic Staging Design Of Data Warehouse

Jan 6, 2006

I have a question about staging design using SSIS. Has anyone come up with an ETL design that would read table names from a generic table and dynamically create the ETL to stage the table.

1. Have a generic table which would have table name and description and whatever else that was required.

2. Have a master ETL that would enumerate through the table and stage all the table names found in the generic table.

This way I wouldn't have to create an ETL which would hardcode the names of 300-500 tables and have the appropriate 300-500 data sources and targets listed.

Not sure if I am making sense but I hope someone understands the attempt.

thanks

View 10 Replies View Related

Node ID - Loading Hierarchical Data

Aug 3, 2013

CREATE TABLE #Source
(
Id int identity(1,1)
,categoryint
,Leaf_Node_code varchar(10) --
,Level1_Name varchar(20)
,Level2_Name varchar(20)

[Code] ....

Here category 1 has 3 levels ,

category 2 has 4 levels ,
category 3 has 5 levels ,

Below is the target table, here Leaf_Node_code should populate to only for leaf nodes for each category .. Need to populate Node_id with hierarchical data

I am unable frame a sql query to handle different levels , in future #Source may have more levels .

How to handle multiple hierarchy levels .. here only leaf node should have Leaf_Node_code

CREATE TABLE TARGET_TABLE
(
ID INT IDENTITY(1,1) primary key
,Node_id HIERARCHYID
,category int
,Parent_id int references TARGET_TABLE(id)
,Leaf_Node_code varchar(10)
,Namevarchar(20)
)

Here is the expected output:

IDcategoryParent_idLeaf_Node_codeName Node_id
11NULLNULLWorld
211NULLAsia
312101India
42NULLNULLa
524NULLaa
625NULLaaa
726102aaaa
83NULLNULLb
938NULLbb
1039NULLbbb
11310NULLbbbb
12311103bbbb

View 1 Replies View Related

Hierarchical Table - How To Get Data From Users

Nov 30, 2013

I have two table. Department is hierarchical table.

Department
--- id (int primary key)
--- name (varchar)
--- parent (int)

Users
--- Id
--- name
--- department_id

This query return all data from departments. But i cannot understand how get data from users

SELECT t1.name AS lvl1, t2.name as lvl2, t3.name as lvl3
FROM Department AS t1
LEFT JOIN Department AS t2 ON t2.parent = t1.id
LEFT JOIN Department AS t3 ON t3.parent = t2.id

View 1 Replies View Related

Implementing Inheritance For Hierarchical Data

Oct 12, 2007



Hi,
in my application, the data is in hierarchical format. there is a tree with set of nodes having parent child relationships. this data can be stored either through adjacency or nested set model approach. this is fine. but the issue here is that each child node inherits the properties of its parent node, parent's parent node and so on until the root node. lets say root node has two attributes A1 and A2 and they are stored in two columns in a table. but its child nodes inherits this data from its parent and it has its own extra attributes. so should I copy parent's data for the child node as two additional columns? the problem is that there are around 15 levels in the tree and the attribute list grows from top to bottom in the tree. lets say I need to find all the attributes for a leaf node in the tree (both direct and inherited), if I am not storing the inherited attributes for each node, then I need to walk-up the tree and find all the inherited attributes. there are around 30K nodes and each node has around ten attributes. xml is not option because of large volumes of data and auditing and reporting on individual nodes. what is the best way to store this type of data? my current approach is to have an attribute table having nodeid as a foreign key and only store the direct and NOT the inherited attributes of the node in the table, but this means to find all the attributes for the node, I need to gather the attributes of all the parents until the root node. I can't see any easy way out for this.

View 4 Replies View Related

SQL Best Practices Analyzer Rule: Use Of Schema Qualified Tables/Views

Dec 17, 2005

SQL BPA says the following:"One or more objects are referencing tables/views withoutspecifying a schema! Performance and predictability of theapplication may be improved by specifying schema names.""When SQL Server looks up a table/view without a schemaqualification, it first searches the default schema and then the'dbo' schema. The default schema corresponds to the currentuser for ad-hoc batches, and corresponds to the schema of astored procedure when inside one. In either case, SQL Serverincurs an additional runtime cost to verify schema binding ofunqualified objects. Applications are more maintainable andmay observe a slight performance improvement if objectreferences are schema qualified."How important is to specify the schame (dbo. in my case) instored procedures? Will it really improve performance if I goand fix each object that is missing "dbo."?The problem is I have thousands and thousands of themwith no schemas. Before I invest a lot of time fixing themI am trying to determine if it's really worth it or not?Thank you

View 1 Replies View Related

Staging Data In ##Table In A OLEDB Command Task

Apr 24, 2008



Hi All,

In one of my interfaces,i am trying to Stage data in a ##table from a Flat file,so that i can iterate through the data in for each loop afterwards.

I have written a stored Proc to do this.But what i am observing is in the ##Table only the last Row will be persisted.

The SP looks like this




Code Snippet
CREATE PROCEDURE USP_SVInsS093Data_V001
@DetailRECTYPE nvarchar(5),
@StoreNumber int,
@Product nvarchar(20),
@Planogramref nvarchar(20),
@DisplayGroup nvarchar(5),
@ModSequence int,
@Shelfnumber int,
@Productposition int,
@StartModnumber nvarchar(20)

AS

SET NOCOUNT ON
BEGIN
if object_id('tempdb..##S093_TempTable') is not null
drop table ##S093_TempTable


CREATE TABLE ##S093_TempTable(
[DetailRECTYPE] [nvarchar](5) NULL,
[StoreNumber] [int] NULL,
[Product] [nvarchar](20) NULL,
[Planogramref] [nvarchar](20) NULL,
[DisplayGroup] [nvarchar](5) NULL,
[ModSequence] [int] NULL,
[Shelfnumber] [int] NULL,
[Productposition] [int] NULL,
[StartModnumber] [nvarchar](20) NULL
)

INSERT INTO ##S093_TempTable
([DetailRECTYPE]
,[StoreNumber]
,[Product]
,[Planogramref]
,[DisplayGroup]
,[ModSequence]
,[Shelfnumber]
,[Productposition]
,[StartModnumber])
VALUES
( @DetailRECTYPE ,
@StoreNumber ,
@Product ,
@Planogramref ,
@DisplayGroup ,
@ModSequence,
@Shelfnumber ,
@Productposition ,
@StartModnumber
)






If i have a select query after the Data Flow task only the last row is returned.

Now how to make the ##Table keep all the rows.

Any help will appreciated.

Thanks in Advance...

Cheers

Srikanth Katte

View 8 Replies View Related

Retrieving Hierarchical Data From A Single Table

Sep 3, 2006

I would like to retrieve a hierarchical list of Product Categories from a single table where the primary key is a ProductCategoryId (int) and there is an index on a ParentProductCategoryId (int) field. In other words, I have a self-referencing table. Categories at the top level of the hierarchy have a ParentProductCategoryId of zero (0). I would like to display the list in a TreeView or similar hierarchical data display control.Is there a way to retrieve the rows in hierarchical order, sorted by CategoryName within level? I would like to do so from a stored procedure. Example data:ProductCategoryID CategoryDescription ParentProductcategoryID ParentCategoryDescription Level------------------------------------------------------------------------------------------------------------------------------------------------1                           Custom Furniture     0                                                                             02                           Boxes                     0                                                                             03                           Toys                       0                                                                             04                           Bedroom                 1                                    Custom Furniture                15                           Dining                     1                                    Custom Furniture                16                           Accessories            1                                    Custom Furniture                17                           Picture Frames        6                                    Accessories                       28                           Serving Trays           6                                    Accessories                       29                           Entertainment          1                                    Custom Furniture                110                         Planes                     3                                    Toys                                  111                         Trains                      3                                    Toys                                  112                         Boats                      3                                    Toys                                  113                         Automobiles             3                                    Toys                                  114                         Jewelry                    2                                    Boxes                                115                         Keepsake                2                                    Boxes                                116                         Specialty                 2                                    Boxes                                1Desired output:Custom Furniture     Accessories          Picture Frames          Serving Trays     Bedroom     Dining     EntertainmentBoxes     Jewelry     Keepsake     SpecialtyToys     Automobiles     Boats     Planes     Trains

View 4 Replies View Related

Designing (or Gathering Data From): A Hierarchical Database Using SQL

Feb 18, 2008

Hi All,

I am attempting to create a Visual C++ application based on displaying financial charts and am using SQL Express to store Stock information such as the Exchanges the stocks are traded on, the indicessectors they belong to and the Closing prices for as long as I can download data for. I am not proficient in C++ nor SQL and am using this project to learn both languages as well as making myself rich beyond my wildest dreams.

I have "designed" a database with the following tables:

tblDate_ 1 column clmDate (Primary Key, smalldatetime, NOT NULL)

tblStockExchange_ 4 column clmStockExchangeID (PK, int, NOT NULL)
clmParentID (int, null)
clmStockExchange (nvarchar(50), NOT NULL)
clmMarkets_ (FK, nchar(20), NOT NULL)

tblMarkets_ 1 column clmMarkets (PK, nchar(20), NOT NULL)

tblIndices_ 1 column clmIndices (PK, nchar(50), NOT NULL)

tblSectors_ 1 column clmSectors (PK, nchar(50), NOT NULL)

tblMarkets_Sectors 3 columns clmMarkets_SectorsID(PK, int, NOT NULL)
clmMarkets_ (FK, nchar(20), NOT NULL)
clmSectors_ (FK, nchar(50), NOT NULL)

tblSecurities_ 4 columns clmEPIC (PK, nchar(10), NOT NULL)
clmSecurity_Type (nchar(5), NOT NULL)
clmSecurty_Name (nchar(50), NOT NULL)
clmSectors_ (FK, nchar(50), NOT NULL)

tblSecurities_Indices 3 columns clmSecurities_IndicesID (PK, int, NOT NULL)
clmEPIC_ (FK, nchar(10), NOT NULL)
clmIndices_ (FK, nchar(50), NOT NULL)

tblSecurities_Date_OHLCV 8 columns clmOHLCVID (PK, int, NOT NULL)
clmEPIC_ (FK, nchar(10), NOT NULL)
clmDate_ (FK, smalldatetime, NOT NULL)
clmOpen (float, NOT NULL)
clmHigh (float, NOT NULL)
clmLow (float, NOT NULL)
clmClose (float, NOT NULL)
clmVolume (float, NOT NULL)

Why so many tables? perhaps you should put some more in...


This was the only way I could work out how to store one-to-one and one-to-many relationships required for:

- Many closing prices for many stocks
- Stocks belonging to many indices
- Stocks belonging to only one sector
- Stocks belonging to only one market (MainMarket or AIM for LSE)
- Stocks belonging to only one Exchange (I am aware of dual listed stocks but one thing at a time)

Why nchar's and not nvarchar's?

Because I didn't realise the benefits of nvarchar's until recently. How can I change this a loose the extra spaces in the cells.

Why do some tables have IDs and others don't?

I decided to put ID columns in for tables that didn't have obvious Primary Keys - if someone could explain the advantages if ID columns I would be grateful.

To the SQL Professional's eye there will be some obvious things wrong with this design and your criticism is welcome. The database I have is achieving what I would like it to do; I can plot charts using the data but I have ran into problems when trying to create a TreeView control which is what I would like to use as a navigational tool in my application.

It would seem that pulling hierarchal data from a relational database, to pass to the TreeView control, is a tricky task to say the least. I have found many articles online which discuss how to do this (using an Adjacency List Model or Nested Set Model) but they define a fairly simple example at the beginning (based on fruit or electrical goods) but don't appear to talk about gathering data from an existing relational database or changing an existing relational database so that it is more suited to storing hierarchal information. I have Joe Celko's - Tree and Hierachies in SQL for Smarties but sadly this fine material is a little beyond me!

I would like the hierarchy to look like this:

StockExchange

Market

Sector

Stock
Indices

Sector

Stock

I have written three queries to get the StockExchangeMarketSectorStock information individually from each table but am struggling with ways to put all the rows together, add left and right values (Nested Set Model) then run queries against this to get individual nodes to pass to the TreeView control. Therefore is there something I need to add to the original design?

Any help would be greatly appreciated.

View 4 Replies View Related

Slow Inserts Into Large Tables

Nov 29, 2000

We are inserting into a table, which includes an identity primary key column. When the table gets really large (i.e. 1.5 million records), the performance of the inserts reduce.

I noticed that when we insert into the table an exclusive lock on the table is obtained. Do inserts into tables with identities always lock the table?

Given the table size is unavoidable, does anyone have a suggestion to improve the performance?

Thanks,
Matt

View 6 Replies View Related

Transact SQL :: Flattening Hierarchical Data In Specific Format

Apr 22, 2015

I have the data in this format

ID        NAME         ParentID
CV1      CV1NAME      CV
CVX1    CVX1NAME    CV1
CVXX1  CVXX1NAME  CVX1
CV2      CV2NAME     CV
CVX2    CVX2NAME    CV2
CVXX2  CVXX2NAME  CVX2

How can i flatten this data into this format

CVID    CVNAME     CVXID   CVXNAME   CVXXID   CVXXNAME
cv1        cv1name  cvx1        cvx1name    cvxx1     cvxx1name
cv2        cv2name  cvx2        cvx2name    cvxx2     cvxx2name

View 4 Replies View Related

Transact SQL :: Select Lowest Level From Hierarchical Data

Jun 22, 2015

I have hierarchical data such as:

Id    Level    ParentId
1     0         1
2     1         1
3     2         2
4     0         4
5     1         4
6     0         6
7     1         6
8     2         7
9     3         8        
10   4         9
11   0         11

As you can see even the parent element has parentId(in this case id = parentid)

How can I select the lowest level data in the hierarchy and get this result:

Id    Level    ParentId
3     2         2
5     1         4
10   4         9
11   0         11

View 4 Replies View Related

Transact SQL :: Check No Inserts Occurring In Tables

Sep 14, 2015

I want to check that no inserts are occurring in 5 tables that are depending on each other and then drop and create those 5 tables. I have scripts to drop and recreate the tables. How do I check that no inserts are happening in these 5 tables?

Table A
Table B dependant on
Table A
Table C dependant on
Table B
Table D dependant on
Table C
Table E dependant on
Table D

View 9 Replies View Related

SQL 2012 :: Using Excel In SSIS To Import Data From Spreadsheet To Staging Table?

Feb 5, 2015

I'm trying to use Excel in SSIS to import the data from spreadsheet to a staging table. The package runs well from the web server using SSMS. But when I deploy and try to execute the package, I'm getting the below error. I've a question, whether I've to install the AccessDatabaseEngine driver in SQL database server or the web server where I'm executing the SSIS?

Error: The requested OLE DB provider Microsoft.Jet.OLEDB.4.0 is not registered. If the 64-bit driver is not installed, run the package in 32-bit mode.

View 3 Replies View Related

Hierarchical (tree) Data Structure Where A Node Can Have Multiple Parents

May 9, 2008

Hi all!
I am trying to organize a hierarchical data structure into a table. I need to have the possibility to set 2 parents for some nodes. Curently I see following two options:
Example 1
id    parent_id    name-----------------------------------1     0                 Level 1 Parent A2     0                 Level 1 Parent B3     1,2              Level 2 Child
Example 2
id    parent_id    name-----------------------------------1     0                 Level 1 Parent A2     0                 Level 1 Parent B3     1                 Level 2 Child3     2                 Level 2 Child
Is any of the two examples valid database logic wise? In fact, is it possible to achieve the requirement by using only one table?
Thanks in advance,

View 4 Replies View Related

SQL Server 2008 :: Query To Show XML Output For Hierarchical Data?

Mar 10, 2015

selecting table data in hierarchical XML .

Here is the sample table DDL and data

Declare @continents Table
(
id int identity (1,1)
,continent_id int
,continent_Name varchar(100)
,continent_surface_area varchar(100)
,country_id int

[code]....

View 8 Replies View Related







Copyrights 2005-15 www.BigResource.com, All rights reserved