Storing Staging Data As XML In SQL 2000.
Aug 25, 2005
Scenario:
We want to stage all data coming into our database from various
applications. We have a generic control where all data goes
through to get to the database.
We want have the data staged in a 'staging database'. My question
is, what would be the best way to store the staging database in the
database.
I'm thinking storing it as XML in a SQL column. The reason for
this is because the data could change from application to
application. It would be impossible to create a relational table
because maintenance would be a nightmare.
Has anyone ever done something like this. Would you save the node names in another table in SQL?
Any help would be appreciated as I'm new to XML.
Thanks,
ScAndal
View 1 Replies
ADVERTISEMENT
Feb 20, 2007
HI
i want to save data larger than 8000 charactors in a colomn
is there any way to do it in mssql server 2000 desktop addition
plz help me
Thanks in advance
View 14 Replies
View Related
Dec 29, 2006
Hello there,I just want to ask if storing data in dbase is much better than storing it in the file system? Because for one, i am currenlty developing my thesis which uploads a blob.doc file to a web server (currently i'm using the localhost of ASP.NET) then retrieves it from the local hostAlso i want to know if im right at this, the localhost of ASP.NET is the same as the one of a natural web server on the net? Because i'm just thinking of uploading and downloading the files from a web server. Although our thesis defense didn't require us to really upload it on the net, we were advised to use a localhost on our PC's. I'll be just using my local server Is it ok to just use a web server for storing files than a database?
View 6 Replies
View Related
Jan 3, 2007
I'm populating a table (B) in SQL Server from a Staging table (A) using a stored procedure.At any point of time, the Staging table holds 60 months' old data.In the first load of the destination table B, I get 13 months of old data whereas for every subsequent load,I need to load the data for the most recent month and delete data for the 1st(oldest) month. For example, if the load procedure runs on December 02,2006, it should pick data for the month of November,2006 from the Staging table and delete data for the 1st month.
I have a column DATA_MONTH_KEY in table B which maps to the column DATA_MONTH in my staging table A. I get the data for the first 13 months using:
(B.DATA_MONTH_KEY BETWEEN ( DATEADD(month,-13,@startdate)) AND @startdate) where startdate is the current date on which the procedure for populating table B is run. I get the value of startdate from a function.
How do i get data for the most recent months and delete the oldest month in subsequent loads?
Any help appreciated. Thanks.
View 7 Replies
View Related
Jun 11, 2008
Hi All,
I am extracting source data which is in txt fille to OLE DB destination. But data of each day I want to save in different staging table. For Eg; tblProduct20081206, tblProduct20081207. How can it be done. I have seen lots of posting and script when destination is Txt. I want to use same table for staging but want to create different table for each day with adding date extension.
Please Help
View 2 Replies
View Related
Apr 12, 2015
I am trying to insert data from staging table to production table. In the staging table I only have period or date but no primary key.
This is my staging table
Create stagingtable(
[Period] [char](7) NOT NULL,
[CompanyCode] [varchar](100) NOT NULL,
[total] [int] NULL,
[status] [varchar](50) NULL
)
Create Production(
[Period] [char](7) NOT NULL,
[CompanyCode] [varchar](100) NOT NULL,
[total] [int] NULL,
[status] [varchar](50) NULL
)
I get this every month. What can I do to make sure only unique record are loaded into production table with no duplicate from previous month.
View 5 Replies
View Related
Jan 6, 2006
I have a question about staging design using SSIS. Has anyone come up with an ETL design that would read table names from a generic table and dynamically create the ETL to stage the table.
1. Have a generic table which would have table name and description and whatever else that was required.
2. Have a master ETL that would enumerate through the table and stage all the table names found in the generic table.
This way I wouldn't have to create an ETL which would hardcode the names of 300-500 tables and have the appropriate 300-500 data sources and targets listed.
Not sure if I am making sense but I hope someone understands the attempt.
thanks
View 10 Replies
View Related
Apr 24, 2008
Hi All,
In one of my interfaces,i am trying to Stage data in a ##table from a Flat file,so that i can iterate through the data in for each loop afterwards.
I have written a stored Proc to do this.But what i am observing is in the ##Table only the last Row will be persisted.
The SP looks like this
Code Snippet
CREATE PROCEDURE USP_SVInsS093Data_V001
@DetailRECTYPE nvarchar(5),
@StoreNumber int,
@Product nvarchar(20),
@Planogramref nvarchar(20),
@DisplayGroup nvarchar(5),
@ModSequence int,
@Shelfnumber int,
@Productposition int,
@StartModnumber nvarchar(20)
AS
SET NOCOUNT ON
BEGIN
if object_id('tempdb..##S093_TempTable') is not null
drop table ##S093_TempTable
CREATE TABLE ##S093_TempTable(
[DetailRECTYPE] [nvarchar](5) NULL,
[StoreNumber] [int] NULL,
[Product] [nvarchar](20) NULL,
[Planogramref] [nvarchar](20) NULL,
[DisplayGroup] [nvarchar](5) NULL,
[ModSequence] [int] NULL,
[Shelfnumber] [int] NULL,
[Productposition] [int] NULL,
[StartModnumber] [nvarchar](20) NULL
)
INSERT INTO ##S093_TempTable
([DetailRECTYPE]
,[StoreNumber]
,[Product]
,[Planogramref]
,[DisplayGroup]
,[ModSequence]
,[Shelfnumber]
,[Productposition]
,[StartModnumber])
VALUES
( @DetailRECTYPE ,
@StoreNumber ,
@Product ,
@Planogramref ,
@DisplayGroup ,
@ModSequence,
@Shelfnumber ,
@Productposition ,
@StartModnumber
)
If i have a select query after the Data Flow task only the last row is returned.
Now how to make the ##Table keep all the rows.
Any help will appreciated.
Thanks in Advance...
Cheers
Srikanth Katte
View 8 Replies
View Related
Feb 5, 2015
I'm trying to use Excel in SSIS to import the data from spreadsheet to a staging table. The package runs well from the web server using SSMS. But when I deploy and try to execute the package, I'm getting the below error. I've a question, whether I've to install the AccessDatabaseEngine driver in SQL database server or the web server where I'm executing the SSIS?
Error: The requested OLE DB provider Microsoft.Jet.OLEDB.4.0 is not registered. If the 64-bit driver is not installed, run the package in 32-bit mode.
View 3 Replies
View Related
Aug 28, 2007
I apologize if this has been asked, but I can't find a complete answer.
We have a situation with parent/child tables which have an identity column as their PK. We need to be able to insert into the live tables from staging tables. The data in the staging tables are related via a surrogate key.
I have found the OUTPUT clause, but that can only refer to columns of the actual table (since there is no FROM clause in an INSERT). Our current best solution to this problem involves adding bogus "staging" columns to the destination tables, and removing them after we've inserted everything from staging. This is an unattractive solution to say the least.
I'll give an example that mirrors our actual solution, and ask if anyone has a better solution?
----------
Code Snippet
CREATE TABLE [dbo].[TABLE_A](
[ID] [int] IDENTITY(1,1) NOT NULL,
[DATA] [nchar](10) NOT NULL,
[STAGING_COLUMN] [bigint] NULL,
CONSTRAINT [PK_TABLE_A] PRIMARY KEY ([ID] ASC)
)
GO
CREATE TABLE [dbo].[TABLE_B](
[ID] [int] IDENTITY(1,1) NOT NULL,
[A_ID] [int] NOT NULL,
[DATA] [nchar](10) NOT NULL,
[STAGING_COLUMN] [bigint] NULL,
CONSTRAINT [PK_TABLE_B] PRIMARY KEY ([ID] ASC)
)
GO
ALTER TABLE [dbo].[TABLE_B]
ADD CONSTRAINT [FK_TABLE_A_TABLE_B] FOREIGN KEY([A_ID]) REFERENCES [dbo].[TABLE_A] ([ID])
GO
CREATE TABLE [dbo].[STAGE_TABLE_A](
[A_Key] [bigint] NOT NULL,
[DATA] [nchar](10) NOT NULL
)
GO
CREATE TABLE [dbo].[STAGE_TABLE_B](
[B_Key] [bigint] NOT NULL,
[DATA] [nchar](10) NOT NULL,
[A_Key] [bigint] NOT NULL
)
GO
The STAGING_COLUMN columns are the ones that will be added before, and dropped after.
Code Snippet
DECLARE @TABLE_A_MAP TABLE (
A_ID INT,
A_Key BIGINT
)
INSERT INTO TABLE_A (DATA, STAGING_COLUMN)
OUTPUT INSERTED.ID, INSERTED.STAGING_COLUMN INTO @TABLE_A_MAP
SELECT DATA, A_Key FROM STAGE_TABLE_A
INSERT INTO TABLE_B (A_ID, DATA)
SELECT TAM.A_ID, STB.DATA
FROM STAGE_TABLE_B STB INNER JOIN @TABLE_A_MAP TAM ON TAM.A_Key = STB.A_Key
This seems to work, but I'd really like another alternative. Even though this is happening when nobody else is using the database, I cringe at the thought of adding and removing columns just to make this work.
Here are a few of my constraints:
The above is a simplification of the actual problem. The actual problem goes about five levels deep (hence the B_Key in STAGE_TABLE_B). At the top level, our larger customer will have 100,000 rows to insert. Each level will average 3 times as many rows as the next higher level, so we're talking about real volumes here.
This has to finish over the course of a weekend.
This has to be delivered to QA this Friday
Thanks for any help or insight.
View 3 Replies
View Related
Jul 31, 2013
I am working on an HR project and I have one final component that I am stuck on.
I have an Excel File that is loaded into a folder every month.
I have built a package that captures the data from the excel file and loads it into a staging table (transforming a few bits of data).
I then combine it with another table in a view.
I have another package that loads that view into a Master table and I have added a Slowly Changing Dimension so that it only updates what has been changed. (it’s a table of all employees, positions, hire dates, term dates etc).
Our HR wants to have this data in a report (with charts and tables) and they wanted it to be in a familiar format. So I made a data connection with Excel loading the data into a series of pivot tables.
I have one final component that i cant seem to figure out. At the end of every year I need to capture a count of all Active Employees and all Termed employees for that year. Just a count.
So the data will look like this.
|Year|HistoricalHC|NumbTermedEmp|
|2010|447 |57 |
|2011|419 |67 |
|2012|420 |51 |
The data is in one table labeled [EEMaster]. To test the count I have the following.
SELECT COUNT([PersNo]) AS HistoricalHC
FROM [dbo].[EEMaster]
WHERE [ChangeStatus] = 'Current' AND [EmpStatusName] = 'Active'
this returns the HistoricalHC for 2013 as 418.
SELECT COUNT([PersNo]) AS NumbOfTermEE
FROM [dbo].[EEMaster]
WHERE [ChangeStatus] = 'Current' AND [EmpStatusName] = 'Withdrawn' AND [TermYear] = '2013'
This returns the Number of Termed employees for 2013 as 42.
I have created a table to report from called [dbo.TORateFY] that I have manually entered previous years data into.
|Year|HistoricalHC|NumbTermedEmp|
|2010|447 |57 |
|2011|419 |67 |
|2012|420 |51 |
I need a script (or possibly a couple of scripts) that will add the numbers every year with the year that the data came from.
(so on Dec 31st this package will run and add |2013|418|42| to the next row, and so on.
View 20 Replies
View Related
Aug 14, 2015
I need to copy data from warehouse tables to master tables of different SQL instances. Refresh need to done once in an hour. What is the best way to do this? SQL agent jobs or SSIS packages?
View 3 Replies
View Related
Apr 9, 2015
I am having one store procedure which use to load data from flat file to staging table dynamically.
Everything is working fine.staging_temp table have single column. All the data stored in that single column. below is the sample row.
AB¯ALBERTA ¯93¯AI
AI¯ALBERTA INDIRECT ¯94¯AI
AL¯ALABAMA ¯30¯
After the staging_temp data gets inserted into main table.my probelm is to handle such a file where number of columns are more than the actual table.
If you see the sample rows there are 4 column separated by "¯".but actual I am having only 3 columns in my main table.so how can I get only first 3 column from the satging_temp table.
Output should be like below.
AB¯ALBERTA ¯93
AI¯ALBERTA INDIRECT ¯94
AL¯ALABAMA ¯30
How to achieve above scenario...
View 1 Replies
View Related
Sep 21, 2005
Hi,
I am aggregating some in-coming XML SOAP envelopes and I need to persist them to the database.
Is there a recommended data type to handle this? Should I be using ntext, text or other data type?
I know in Sql Server 2005 there will be a specific Xml Data Type; but, that is not an option until Nov 7, 2005.
View 3 Replies
View Related
Sep 16, 2002
Hello,
I am using Jrun 3.1 + MS SQL Server 2000 + JSPs + Java on W 2000. I am having a lot of trouble storing Unicode characters in MS SQL Server since my 1.2 JDBC driver seems to assume that my characters belong to the Windows basic-Latin character set. Does anyone know how to solve this problem?
Many thanks.
Philippe
View 2 Replies
View Related
Apr 15, 2007
HI,
Kindly Guide me how to store picture files in SQL server 2000 using Visual studio 2005
View 3 Replies
View Related
Dec 20, 2004
hi,
i would like to store just the time part in sqlserver 2000. time part with default date which is 1899 - 12- 30 in my sql server. but i read some articles which mentioned default date as 1900 - 01 - 01. i am not sure which one but if i enter just the time part in my sql server and do the query i get 1899 -12-30 and (time part). anyway, i am just wondering whether it is possible to set the trigger or constraints to strip off the date part of entered value and replacing it with default is ok.
the reason is i need to some queries on time part only which is giving some problem now.
View 2 Replies
View Related
Jul 23, 2005
Hi folks,Is it possible to store Binary Files in MS SQL 2000 ??? Say I have a100K PDF or a 150K word document. Is it possible to store this in afield in MS SQL and pull it out somehow? We're using ColdFusion on theserver on Apache.Thanks,Ringo
View 3 Replies
View Related
Jul 20, 2005
HI there,I currently store the date using the getdate() functionbut how can I store just the time or seperate the time off from adatetime datatype?M3ckon*** Sent via Devdex http://www.devdex.com ***Don't just participate in USENET...get rewarded for it!
View 4 Replies
View Related
Jan 14, 2014
What is the best way to transfer data from the staging table into the main table.
Example:
Staging Table Name: TableA_satge (# of rows - millions)
Main Table Name: TableA_main (# of rows - billions)
Note: Staging table may have some data same as the main table.
Currently I am doing:
- Load data into staging table (TableA_stage)
- Remove any duplication of rows from the staging table (TableA_stage)
- Disable all indexes on main table (TableA_main)
- Insert into main table (TableA_main) from staging table (TableA_stage)
- Remove any duplication of rows from the main table using CTE (TableA_main)
- Rebuild indexes on main_table (TableA_main)
The problem with the above method is that, it takes a lot of time and log file size grows very big.
View 9 Replies
View Related
Feb 23, 2015
I am trying to insert bulk data into main table from staging table in sql server 2012. If any error comes, this total activity is rollbacked. I don't want that to happen. I want to know the records where ever the problem persists, and the rest has to be inserted.
View 2 Replies
View Related
Sep 17, 2007
Hello,
Please help me find solution to these questions?
1. How do I store audio and video files in sql server 2000.
2. Is it possible to store and retrieve audio and video files using t-sql
3. Which is the most efficient way to store and retrieve audio and video files in sql server
Your suggestions are most valued.
Thanks!
View 6 Replies
View Related
Jun 20, 2006
Hi again.
How do I store in SQL SERVER with 2 decimal places?
I set Data Type to Decimal with Scale set to 4 and Precision 19. But this doesn't work.
Any ideas? Thanks.
View 1 Replies
View Related
Jul 16, 2006
CREATE TABLE [dbo].[blogs] (
[blog_title] [nvarchar] (500) ,
[blog_desc_full] [varchar] (8000) ,
[blogger_name] [nvarchar] (100) ,
[mailid] [nvarchar] (100) ,
[blogid] [numeric](10, 0) IDENTITY (1, 1) NOT NULL ,
[blog_desc] [nvarchar] (125) ,
[cat_name] [varchar] (100) ,
[b_url] [varchar] (250) ,
[b_date] [datetime] NULL ,
[author] [nvarchar] (250) ,
[approval] [char] (1)
) ON [PRIMARY]
GO
using this script i have created my blog table.
and a procedure given below. i am using to insert data in it.
CREATE PROCEDURE SP_BlogAdd
@blog_title nvarchar(500),
@blog_desc_full varchar(8000),
@blogger_name nvarchar(100),
@mailid nvarchar(100),
@blog_desc nvarchar(125),
@cat_name varchar(100),
@b_url varchar(250),
@author nvarchar(250)
AS
INSERT INTO blogs(blog_title, blog_desc_full, blogger_name, mailid, blog_desc, cat_name, b_url,author)
VALUES(@blog_title ,@blog_desc_full ,@blogger_name,@mailid ,@blog_desc ,@cat_name ,@b_url,@author)
GO
now, the problem i m facing is.
i am using varchar datatype for [blog_desc_full] [varchar] (8000).
i want more than this size to store data in it.
please give me some detailed code.
i tried text datatype but i didnt succeed. how to use text datatype.
i replaced with text datatype.but in length i couldnt type. it shows only 16.
Please help me out.
regards,
ASIF
View 1 Replies
View Related
Jul 16, 2006
CREATE TABLE [dbo].[blogs] (
[blog_title] [nvarchar] (500) ,
[blog_desc_full] [varchar] (8000) ,
[blogger_name] [nvarchar] (100) ,
[mailid] [nvarchar] (100) ,
[blogid] [numeric](10, 0) IDENTITY (1, 1) NOT NULL ,
[blog_desc] [nvarchar] (125) ,
[cat_name] [varchar] (100) ,
[b_url] [varchar] (250) ,
[b_date] [datetime] NULL ,
[author] [nvarchar] (250) ,
[approval] [char] (1)
) ON [PRIMARY]
GO
using this script i have created my blog table.
and a procedure given below. i am using to insert data in it.
CREATE PROCEDURE SP_BlogAdd
@blog_title nvarchar(500),
@blog_desc_full varchar(8000),
@blogger_name nvarchar(100),
@mailid nvarchar(100),
@blog_desc nvarchar(125),
@cat_name varchar(100),
@b_url varchar(250),
@author nvarchar(250)
AS
INSERT INTO blogs(blog_title, blog_desc_full, blogger_name, mailid, blog_desc, cat_name, b_url,author)
VALUES(@blog_title ,@blog_desc_full ,@blogger_name,@mailid ,@blog_desc ,@cat_name ,@b_url,@author)
GO
now, the problem i m facing is.
i am using varchar datatype for [blog_desc_full] [varchar] (8000).
i want more than this size to store data in it.
please give me some detailed code.
i tried text datatype but i didnt succeed. how to use text datatype.
i replaced with text datatype.but in length i couldnt type. it shows only 16.
Please help me out.
regards,
ASIF
View 1 Replies
View Related
Jan 22, 2007
I have a table for articles that I want as
the basis for a blog. I have a field of description where the actual
article will go, I have only ever really
used tables to put in 'blocks' of text, how would I go about storing/displaying
data that is in my database table in a more formatted way, for example line
breaks, indents etc?
View 4 Replies
View Related
May 11, 2007
Hi everyone,
I have some data which is in korean Language and i want to store that data in sql server 2000 table's row.But i am not able to store it.
When i try to store it then it display me square box in table.
Does anybody have any idea about this matter?
Plz reply me back as soon as possible.
Thanks
-------Nimesh Patel
View 6 Replies
View Related
Dec 19, 2007
Hi, I'm in the process of creating a database table and was wondering if it's better to store calculated values or recalculate them each time. So for example, I am creating a table that stores articles and then a table to store the pages to the article. If a new page is added should I update the pages field in the articles table or should I calculate the number of pages for an article when it's queried? Thanks,John
View 3 Replies
View Related
Apr 7, 2008
From what I've read this is called 'slowly changing dimensions'. Bassically the system I'm working on needs to store the history of certain data so that at any time a user can look up an old project and view it exactly as is, even though the associated parts might have had certain changes over time. From what I can tell Type 2 ( current and historical records are stored in the same table) seems to be the most popular. Type 4 (current records in one table and historical records in a seperate history table) seems like it would also work but I've been unable to find any articles comparing the two. Does anybody have any info on the dis/advantages of one v.s. the other?
View 3 Replies
View Related
Mar 5, 2008
Can I know answer of this question ? in detail explanation.
View 1 Replies
View Related
Dec 15, 2005
I am trying to make my tables so that when data is entered using a formview even if a textbox is left blank there is still a value like for example NA entered into that field. Is that possible?
also in the column properties under general what is the Default value or binding?
View 3 Replies
View Related
Nov 15, 2006
i've got an excel spreadsheet generated everyday which stores the duration of talk time as 0:02:09
When i use the Integration Services Package to move the data into a table. I've used datetime column. It comes up as 1899-12-30 00:00:02:09.000
What's best practice to store duration as in a SQL Server table? or should i convert it to seconds.
View 2 Replies
View Related
Jul 5, 2007
Hi
i m new in sqlserver databases
i need to know how to "store output of data from stored procedure in a text file "
suppose i have a stored procedure which has to cuculate some out put from some tables and in the end i want that all out put in comma delimited text file.
my databse name is check1
i need help please
thanks in advance
take care
bye
View 1 Replies
View Related
