DB Design :: Alter Partition Function In Data Vault Where There Is 4 Table Dependency?
Aug 28, 2015
I'm currently stuck with a table that has 350 mil records. Querying this table is insanely slow so I had a better look at existing yearly partitioning. I already managed to partition on a month level which increased the performance/querrying a lot. I did this on the staging table where I used an alter statement to split the 2015 partition by 12 months.
However, in our project we used Data Vault. This means that we have 4 tables (hub, sathub, link, satlink), all carrying 350 mil records. The problem is that altering the partition function does not work. The server cannot handle this action. What the best way is to do this, without having to drop/reload all tables.
A customer wants to implement table partitioning on a replicated table.
They want to hold 13 months of data in the table and roll off the earliest/oldest month to an identical archive table. The table has a date field and partitioning by month makes sense all around.
So SWITCH PARTITION is the obvious solution to this, except for the fact that the table is replicated (transactional w/no subscriber updates).
What are his architectural or practical solutions to using table partitioning and replication?
I have a heavy database , More than 100 GB only for six month .every Query on it takes me along time and I dont have enough space to add more indexes.by a way I decided to do partitioning. I create a partition function , on date filed and all Data records per month was appointed to a separate file.And is partitioning only for Future data entry?
I'm working on re-indexing a table using some commands from ALTER INDEX REBUILD from Microsoft. The indexes will be tested for threshold fragmentation. my plan is once the reindex is executed, a transaction backup will occur while controlling the size of the log file. Â The query impose time limitations or stop reindexing after the specified amount of time has elapsed.
my question, 1. How can I integrate a query which checks if transaction log is getting full and which runs a Tlog backup if over 70% 2. How do I impose time limitation?
Hi, I have a big table which is partitioned , I need to change the portioning function (including the column used by partition key). I don€™t want to change the table or use a temp table as there are a lot of dependent objects to this table
I wrote:
DECLARE @partition_count INT,@cmd VARCHAR(8000) SELECT @partition_count=MAX(p.partition_Number) FROM sys.allocation_units A,sys.partitions P ,sys.data_spaces D WHERE P.OBJECT_ID=OBJECT_ID('TABLEA') AND A.container_id=P.Partition_id AND A.data_space_id=D.data_space_id SELECT @partition_count
WHILE (@partition_count>1) BEGIN SELECT @cmd='ALTER PARTITION FUNCTION MainPartitionFunction() MERGE range ('+CONVERT(VARCHAR(10),@partition_count-1)+')' exec (@cmd) SELECT @partition_count=@partition_count-1 End
Which will merge all the existing partitions to one. But I don€™t seem to be able to find a way change the existing funtion. ( in need to change the partition key all together) Is there a way to disable or drop parition of the table all together (convert the table to a non-partitioned table) and then I can re define the parition function again. Any ideas? Shaunt
I am trying to find the list of active connections on SQL SERVER , more and more granular way ...like below , I know a way using active monitor or Profiler  or some DMVs give me some list , but is there a way to find what is the server name ,instance name connectedc to it  , who is the clinet connected to it , whats the application being used now?
what database  it is connected to it ? what login being used to connect ? what table and column being used right now and what kind of DML or DDL are happeming ?
I want to create a report and find Dependancay evaluation so that we will re architech or re design the existing databases ...
ServerName  InstanceName  Client  Application  Database  Login  Table  Column  Select  Insert  Update  Delete  Truncate  Alter
Is it possible to use a variable to specify the filegroup in the ALTER/CREATE PARTITION SCHEME command?
I want the partition scheme to use the default filegroup for ALTER and CREATE PARTITION SCHEME. At the time the script is created, I don't know the default filegroup in the database.
My code:
declare @fileGroupName VARCHAR(50) = (select top 1 name from sys.filegroups where is_default = 1) ALTER PARTITION SCHEME MyScheme NEXT USED @fileGroupName
Is failing:
Incorrect syntax near '@fileGroupName'.
Q: Is it possible to use a variable for the filegroup in the ALTER/CREATE commands? Is so, what is the correct syntax?
Q: If using a variable is not possible, is there another way to specify the default filegroup?
Hi Column with TEXT datatype is not stored in the same data row any way. I am wondering if there is any performance gain to put it in a seperate table. Thanks
When you load the data into a new partition table, can it to done online without any downtime? because I have few tables that are around 250 gigs and more.
I have names in the database which I want partition by last name - for example last names starting with A, B, C, D should go to the file group 1. last names starting with E, F, G, H should go to file group 2.
I am trying to use the following function - but do I specify in the function that last names with with A, B, C, D should go to the file group 1
CREATE PARTITION FUNCTION myRangePF3 (char(20)) AS RANGE RIGHT FOR VALUES ('EX', 'RXE', 'XR');
Is there any way to modify partition function to accomplish this?
It is first time that I'm having this problem; I had developed an application which has been deployed on client's computer. Now he requested for some changes in application for which i have to change the tables structure as well as lot of coding too. What i am worry about is that if i make changes to the structure of tables so how i will incorporate the old data (the client has started working upon the application)into this newly structure tables because this newly structured tables will have more or less columns than the previous one.
I ran the alter query and it worked fine for me. But the problem is that If i give this new DB to client, so how i will incorporate the old data into this newly altered DB?
Let me briefly explain my application. I developed a standalone PC application where both DB and application are installed on a single PC.
I can't seem to find a way to do the following:create table part_table (col1 int,col2 datetime) on psX (datename(week,col2))I want to partition based on the week number of a date field.So if I enter in data like the following in my part_table:(1, 1/1/2007) should go into partition 1 for week #1(52, 12/21/2007) should go into partition 52 for week #52 of the yearI tried adding in a computed column, but it says its nondeterministic.
I would like to set a field to not null which I can do like this: ALTER TABLE table1 ALTER COLUMN field1 BIGINT NOT NULL
My question is there anyway to set this field to NOT NULL without having to specify the data type BIGINT? I'm using dynamic sql and if there was a way then I wouldn't need to go get the data type of the field dynamically and it would save me a step. Thanks.
I am facing issue in generating total sum and daily sum from table ThresholdData.
DailyTransactionAmount should be sum of todays amount in the table TransactionAmount should be sum of all amount in the table.
Basically,
1. I don't want to scan ThresholdData table twice. 2. I don't want to create temporary table/table variable/CTE for this. 3. Is there is any way to make it done in single query.
I hope, where criteria is not possible in partition function. I am trying query something as given below,
SELECT  TransactionDate,   TransactionAmount,   ROW_NUMBER() over (order by TransactionDate) AS TransactionCount,   SUM(TransactionAmount) over (partition by id ) AS TransactionAmount,   SUM(TransactionAmount) over (partition by id ,CONVERT (DATE, @TodaysTransactionDate)) AS DailyTransactionAmount  FROM ThresholdData  WHERE id = @id  AND transactiondate >= dateadd(d,-@TransactionDaysLimit,@TodaysTransactionDate)
Hi people,I?m trying to alter a integer field to a decimal(12,4) field in MSACCESS 2K.Example:table : item_nota_fiscal_forn_setor_publicofield : qtd_mercadoria integer NOT NULLALTER TABLE item_nota_fiscal_forn_setor_publicoALTER COLUMN qtd_mercadoria decimal(12,4) NOT NULLBut, It doesn't work. A sintax error rises.I need to change that field in a Visual Basic aplication, dinamically.How can I do it? How can I create a decimal(12,4) field via script in MSACCESS?Thanks,Euler Almeida--Message posted via http://www.sqlmonster.com
Hi,I need to create a partition table but the column on which I need tocreate a partition may not have any logical ranges. So while creatingor defining partition function I can not use any range.likeCREATE PARTITION FUNCTION my_part_func (NUMERIC(7)) AS RANGE LEFT FORVALUES (1,100,1000);Is there any way to define partition function in SQL Server somethinglike Oracle HASH partitions where logical range is unkown?ThanksSameer
I would like to alter view to add columns. My base table is populated by application and the base table name is always change. Is there any way to add some extra columns to this view.
One way is to create a sp to add columns and use the view inside the sp. But I would like to know is there any way to alter the view.Â
I have a requirement of table partitioning. we have 10 years of data on a table which is 30 billion up rows on 2005 server we are upgrading it to 2014. we have to keep 7 years of data. there is no keys on table or date column. since its a huge amount of data and many users its slow down the process speed. we are thinking to do partition on 7 years for Quarterly based. but as i said there is no date column on table we have to use reference table to get date. is there a way i can do the partitioning with out adding date column on table? also does partition will make query faster?Â
I have think three ways to do it. 1. leave as it is. 2. 7 years partition on one server 3. 3 years partition on server1 and 4 years partition on server2 (for 4 years is snapshot better?)
DECLARE @DatePartitionFunction nvarchar(max) = N'CREATE PARTITION FUNCTION DatePartitionFunction (datetime) AS RANGE RIGHT FOR VALUES ('; DECLARE @i datetime = '2007-09-01 00:00:00.000'; WHILE @i < '2008-10-01 00:00:00.000' BEGIN SET @DatePartitionFunction += '''' + CAST(@i as nvarchar(10)) + '''' + N', ';
[Code] ....
Msg 7705, Level 16, State 2, Line 1 Could not implicitly convert range values type specified at ordinal 1 to partition function parameter type.
However if I change to datetime2 it works
DECLARE @DatePartitionFunction nvarchar(max) = N'CREATE PARTITION FUNCTION DatePartitionFunction (datetime2) AS RANGE RIGHT FOR VALUES ('; DECLARE @i datetime2 = '2007-09-01 00:00:00.000'; WHILE @i < '2008-10-01 00:00:00.000' BEGIN SET @DatePartitionFunction += '''' + CAST(@i as nvarchar(10)) + '''' + N', ';
[Code] ...
Is the data type of the column used for partitioning. All data types are valid for use as partitioning columns, except text, ntext, image, xml, timestamp, varchar(max), nvarchar(max), varbinary(max), alias data types, or CLR user-defined data types.
In this case why isn't datetime works?
version is as follow:
Microsoft SQL Server 2012 (SP1) - 11.0.3128.0 (X64) Dec 28 2012 20:23:12 Copyright (c) Microsoft Corporation Enterprise Evaluation Edition (64-bit) on Windows NT 6.1 <X64> (Build 7601: Service Pack 1)
from [URL] .....
Table and index partitioning is supported in this edition
I have to tables like given below Landing table "A" (Data load will happen over here, No primary keys mentioned over here) table "B"Â .Now I want to move the data from A to B.I have made use of below query insert into B select * from A...Landing table "A" has huge no of records, MS SQL server is taking huge amount of time.any alternative way to make this insertion process faster?
In sqlserver 2000 I have a UDF which works fine but I want to make achange to it. When I do an ALTER FUNCTION ... I get an error sayingthat I can't alter the function because it is referenced by an object.Is there any way around this? I reference the UDF in over 100 tables,do I have to go to each table, remove the all references alter thefunction then edit each 100 tables again? How clumsy can it be?Barry
Hi, I am new to SQL 2005. I have to design schema for scientific data warehouse. Data is available in 2 or more flat data files recorded at 1 sec interval. At Least 2 of the data files have 100+ columns. I am inclined to create a table per data file type. I want to know If this is correct/optimal for me to do?
I don't think I can create normalize tables based on the headers in these Data files.
Primary Objective of this data warehouse is make it available for reporting services and Analysis Services.
Hi, I am new to SQL 2005. I have to design schema for scientific data warehouse. Data is available in 2 or more flat data files recorded at 1 sec interval. At Least 2 of the data files have 100+ columns. I am inclined to create a table per data file type. I want to know If this is correct/optimal for me to do?
I don't think I can create normalize tables based on the headers in these Data files.
Primary Objective of this data warehouse is make it available for reporting services and Analysis Services.
I have data that is similar in nature to stock market data. There are about 100 entries per day. I would like to setup a second table to supply chart information. (assuming 1000 pixels per chart max)Â
What is the best way to condense the information in this table so that if I query for a chart, the table returns about 1000 points? I could have several tables setup, one for short term, and one or two for longer term. But I still don't know how to condense them.Â
I would like to add an Identity to an existing column in a table using astored procedure then add records to the table and then remove the identityafter the records have been added or something similar.here is a rough idea of what the stored procedure should do. (I do not knowthe syntax to accomplish this can anyone help or explain this?Thanks much,CBLCREATE proc dbo.pts_ImportJobsas/* add identity to [BarCode Part#] */alter table dbo.ItemTestalter column [BarCode Part#] [int] IDENTITY(1, 1) NOT NULL/* add records from text file here *//* remove identity from BarCode Part#] */alter table dbo.ItemTestalter column [BarCode Part#] [int] NOT NULLreturnGOSET QUOTED_IDENTIFIER OFFGOSET ANSI_NULLS ONGOhere is the original tableCREATE TABLE [ItemTest] ([BarCode Part#] [int] NOT NULL ,[File Number] [nvarchar] (20) COLLATE SQL_Latin1_General_CP1_CI_AS NULLCONSTRAINT [DF_ItemTest_File Number] DEFAULT (''),[Item Number] [nvarchar] (50) COLLATE SQL_Latin1_General_CP1_CI_AS NULLCONSTRAINT [DF_ItemTest_Item Number] DEFAULT (''),[Description] [nvarchar] (50) COLLATE SQL_Latin1_General_CP1_CI_AS NULLCONSTRAINT [DF_ItemTest_Description] DEFAULT (''),[Room Number] [nvarchar] (50) COLLATE SQL_Latin1_General_CP1_CI_AS NULLCONSTRAINT [DF_ItemTest_Room Number] DEFAULT (''),[Quantity] [int] NULL CONSTRAINT [DF_ItemTest_Quantity] DEFAULT (0),[Label Printed Cnt] [int] NULL CONSTRAINT [DF_ItemTest_Label Printed Cnt]DEFAULT (0),[Rework] [bit] NULL CONSTRAINT [DF_ItemTest_Rework] DEFAULT (0),[Rework Cnt] [int] NULL CONSTRAINT [DF_ItemTest_Rework Cnt] DEFAULT (0),[Assembly Scan Cnt] [int] NULL CONSTRAINT [DF_ItemTest_Assembly Scan Cnt]DEFAULT (0),[BarCode Crate#] [int] NULL CONSTRAINT [DF_ItemTest_BarCode Crate#] DEFAULT(0),[Assembly Group#] [nvarchar] (50) COLLATE SQL_Latin1_General_CP1_CI_AS NULLCONSTRAINT [DF_ItemTest_Assembly Group#] DEFAULT (''),[Assembly Name] [nvarchar] (50) COLLATE SQL_Latin1_General_CP1_CI_AS NULLCONSTRAINT [DF_ItemTest_Assembly Name] DEFAULT (''),[Import Date] [datetime] NULL CONSTRAINT [DF_ItemTest_Import Date] DEFAULT(getdate()),CONSTRAINT [IX_ItemTest] UNIQUE NONCLUSTERED([BarCode Part#]) ON [PRIMARY]) ON [PRIMARY]GO
I am using sql server ce.I am changing my tables sometimes.how to use 'alter table alter column...'.for example:I have table 'customers', I delete column 'name' and add column 'age'.Now I drop Table 'customers' and create again.but I read something about 'alter table alter column...'.I use thi command but not work.I thing syntax not true,that I use..plaese help me?
I have a very large table that I am trying to partition and use to reduce maintenance overhead as well as improve performance. The table contains about 12 years worth of data but only the most recent years is inserted/updated/deleted from thru the app. I created partitions on a computed(persisted) column which holds the "year" value derived from a date column. I have created the partitions with all the default set options, and the stored procedure which performs the delete against this table also was created with no special set options(basically database/session default). Yet, every time I try to run the proc to delete data thru the app, I get this error:
Msg 1934, Level 16, State 1, Procedure xxxx, Line 118 DELETE failed because the following SET options have incorrect settings: 'ANSI_WARNINGS'. Verify that SET options are correct for use with indexed views and/or indexes on computed columns and/or filtered indexes and/or query notifications and/or XML data type methods and/or spatial index operations.
I've tried setting ANSI_WARNINGS on and off when creating the proc, inside the proc etc.., its always the same error whatever I set the option to.
I have 3 tables (accnt, jobcost, and servic15). all with the same fields (code, jno, ven, date). I need to insert the data from these tables into another table called dummy with the same fields, in one statement or query.
Hi,I want to add Sql data dependency in GridView. Whenever any changes in database, same will be reflected in Grid. but this should work in both Sql server 2000 and sql server 2005. And when we click on "Pause" button it should stop refreshing.
PAUSE
Index
First name
Last Name
Address
1
Xyz
Abc
Asdf, asdf
2
Pqr
Asd
Sdsa, asad
Is there any way to fulfill this kind of requirement? Please suggest me optimal solution to do this.Regards,Mustakim Mansuri
