Optimize Dedup TSQL?
May 30, 2004
The following is fairly basic cursor based merge code. Order by UserName and DomainID; INSERT a single merged record for every distinct combination.
I had originally written this as a single INSERT statement with subqueries. Avoiding cursors is supposed to be faster but comparing the execution plans for both strategies suggested the cursor based code was MUCH faster, so this is what I'm going to try and use.
If anyone can think of any optimizations to this I would be very grateful. Thanks!
DECLARE @emailUser VARCHAR(50)
DECLARE @domainID INT
DECLARE @firstName VARCHAR(24)
DECLARE @lastName VARCHAR(24)
DECLARE @streetAddress VARCHAR(32)
DECLARE @city VARCHAR(24)
DECLARE @state VARCHAR(24)
DECLARE @postal VARCHAR(10)
DECLARE @sourceID INT
DECLARE @numNameNULLs INT
DECLARE @numAddressNULLs INT
DECLARE @rowEmailUser VARCHAR(50)
DECLARE @rowDomainID INT
DECLARE @rowFirstName VARCHAR(24)
DECLARE @rowLastName VARCHAR(24)
DECLARE @rowStreetAddress VARCHAR(32)
DECLARE @rowCity VARCHAR(24)
DECLARE @rowState VARCHAR(24)
DECLARE @rowPostal VARCHAR(10)
DECLARE @rowSourceID INT
DECLARE @rowNumNameNULLs INT
DECLARE @rowNumAddressNULLs INT
DECLARE stagingCursor CURSOR FOR
SELECT
UserName, DomainID, First, Last, StreetAddress, City, State, Postal, SourceID
, (CASE WHEN Stages.[First] IS NULL THEN 1 ELSE 0 END
+ CASE WHEN Stages.[Last] IS NULL THEN 1 ELSE 0 END) AS NumNameNULLs
,(CASE WHEN Stages.[StreetAddress] IS NULL THEN 1 ELSE 0 END
+ CASE WHEN Stages.[City] IS NULL THEN 1 ELSE 0 END
+ CASE WHEN Stages.[State] IS NULL THEN 1 ELSE 0 END
+ CASE WHEN Stages.[Postal] IS NULL THEN 1 ELSE 0 END) AS NumAddressNULLs
FROM Stages
WHERE NOT EXISTS
(SELECT UserName FROM Recipients
WHERE Stages.UserName = Recipients.UserName
AND Stages.DomainID = Recipients.DomainID)
ORDER BY UserName, DomainID
OPEN stagingCursor
FETCH NEXT FROM stagingCursor INTO
@rowEmailUser, @rowDomainID, @rowFirstName, @rowLastName, @rowStreetAddress, @rowCity, @rowState, @rowPostal, @rowSourceID, @rowNumNameNULLs, @rowNumAddressNULLs
WHILE @@FETCH_STATUS = 0
BEGIN
IF (@emailUser = @rowEmailUser AND @domainID = @rowDomainID) BEGIN
-- Merge a consecutive row for the current UserName/DomainID combination
IF (@rowNumNameNULLs < @numNameNULLs) BEGIN
SET @numNameNULLs = @rowNumNameNULLs
SET @firstName = @rowFirstName
SET @lastName = @rowLastName
END
IF (@rowNumAddressNULLs < @numAddressNULLs) BEGIN
SET @streetAddress = @rowStreetAddress
SET @city = @rowCity
SET @state = @rowState
SET @postal = @rowPostal
SET @numAddressNULLs = @rowNumAddressNULLs
END
END ELSE BEGIN
IF (@emailUser IS NOT NULL) BEGIN
-- Finished iterating 1+ records of a UserName/DomainID combination. INSERT the merged record.
INSERT INTO Recipients(UserName, DomainID, First, Last, StreetAddress, City, State, Postal, SourceID)
VALUES (@emailUser, @domainID, @firstName, @lastName, @streetAddress, @city, @state, @postal, @sourceID)
END
-- Reached a new UserName/DomainID combination. Set current data.
SET @emailUser = @rowEmailUser
SET @domainID = @rowDomainID
SET @firstName = @rowFirstName
SET @lastName = @rowLastName
SET @streetAddress = @rowStreetAddress
SET @city = @rowCity
SET @state = @rowState
SET @postal = @rowPostal
SET @sourceID = @rowSourceID
SET @numNameNULLs = @rowNumNameNULLs
SET @numAddressNULLs = @rowNumAddressNULLs
END
FETCH NEXT FROM stagingCursor INTO
@rowEmailUser, @rowDomainID, @rowFirstName, @rowLastName, @rowStreetAddress, @rowCity, @rowState, @rowPostal, @rowSourceID, @rowNumNameNULLs, @rowNumAddressNULLs
END
CLOSE stagingCursor
DEALLOCATE stagingCursor
IF (@emailUser IS NOT NULL) BEGIN
-- Finished iterating 1+ records of a UserName/DomainID combination. INSERT the merged record.
INSERT INTO Recipients(UserName, DomainID, First, Last, StreetAddress, City, State, Postal, SourceID)
VALUES (@emailUser, @domainID, @firstName, @lastName, @streetAddress, @city, @state, @postal, @sourceID)
END
A coworker has a C++ solution that uses FAST INSERT operations (directly through the ADODB API; like bcp or BULK INSERT). He claims to get twice the performance over the above T-SQL although I can't use his application due to bizarre errors. The above T-SQL is obviously using plain old INSERT statements although I hope the looping and everything would be faster since its 100% native database code. Is there anyway I can get FAST/BULK INSERT like performance through the above T-SQL?
View 9 Replies
ADVERTISEMENT
Feb 12, 2002
Hi
I have a table PRODATE with 3 fields:
ID EMAIL DATE
1 aaa@a.com 08/15/2000
2 bbb@b.com 11/20/2000
3 ccc@c.com 02/04/1999
4 bbb@b.com 05/04/1998
5 aaa@a.com 11/26/2001
6 aaa@a.com 06/08/1999
What I need returned is 1 record per EMAIL only(dedup), based on most recent(MAX) date:
ID EMAIL DATE
5 aaa@a.com 11/26/2001
2 bbb@b.com 11/20/2000
3 ccc@c.com 02/04/1999
I've tryied several max and inner join statements with no luck.
Thanks
Nathan
View 1 Replies
View Related
Jan 29, 2002
Which method in MSSQL 7.0 would best suit being able to de-dupe, basically leave the dupes in the import file.
I have two process, in the first process, I'm importing from two different tables, so that any potential dupes would have their unique RecID's given from either table. I can then de-dupe on the unique ucase(entry)+RecID combo.
This works fine, however in the second process, the import file that has only one source, and therefore I could have real dupes. Currently I've only used a TSQL cursor process to copy all the data into a temp table, delete the data in the live table, then use another cursor in the same process to only copy one instance of an (account_number + RecID) back into the live table.
This too works, but I'd like to make a DTS package that can do this on import in as few steps as possible. I'm thinking to use one connection and a proc(?)
TIA
JeffP....
View 1 Replies
View Related
Nov 19, 2007
Can anyone please give me the equivalent tsql for sql server 2000 for the following two queries which works fine in sql server 2005
1
-- Full Table Structure
select t.object_id, t.name as 'tablename', c.name as 'columnname', y.name as 'typename', case y.namewhen 'varchar' then convert(varchar, c.max_length)when 'decimal' then convert(varchar, c.precision) + ', ' + convert(varchar, c.scale)else ''end attrib,y.*from sys.tables t, sys.columns c, sys.types ywhere t.object_id = c.object_idand t.name not in ('sysdiagrams')and c.system_type_id = y.system_type_idand c.system_type_id = y.user_type_idorder by t.name, c.column_id
2
-- PK and Index
select t.name as 'tablename', i.name as 'indexname', c.name as 'columnname' , i.is_unique, i.is_primary_key, ic.is_descending_keyfrom sys.indexes i, sys.tables t, sys.index_columns ic, sys.columns cwhere t.object_id = i.object_idand t.object_id = ic.object_idand t.object_id = c.object_idand i.index_id = ic.index_idand c.column_id = ic.column_idand t.name not in ('sysdiagrams')order by t.name, i.index_id, ic.index_column_id
This sql is extracting some sort of the information about the structure of the sql server database[2005]
I need a sql whihc will return the same result for sql server 2000
View 1 Replies
View Related
Jan 27, 2008
Hi.I have a sql: 1 SELECT ForumID, Title, Description,2 (SELECT PostID3 FROM (SELECT TOP (1) PostID4 FROM Forum_Posts5 WHERE (ForumID = Forum_Forums.ForumID)6 ORDER BY PostedDate DESC) AS derivedtbl_1) AS LastPostID,7
8
9 (SELECT Title10 FROM (SELECT TOP (1) Title11 FROM Forum_Posts12 WHERE (ForumID = Forum_Forums.ForumID)13 ORDER BY PostedDate DESC) AS derivedtbl_2) AS LastPostTitle,14
15 (SELECT PostedBy16 FROM (SELECT TOP (1) PostedBy17 FROM Forum_Posts18 WHERE (ForumID = Forum_Forums.ForumID)19 ORDER BY PostedDate DESC) AS derivedtbl_3) AS PostedBy20 FROM Forum_Forums21 WHERE (BoardID = @BoardID)I must use this code 3 times: SELECT TOP (1) PostID / Title / PostedBy FROM Forum_Posts WHERE (ForumID = Forum_Forums.ForumID)ORDER BY PostedDate DESCPlease help me optimize this.
View 4 Replies
View Related
Jul 3, 2006
A table fin_cutoff contains following type of data.
fin_beg_dt | fin_end_dt
------------------------------------------------
2005-12-01 00:00:00.000 | 2005-12-31 00:00:00.000
2005-10-29 00:00:00.000 | 2005-12-01 00:00:00.000
2005-10-01 00:00:00.000 | 2005-10-29 00:00:00.000
I want to extract a row where the fin_beg_dt & fin_end_dt columns entries belong to current month
Following query is written for the same:
select fin_beg_dt,
fin_end_dt
from ecr_fin_cutoff
where month(fin_beg_dt_id) = month(getdate())
andyear(fin_dt_id)= year(getdate())
and month(fin_end_dt) = month(getdate())
and year(fin_end_dt)= year(getdate())
It is giving the appropriate results.
Please help me to optimize it.
Thanks in advance.
Shrirang
View 1 Replies
View Related
Nov 8, 2006
I have a sql such sa below:
select
coalesce(T1.LocalNetID, T2.LocalNetID)as LocalNetID,coalesce(T1.BSCID, T2.BSCID)as BSCID,coalesce(T1.SiteID, T2.SiteID)as SiteID, coalesce(T1.CellID, T2.CellID)as CellID,
coalesce(T1.StrTime, T2.StrTime) as StrTime,
C10701 as '10701',C11601 as '11601'from
(
--Start for Opt_Cell_pBasicM of inner table--
(
select
LocalNetID,BSCID,SiteID, CellID,
StrTime,RecordDataTime,
C11601
from
Opt_Cell_pBasicM
where
((LocalNetID = 13 and BSCID = 2))
and
((RecordDataTime between '2006-01-01 00:01:00' and '2006-01-01 12:00:00'))
and
(GranularityPeriod <= 12)
)
--End for Opt_Cell_pBasicM of inner table--
as T1
full outer join
--Start for Opt_Cell_pCellRadioM of inner table--
(
select
LocalNetID,BSCID, SiteID, CellID,
StrTime,RecordDataTime,
C10701
from
Opt_Cell_pCellRadioM
where
(
((LocalNetID = 13 and BSCID = 2))
and
((RecordDataTime between '2006-01-01 00:01:00' and '2006-01-01 12:00:00'))
and
(GranularityPeriod <= 12)
)
)
--End for Opt_Cell_pCellRadioM of inner table--
as T2
on coalesce(T1.LocalNetID,NULL) = T2.LocalNetID and coalesce(T1.BSCID,NULL) = T2.BSCID and coalesce(T1.SiteID,NULL) = T2.SiteID and coalesce(T1.CellID,NULL) = T2.CellID
and coalesce(T1.RecordDataTime,NULL) = T2.RecordDataTime
--End for first join--
)
when num of the records of Opt_Cell_pBasicM and Opt_Cell_pCellRadioM
is greater than 1000000, the speed is very slow , how to optimize it ?
thks
View 3 Replies
View Related
Nov 16, 2006
the sql of create table is below:
CREATE TABLE [dbo].[Examp] (
[LocalNetID] [int] NOT NULL ,
[BSCID] [int] NOT NULL ,
[SiteID] [int] NOT NULL ,
[CellID] [int] NOT NULL ,
[StrTime] [varchar] (30) COLLATE Chinese_PRC_CI_AS NOT NULL ,
[M1] [int] NULL ,
[M2] [int] NULL ,
[M3] [int] NULL ,
[M4] [int] NULL ,
[M5] [int] NULL ,
[M6] [int] NULL ,
[M7] [int] NULL ,
[M8] [int] NULL ,
[M9] [int] NULL ,
[M10] [int] NULL ,
[M11] [int] NULL ,
[M12] [int] NULL ,
[M13] [int] NULL ,
[M14] [int] NULL ,
[M15] [int] NULL ,
[M16] [int] NULL ,
[M17] [int] NULL ,
[M18] [int] NULL ,
[M19] [int] NULL ,
[M20] [int] NULL ,
[M21] [int] NULL ,
[M22] [int] NULL ,
[M23] [int] NULL ,
[M24] [int] NULL ,
[M25] [int] NULL ,
[M26] [int] NULL ,
[M27] [int] NULL ,
[M28] [int] NULL ,
[M29] [int] NULL ,
[M30] [int] NULL ,
[M31] [int] NULL ,
[M32] [int] NULL ,
[M33] [int] NULL ,
[M34] [int] NULL ,
[M35] [int] NULL ,
[M36] [int] NULL ,
[M37] [int] NULL ,
[M38] [int] NULL ,
[M39] [int] NULL ,
[M40] [int] NULL ,
[M41] [int] NULL ,
[M42] [int] NULL ,
[M43] [int] NULL ,
[M44] [int] NULL ,
[M45] [int] NULL ,
[M46] [int] NULL ,
[M47] [int] NULL ,
[M48] [int] NULL ,
[M49] [int] NULL ,
[M50] [int] NULL ,
[M51] [int] NULL ,
[M52] [int] NULL ,
[M53] [int] NULL ,
[M54] [int] NULL ,
[M55] [int] NULL ,
[M56] [int] NULL ,
[M57] [int] NULL ,
[M58] [int] NULL ,
[M59] [int] NULL ,
[M60] [int] NULL ,
[M61] [int] NULL ,
[M62] [int] NULL ,
[M63] [int] NULL ,
[M64] [int] NULL ,
[M65] [int] NULL ,
[M66] [int] NULL ,
[M67] [int] NULL ,
[M68] [int] NULL ,
[M69] [int] NULL ,
[M70] [int] NULL ,
[M71] [int] NULL ,
[M72] [int] NULL ,
[M73] [int] NULL ,
[M74] [int] NULL ,
[M75] [int] NULL ,
[M76] [int] NULL ,
[M77] [int] NULL ,
[M78] [int] NULL ,
[M79] [int] NULL ,
[M80] [int] NULL ,
[M81] [int] NULL ,
[M82] [int] NULL ,
[M83] [int] NULL ,
[M84] [int] NULL ,
[M85] [int] NULL ,
[M86] [int] NULL ,
[M87] [int] NULL ,
[M88] [int] NULL ,
[M89] [int] NULL ,
[M90] [int] NULL ,
[M91] [int] NULL ,
[M92] [int] NULL ,
[M93] [int] NULL ,
[M94] [int] NULL ,
[M95] [int] NULL ,
[M96] [int] NULL ,
[M97] [int] NULL ,
[M98] [int] NULL ,
[M99] [int] NULL ,
[M100] [int] NULL
)
and LocalNetID, BSCID, SiteID, CellID, StrTime are set index,
the sql i want to query is :
select
LocalNetID, BSCID, SiteID, CellID, StrTime,
Sum(M1) as M1,Sum(M2) as M2,Sum(M3) as M3,Sum(M4) as M4,Sum(M5) as M5,Sum(M6) as M6,Sum(M7) as M7,Sum(M8) as M8,Sum(M9) as M9,Sum(M10) as M10,Sum(M11) as M11,Sum(M12) as M12,Sum(M13) as M13,Sum(M14) as M14,Sum(M15) as M15,Sum(M16) as M16,Sum(M17) as M17,Sum(M18) as M18,Sum(M19) as M19,Sum(M20) as M20,Sum(M21) as M21,Sum(M22) as M22,Sum(M23) as M23,Sum(M24) as M24,Sum(M25) as M25,Sum(M26) as M26,Sum(M27) as M27,Sum(M28) as M28,Sum(M29) as M29,Sum(M30) as M30,Sum(M31) as M31,Sum(M32) as M32,Sum(M33) as M33,Sum(M34) as M34,Sum(M35) as M35,Sum(M36) as M36,Sum(M37) as M37,Sum(M38) as M38,Sum(M39) as M39,Sum(M40) as M40,Sum(M41) as M41,Sum(M42) as M42,Sum(M43) as M43,Sum(M44) as M44,Sum(M45) as M45,Sum(M46) as M46,Sum(M47) as M47,Sum(M48) as M48,Sum(M49) as M49,Sum(M50) as M50,Sum(M51) as M51,Sum(M52) as M52,Sum(M53) as M53,Sum(M54) as M54,Sum(M55) as M55,Sum(M56) as M56,Sum(M57) as M57,Sum(M58) as M58,Sum(M59) as M59,Sum(M60) as M60,Sum(M61) as M61,Sum(M62) as M62,Sum(M63) as M63,Sum(M64) as M64,Sum(M65) as M65,Sum(M66) as M66,Sum(M67) as M67,Sum(M68) as M68,Sum(M69) as M69,Sum(M70) as M70,Sum(M71) as M71,Sum(M72) as M72,Sum(M73) as M73,Sum(M74) as M74,Sum(M75) as M75,Sum(M76) as M76,Sum(M77) as M77,Sum(M78) as M78,Sum(M79) as M79,Sum(M80) as M80,Sum(M81) as M81,Sum(M82) as M82,Sum(M83) as M83,Sum(M84) as M84,Sum(M85) as M85,Sum(M86) as M86,Sum(M87) as M87,Sum(M88) as M88,Sum(M89) as M89,Sum(M90) as M90,Sum(M91) as M91,Sum(M92) as M92,Sum(M93) as M93,Sum(M94) as M94,Sum(M95) as M95,Sum(M96) as M96,Sum(M97) as M97,Sum(M98) as M98,Sum(M99) as M99,Sum(M100) as M100
from
(
select
LocalNetID, BSCID, SiteID, CellID, StrTime,
Sum(M1) as M1,Sum(M2) as M2,Sum(M3) as M3,Sum(M4) as M4,Sum(M5) as M5,Sum(M6) as M6,Sum(M7) as M7,Sum(M8) as M8,Sum(M9) as M9,Sum(M10) as M10,Sum(M11) as M11,Sum(M12) as M12,Sum(M13) as M13,Sum(M14) as M14,Sum(M15) as M15,Sum(M16) as M16,Sum(M17) as M17,Sum(M18) as M18,Sum(M19) as M19,Sum(M20) as M20,Sum(M21) as M21,Sum(M22) as M22,Sum(M23) as M23,Sum(M24) as M24,Sum(M25) as M25,Sum(M26) as M26,Sum(M27) as M27,Sum(M28) as M28,Sum(M29) as M29,Sum(M30) as M30,Sum(M31) as M31,Sum(M32) as M32,Sum(M33) as M33,Sum(M34) as M34,Sum(M35) as M35,Sum(M36) as M36,Sum(M37) as M37,Sum(M38) as M38,Sum(M39) as M39,Sum(M40) as M40,Sum(M41) as M41,Sum(M42) as M42,Sum(M43) as M43,Sum(M44) as M44,Sum(M45) as M45,Sum(M46) as M46,Sum(M47) as M47,Sum(M48) as M48,Sum(M49) as M49,Sum(M50) as M50,Sum(M51) as M51,Sum(M52) as M52,Sum(M53) as M53,Sum(M54) as M54,Sum(M55) as M55,Sum(M56) as M56,Sum(M57) as M57,Sum(M58) as M58,Sum(M59) as M59,Sum(M60) as M60,Sum(M61) as M61,Sum(M62) as M62,Sum(M63) as M63,Sum(M64) as M64,Sum(M65) as M65,Sum(M66) as M66,Sum(M67) as M67,Sum(M68) as M68,Sum(M69) as M69,Sum(M70) as M70,Sum(M71) as M71,Sum(M72) as M72,Sum(M73) as M73,Sum(M74) as M74,Sum(M75) as M75,Sum(M76) as M76,Sum(M77) as M77,Sum(M78) as M78,Sum(M79) as M79,Sum(M80) as M80,Sum(M81) as M81,Sum(M82) as M82,Sum(M83) as M83,Sum(M84) as M84,Sum(M85) as M85,Sum(M86) as M86,Sum(M87) as M87,Sum(M88) as M88,Sum(M89) as M89,Sum(M90) as M90,Sum(M91) as M91,Sum(M92) as M92,Sum(M93) as M93,Sum(M94) as M94,Sum(M95) as M95,Sum(M96) as M96,Sum(M97) as M97,Sum(M98) as M98,Sum(M99) as M99,Sum(M100) as M100
from
(
select
LocalNetID, BSCID, SiteID, CellID, StrTime,
Sum(M1) as M1,Sum(M2) as M2,Sum(M3) as M3,Sum(M4) as M4,Sum(M5) as M5,Sum(M6) as M6,Sum(M7) as M7,Sum(M8) as M8,Sum(M9) as M9,Sum(M10) as M10,Sum(M11) as M11,Sum(M12) as M12,Sum(M13) as M13,Sum(M14) as M14,Sum(M15) as M15,Sum(M16) as M16,Sum(M17) as M17,Sum(M18) as M18,Sum(M19) as M19,Sum(M20) as M20,Sum(M21) as M21,Sum(M22) as M22,Sum(M23) as M23,Sum(M24) as M24,Sum(M25) as M25,Sum(M26) as M26,Sum(M27) as M27,Sum(M28) as M28,Sum(M29) as M29,Sum(M30) as M30,Sum(M31) as M31,Sum(M32) as M32,Sum(M33) as M33,Sum(M34) as M34,Sum(M35) as M35,Sum(M36) as M36,Sum(M37) as M37,Sum(M38) as M38,Sum(M39) as M39,Sum(M40) as M40,Sum(M41) as M41,Sum(M42) as M42,Sum(M43) as M43,Sum(M44) as M44,Sum(M45) as M45,Sum(M46) as M46,Sum(M47) as M47,Sum(M48) as M48,Sum(M49) as M49,Sum(M50) as M50,Sum(M51) as M51,Sum(M52) as M52,Sum(M53) as M53,Sum(M54) as M54,Sum(M55) as M55,Sum(M56) as M56,Sum(M57) as M57,Sum(M58) as M58,Sum(M59) as M59,Sum(M60) as M60,Sum(M61) as M61,Sum(M62) as M62,Sum(M63) as M63,Sum(M64) as M64,Sum(M65) as M65,Sum(M66) as M66,Sum(M67) as M67,Sum(M68) as M68,Sum(M69) as M69,Sum(M70) as M70,Sum(M71) as M71,Sum(M72) as M72,Sum(M73) as M73,Sum(M74) as M74,Sum(M75) as M75,Sum(M76) as M76,Sum(M77) as M77,Sum(M78) as M78,Sum(M79) as M79,Sum(M80) as M80,Sum(M81) as M81,Sum(M82) as M82,Sum(M83) as M83,Sum(M84) as M84,Sum(M85) as M85,Sum(M86) as M86,Sum(M87) as M87,Sum(M88) as M88,Sum(M89) as M89,Sum(M90) as M90,Sum(M91) as M91,Sum(M92) as M92,Sum(M93) as M93,Sum(M94) as M94,Sum(M95) as M95,Sum(M96) as M96,Sum(M97) as M97,Sum(M98) as M98,Sum(M99) as M99,Sum(M100) as M100
from
(
select
LocalNetID, BSCID, SiteID, CellID, StrTime,
Sum(M1) as M1,Sum(M2) as M2,Sum(M3) as M3,Sum(M4) as M4,Sum(M5) as M5,Sum(M6) as M6,Sum(M7) as M7,Sum(M8) as M8,Sum(M9) as M9,Sum(M10) as M10,Sum(M11) as M11,Sum(M12) as M12,Sum(M13) as M13,Sum(M14) as M14,Sum(M15) as M15,Sum(M16) as M16,Sum(M17) as M17,Sum(M18) as M18,Sum(M19) as M19,Sum(M20) as M20,Sum(M21) as M21,Sum(M22) as M22,Sum(M23) as M23,Sum(M24) as M24,Sum(M25) as M25,Sum(M26) as M26,Sum(M27) as M27,Sum(M28) as M28,Sum(M29) as M29,Sum(M30) as M30,Sum(M31) as M31,Sum(M32) as M32,Sum(M33) as M33,Sum(M34) as M34,Sum(M35) as M35,Sum(M36) as M36,Sum(M37) as M37,Sum(M38) as M38,Sum(M39) as M39,Sum(M40) as M40,Sum(M41) as M41,Sum(M42) as M42,Sum(M43) as M43,Sum(M44) as M44,Sum(M45) as M45,Sum(M46) as M46,Sum(M47) as M47,Sum(M48) as M48,Sum(M49) as M49,Sum(M50) as M50,Sum(M51) as M51,Sum(M52) as M52,Sum(M53) as M53,Sum(M54) as M54,Sum(M55) as M55,Sum(M56) as M56,Sum(M57) as M57,Sum(M58) as M58,Sum(M59) as M59,Sum(M60) as M60,Sum(M61) as M61,Sum(M62) as M62,Sum(M63) as M63,Sum(M64) as M64,Sum(M65) as M65,Sum(M66) as M66,Sum(M67) as M67,Sum(M68) as M68,Sum(M69) as M69,Sum(M70) as M70,Sum(M71) as M71,Sum(M72) as M72,Sum(M73) as M73,Sum(M74) as M74,Sum(M75) as M75,Sum(M76) as M76,Sum(M77) as M77,Sum(M78) as M78,Sum(M79) as M79,Sum(M80) as M80,Sum(M81) as M81,Sum(M82) as M82,Sum(M83) as M83,Sum(M84) as M84,Sum(M85) as M85,Sum(M86) as M86,Sum(M87) as M87,Sum(M88) as M88,Sum(M89) as M89,Sum(M90) as M90,Sum(M91) as M91,Sum(M92) as M92,Sum(M93) as M93,Sum(M94) as M94,Sum(M95) as M95,Sum(M96) as M96,Sum(M97) as M97,Sum(M98) as M98,Sum(M99) as M99,Sum(M100) as M100
from
(
select
LocalNetID, BSCID, SiteID, CellID, StrTime,
M1 , M2 , M3 , M4 , M5 , M6 , M7 , M8 , M9 , M10 ,M11 , M12 , M13 , M14 , M15 , M16 , M17 , M18 , M19 , M20 ,M21 , M22 , M23 , M24 , M25 , M26 , M27 , M28 , M29 , M30 ,M31 , M32 , M33 , M34 , M35 , M36 , M37 , M38 , M39 , M40 ,M41 , M42 , M43 , M44 , M45 , M46 , M47 , M48 , M49 , M50 ,M51 , M52 , M53 , M54 , M55 , M56 , M57 , M58 , M59 , M60 ,M61 , M62 , M63 , M64 , M65 , M66 , M67 , M68 , M69 , M70 ,M71 , M72 , M73 , M74 , M75 , M76 , M77 , M78 , M79 , M80 ,M81 , M82 , M83 , M84 , M85 , M86 , M87 , M88 , M89 , M90 ,M91 , M92 , M93 , M94 , M95 , M96 , M97 , M98 , M99 , M100
from examp
) as T1
group by
LocalNetID, BSCID, SiteID, CellID, StrTime
) as T2
group by
LocalNetID, BSCID, SiteID, CellID, StrTime
) as T3
group by
LocalNetID, BSCID, SiteID, CellID, StrTime
) as T4
group by
LocalNetID, BSCID, SiteID, CellID, StrTime
when there is no data in table, it will take more than 3 minutes on my computer!
how can i optimize the sql to speed it?
thks
View 2 Replies
View Related
Feb 6, 2007
It's bad enough that SQL CE 2.0 doesn't support views, when I use "LIKE" in a SELECT statement (eg., SELECT ... WHERE fldname LIKE '%word%'), the response is terrible to the point that it doesn't come back. In my case, the 'word' can be anywhere in 'fldname'.
How do you optimize the LIKE operator? I created an index on the field but it didn't make a bit of difference.
Thank you.
View 1 Replies
View Related
Feb 2, 2007
Hello friends,
I’m facing performance related problem while running following query on SQL Server 2000.
This query is basically used to find last location of each unit that are passed. Here I am passing data like “‘26','27','28','29','30','31','32','33','34','35','36','37','38','39','40','41','42','43','44','45','46','47','48','49','50'" in @Units variable. But it takes too much time and I don’t get output. Table is having around 5 Million records.
Query:
SELECT Alias, tblUnit.UnitID, Location, Latitude, Longitude,Convert(varchar(10),UnitHistoryDate,1) + ' ' + substring(Convert(varchar(40),UnitHistoryDate,109),12,9) + ' ' + substring(Convert(varchar(40),UnitHistoryDate,109),25,2) as [Date/Time], unithistoryid
FROM tblUnit INNER JOIN
tblUnitHistory ON tblUnit.UnitID = tblUnitHistory.UnitID
WHERE tblUnitHistory.UnitHistoryDate
IN (SELECT MAX(UnitHistoryDate) FROM tblUnitHistory WHERE tblUnitHistory.UnitID in (' + @Units + ') GROUP BY tblUnitHistory.UnitID)
AND tblUnit.UnitID in (' + @Units + ')
ORDER BY tblUnit.UnitID
Table Structure:
UnitHistoryID int Primary Key
UnitID int
Location varchar(200)
Latitude decimal 9
Longitude decimal 9
Speed decimal 5
BatteryVoltage decimal 5
ReasonCode int
DistanceFromLastLocation decimal 9
UnitHistoryDate datetime
Indexes:
1. Clustered Index on Column UnitID
2. Non-clustered Index on Column UnitHistoryDate
3. Non-clustered Index on Column UnitHistoryID
Please help me to write optimize query and suggest me the steps to require making this query faster. Any help is appreciated. Thanks in advance.
Regards,
Sandeep
View 1 Replies
View Related
Oct 12, 2007
I have a nightly job that aggregates data into a summary table. Each night, an additional item may or may not be added, so the table is cleared and repopulated. I've got this working fine. Now, I am being asked to track when an item first appears in the summary table. Here is the code I am using to do this. I realize it is very poor performance, but I don't know how to make it better.
MyTable Columns: id INT PK ,v1 VARCHAR(4) ,v2 VARCHAR(10 ,v3 DECIMAL ,item1 BIT ,item2 BIT ,item3 BIT
MyTable2 Columns: v1 VARCHAR(4) ,v2 VARCHAR(10 ,v3 DECIMAL ,item INT ,FirstKnownDate DATETIME
All columns are NOT NULL.1 DECLARE @iNextRowID INT
2 ,@iCurrentRowID INT
3 ,@iLoopControl INT
4 ,@v1 VARCHAR(4)
5 ,@v2 VARCHAR(10)
6 ,@v3 DECIMAL
7
8 /* This is a loop that executes for a dynamic number of items, eg item1=1, then item2=1, etc */
9 SET @iLoopControl=1
10 SELECT @iNextRowID=MIN(id) FROM MyTable WHERE item1=1
11
12 IF ISNULL(@iNextRowID,0) <> 0 BEGIN
13 SELECT @iCurrentRowID = id
14 ,@v1 = v1
15 ,@v2 = v2
16 ,@v3 = v3
17 FROM MyTable
18 WHERE id=@iNextRowID
19
20 WHILE @iLoopControl = 1 BEGIN
21 IF NOT EXISTS(SELECT * FROM MyTable2 WHERE v1=@v2 AND v2=@v2 AND v3=@v3 AND item=1) BEGIN
22 INSERT INTO MyTable2
23 /* COLUMN */ (v1,v2,v3,item,firstknowndate)
24 SELECT @v1, @v2, @v3, 1, GETDATE()
25 END
26
27 SELECT @iNextRowID = NULL
28 SELECT @iNextRowID = MIN(id) FROM MyTable WHERE id>@iCurrentRowID AND item1=1
29
30 IF ISNULL(@iNextRowID,0) = 0 BEGIN
31 BREAK
32 END
33
34 SELECT @iCurrentRowID = id
35 ,@v1 = v1
36 ,@v2 = v2
37 ,@v3 = v3
38 FROM MyTable
39 WHERE id=@iNextRowID
40 END
41 END42 /* This is the end of the items loop */
This query takes 5 minutes to do straight inserts without the test for exists in the loop for 3 items. MyTable has just under 96,000 rows in it. MyTable2 has 121,857 rows in it. It is not about 55 minutes and it is still running with the exists so that a row isn't inserted twice. Is there anything I can do to optimize this? EDIT: The query just finished running again with the test for exists on each row. It shouldn't have inserted any new rows, but it took 57 minutes and inserted another 114,115 rows. Not good. Anyone see what is wrong with the query?
View 4 Replies
View Related
Mar 9, 2008
Hello, I have three sql select statments I would like to combine into one. I have created a statment that works but I am not sure if it is a good solution performance wise. Is there a better way to run this query?
Thanks Very Much!!
if exists (Select Top 1 snapsht_id, snpsht_flname, site_url, iWidth, iHeight, isFullPage, fp_flname, fp_iWidth, fp_iHeight
From SiteIndex Where update_freq = 0 and nextupdate < GetDate() Order By nextupdate)
Begin
Select Top 1 snapsht_id, snpsht_flname, site_url, iWidth, iHeight, isFullPage, fp_flname, fp_iWidth, fp_iHeight
From SiteIndex Where update_freq = 0 and nextupdate < GetDate() Order By nextupdate
End
else
if exists (Select Top 1 snapsht_id, snpsht_flname, site_url, iWidth, iHeight, isFullPage, fp_flname, fp_iWidth, fp_iHeight
From SiteIndex Where nextupdate < GetDate() Order By importance desc)
begin
Select Top 1 snapsht_id, snpsht_flname, site_url, iWidth, iHeight, isFullPage, fp_flname, fp_iWidth, fp_iHeight
From SiteIndex Where nextupdate < GetDate() Order By importance desc
end
else
Select Top 1 snapsht_id, snpsht_flname, site_url, iWidth, iHeight, isFullPage, fp_flname, fp_iWidth, fp_iHeight
From SiteIndex Order By nextupdate
View 1 Replies
View Related
Nov 14, 2007
Hi, We have surveys that gather data and we're trying to find the best way to query this data for reporting. It is not too late to change everything so we're open to any suggestions. Here is my current setup:
Results table from which each row represents a user that filled in the survey so it contains a unique identifier, a survey identifier and some other data which we use elsewhere.
Result_Answers table which holds the answers for each result. So it contains the result ID, the question ID and the answer ID
Questions and Answers table which hold the actual text for each identifier used in result_answers.
What we're trying to do is retrieve the count for each answers (thats easy) based on a set of conditional answers. For exemple:
Q1 - Are you male or female
Q2 - What is your age group
Q3 - What type of music do you listen to.
So lets say I want the count of each answers for each question but limited to the results that answered male on Q1 and 18-25 OR 25-35 on Q2. The only way I found to do this is create a cursor to loop trough each questions, select the result IDs that answered Q1 correctly and then remove from that temp table each result ID that didn't answer the subsequent questions correctly. This gives me a page long query with a cursor and multiple IN commands (for when there's more than one answer valid for a single question)
I'm not saying the results are abysmal, I'm still getting my results inside 2s with a bit over 1 million rows in those tables. But unfortunatly, this is basically only one survey... I can only imagine that it will get longer and longer as we add more surveys.
So my question is: Is there a better way to query this data or worst case, is my table schema not efficient enough? In both cases, how can I fix/optimize it?
I'm really hoping you guys can help me out!
View 1 Replies
View Related
May 25, 2005
Hi all,
I need help to optimize this query:
Code:
SELECT s.konto, s.sifra_partnera, s.temeljnica, s.rbr, s.duguje, s.potrazuje, s.saldo
FROM saldakonti s
WHERE
s.datum<'2005-01-01' AND
s.uplata='R' AND
EXISTIS (SELECT t.temeljnica, t.rbr FROM saldakonti t WHERE
t.konto=s.konto AND
t.sifra_partnera=s.sifra_partnera AND
t.vezni_broj=s.vezni_broj and
t.datum<'2005-01-01' AND
(t.uplata='S' or t.uplata='U')
GROUP BY t.temeljnica, t.rbr
HAVING
abs(sum(t.duguje-t.potrazuje))<>abs(s.saldo))
ORDER BY
s.konto, s.sifra_partnera, s.temeljnica, s.rbr, s.duguje, s.potrazuje, s.saldo
Currently this query takes forever to execute. Any suggestions appreciated.
Thanks in advance.
View 2 Replies
View Related
Sep 1, 2004
What are the best pactices to optimize performance accsiing an SQLServer DB ?
commands, mantenance plan...
Thanks
View 1 Replies
View Related
Feb 17, 2012
is there any command to optimize a database for eg: after 1 month if DB is full of mess then with one command every thing will be in sorted order or something like that to increase the efficiency and speed, is there any thing like that in MSSQL 2005?
View 2 Replies
View Related
Feb 26, 2004
Can someone look at this sql statement and tell me if it can be sped up? Also I have to add to it by joining it with another table. How do I do that? Just by nesting another join?
Thanks!
Set rs=Server.CreateObject("ADODB.Recordset")
sql = "SELECT td.TeamID, td.TeamName, rt.PartID, rt.Effort, rt.UnitMeas, pd.MinMilesConv "
sql = sql & "FROM TeamData td INNER JOIN PartData pd ON td.TeamID = pd.TeamID "
sql = sql & "JOIN RunTrng rt ON pd.PartID = rt.PartID "
sql = sql & "WHERE rt.TrngDate >= '" & Session("beg_date") & "' AND rt.TrngDate < '" & Session("end_date")
sql = sql & "' AND pd.Archive = 'N' AND pd.Gender = '" & sGender & "' AND pd.Grade >= " & iMinGrade
sql = sql & " AND pd.Grade <= " & iMaxGrade & " ORDER BY td.TeamID"
rs.Open sql, conn, 1, 2
View 2 Replies
View Related
Mar 25, 2004
Hi all,
Table 'A' is having 105 fields & 233000 records.
One Clusterd Primary Key & 10 nonclusterd key.
If I joined with table 'A' or 'Select * from 'A' , Query takes more time so please let me know best way to structure query or table, Indexes etc.
Reply to me asap positivaly ....
Regards,
M. G.
View 1 Replies
View Related
Apr 1, 2004
Hi All,
please look into below query.
SELECT TOP 101 Cert_WarehouseDetails.CertID, Cert_WarehouseDetails.UnderwriterID,
Cert_WarehouseDetails.WarehouseEntriesID, Cert_WarehouseDetails.DivisionWarehouseLocID,
Cert_WarehouseDetails.TypeID, Cert_WarehouseDetails.ReportingType, Cert_WarehouseDetails.CertWHID,
Cert_WarehouseDetails.ClientPolicyDivisionID, Cert_WarehouseDetails.RecordNO, Cert_WarehouseDetails.InsuredValueX,
Cert_WarehouseDetails.PremiumTotalX, Cert_WarehouseDetails.StatusX, Cert_WarehouseDetails.StatusID,
Cert_WarehouseDetails.AuthorizeDateX, Cert_WarehouseDetails.CodeX, Cert_WarehouseDetails.IssuedDateX, Cert_WarehouseDetails.BillofLadingDateX,
DOGX, Cert_WarehouseDetails.ConveyanceNameX, Cert_WarehouseDetails.LogonNameX , Cert_WarehouseDetails.ClientNameX,
(CASE WHEN UnderwriterID = 0 THEN ' ' ELSE tblClientUsers.FirstName + ' ' + tblClientUsers.LastName END) AS OwnedBy,
(CASE WHEN UnderwriterID = 0 THEN CAST(' ' AS NVARCHAR) ELSE CAST(TakeOwnershipDate as nvarchar) END) AS OwnedDate
FROM
( SELECT tblCertificates.[CertID] AS CertID, tblCertificates.[UnderwriterID] AS UnderwriterID, tblReportingType.[TypeID] AS TypeID, tblReportingType.[ReportingType] AS ReportingType, NULL AS WarehouseEntriesID, NULL AS DivisionWarehouseLocID , tblCertificates.CertID AS CertWHID, tblCertificates.ClientPolicyDivisionsID AS ClientPolicyDivisionID, tblCertificates.CertificateNo AS RecordNO, tblCertificates.TotalInsured AS InsuredValueX, (CASE WHEN tblCertificates.Status=101 or tblCertificates.Status=104 or tblCertificates.DivReferral=1 THEN 0 ELSE PremiumTotal-tblCertificates.Discount END) AS PremiumTotalX, tblStatus.Description AS StatusX, tblStatus.StatusID AS StatusID, (CASE WHEN tblCertificates.Status < '105' and tblCertificates.Status <> '103' THEN null ELSE AuthorizeDate END) AS AuthorizeDateX, tblCurrency.Code AS CodeX, tblCertificates.IssuedDate AS IssuedDateX, tblCertificates.BillofLadingDate AS BillofLadingDateX, tblCertificates.DescriptionofGoods AS DOGX, tblCertificates.ConveyanceName AS ConveyanceNameX, tblClientUsers.LogonName AS LogonNameX , tblClient.ClientName AS ClientNameX, tblCertificates.TakeOwnershipDate AS TakeOwnershipDate, tblCertificates.ClientID AS ClientID, tblCertificates.Producer AS BrokerID, tblCertificates.SBU AS SBU, tblCertificates.AssociationID AS AssociationID, tblCertificates.AssuredName AS AssuredName, tblCertificates.UserID AS UserID, tblCertificates.Demoflag AS Demoflag FROM tblCertificates, tblReportingType,tblcurrency,tblClientUsers,tblCli ent,tblStatus WHERE tblCertificates.reportType = tblReportingType.TypeID AND tblCertificates.PremiumCurrencyType = tblCurrency.ID AND tblclientUsers.UserID = tblCertificates.UserID AND tblcertificates.ClientId = tblclient.ClientID AND tblStatus.StatusID = tblCertificates.Status
UNION
SELECT NULL AS CertID, NULL AS UnderwriterID, tblReportingType.[TypeID] AS TypeID, tblReportingType.[ReportingType] AS ReportingType, tblWarehouseEntries.[WarehouseEntriesID] AS WarehouseEntriesID, tblWarehouseEntries.[DivisionWarehouseLocID] AS DivisionWarehouseLocID , tblWarehouseEntries.WarehouseEntriesID AS CertWHID, tblWarehouseEntries.ClientPolicyDivisionID AS ClientPolicyDivisionID, tblWarehouseEntries.WarehouseEntriesID AS RecordNO, (CASE WHEN ValueCurrencyType=0 THEN 0 ELSE UnitValue END) AS InsuredValueX, (CASE WHEN tblWarehouseEntries.StatusID=101 THEN 0 ELSE PremiumSum END) AS PremiumTotalX, tblStatus.Description AS StatusX, tblStatus.StatusID AS StatusID, (CASE WHEN tblWarehouseEntries.StatusID < '105' THEN null ELSE ApprovalDate END) AS AuthorizeDateX, tblCurrency.Code AS CodeX, tblWarehouseEntries.IssuedDate AS IssuedDateX, tblWarehouseEntries.PeriodEndDate AS BillofLadingDateX, LocName AS DOGX, '' AS ConveyanceNameX, tblClientUsers.LogonName AS LogonNameX , tblClient.ClientName AS ClientNameX, tblWarehouseEntries.TakeOwnershipDate AS TakeOwnershipDate, tblWarehouseEntries.ClientID AS ClientID, tblWarehouseEntries.BrokerID AS BrokerID, tblWarehouseEntries.SBU AS SBU, tblWarehouseEntries.AssociationID AS AssociationID, tblWarehouseEntries.AssuredName AS AssuredName, tblWarehouseEntries.UserID AS UserID, tblWarehouseEntries.Demoflag AS Demoflag FROM tblWarehouseLoc, tblWarehouseEntries, tblReportingType,tblCurrency ,tblClientUsers,tblClient, tblDivisionWarehouseLoc,tblStatus WHERE tblWarehouseEntries.reportTypeID = tblReportingType.TypeID and tblWarehouseLoc.WarehouseLocationID = tblDivisionWarehouseLoc.WarehouseLocID and tblDivisionWarehouseLoc.DivisionWarehouseLocID = tblWarehouseEntries.DivisionWarehouseLocID and tblWarehouseEntries.PremiumCurrencyType = tblCurrency.ID and tblWarehouseEntries.UserID = tblClientUsers.UserID and tblClient.ClientID = tblWarehouseEntries.ClientID AND tblStatus.StatusID = tblWarehouseEntries.StatusID
) AS Cert_WarehouseDetails LEFT JOIN tblClientUsers ON Cert_WarehouseDetails.UnderwriterID = tblClientUsers.UserID
WHERE 1 = 1 AND Cert_WarehouseDetails.Demoflag = 0 and
(convert(datetime,convert(nvarchar,Cert_WarehouseD etails.IssuedDateX,101)) >= '1/1/2003') and
(Cert_WarehouseDetails.IssuedDateX is not null and
convert(datetime,convert(nvarchar,Cert_WarehouseDe tails.IssuedDateX,101)) <= '3/31/2004')
ORDER BY Cert_WarehouseDetails.RecordNO
In above query,as
1. Union (INNER) query returns me 200000 records.
2. If I run above query it takes 18 sec because no index on Cert_WarehouseDetails.RecordNO and i'm ordering on that.
3. When I avoid to 'ORDER BY Cert_WarehouseDetails.RecordNO' then query takes 2 sec.
4. In this case Do I have to use 'nvarchar'
convert(datetime,convert(nvarchar,Cert_WarehouseDe tails.IssuedDateX,101)) <= '3/31/2004')
5. Why do we have to use 'nvarchar' can you explain to me in above statement..
Can you provide me alternate solution so I can make it fast.
Please reply to me asap. Thanks in advance.
Regards,
M. Jain
View 10 Replies
View Related
May 1, 2004
I have a very simple T-SQL statement which takes hours to run. The statement is something like:
update table1 set column1 = coalesce(table2.column1, ''),
column2 = table2.column2, column3 = table2.column3
from table1, table2
where table1.column4 = table2.column4
The machine is a P3 800 512 ram and 80G HD with 8M cache. I'm running Windows 2000 Server and SQL Server 2000. Table1 has about 500,000 records and Table2 has about 2 millions records. I have added non-clustered indexes for table1.column4 and table2.column4.
When I check Execution Plan, it has Hash Match/Aggregate which costs about 50%.
Thank you for any help.
View 14 Replies
View Related
Jun 24, 2008
Hello,
Can any one suggest me in optimizing the SP.
To execute single SP it takes nearly 50 seconds.
Can i know what are ways to optimize the SP.
Thanks
Ganesh
Solutions are easy. Understanding the problem, now, that's the hard part
View 4 Replies
View Related
Jul 23, 2005
I have a SqlServer 2000 running 7 databases. One in particular is slowerthan the rest specificaly on certain tables. It seems the problem comesfrom the volume of data, but I don't think 3000 lines can be considered abig table. I have indexes on the principal keys, but it did not help.thank you
View 2 Replies
View Related
May 18, 2006
Hi,I have these two tables in a DatabaseITEMSIDnumeric (Primary key)ZDIDnvarchar 3 (not null)IDF_Familynumeric(not null)Descriptionnvarchar40 (not null)DATASIDnumeric(Primary Key)IDF_Itemnumeric(Foreign key)IDF_Referencenumeric(Foreign Key)[Date]smalldatetime(not null)Containernchar10(not null)Averagedecimal(not null)[%Compliance]decimal(not null)[%OutOfRange<MinTg]decimal(not null)[%OutOfRange>MaxTg]decimal(not null)Targetdecimal(not null)[Min]decimal(not null)[Max]decimal(not null)The table DATAS has 4000000+ recordsI'm running this query:SELECT DISTINCT I.ID, I.ZDID, I.IDF_Family, I.DescriptionFROM Items as I, Datas as DWHERE D.IDF_Item = I.ID AND I.IDF_Family = 84AND D.Date BETWEEN '5/18/2004' AND '5/18/2004'it's taking 4-5 minutes to run.The result is correct, there is no thing on that date.I've done a reindex, but still the same thing.What can I do?Thanks
View 4 Replies
View Related
Jun 6, 2006
This code is attempting to find records that have a RegJrnID that doesnot occur more than one time in the table.The reason that I want to find records with non-duplicated RegJrnIDvalues is to create "reversal" records for these such that the reversalrecord has identical values for every column except the TaxableAmountwhich will contain a negative amount. (see: example data below)./* Set up */CREATE TABLE t1(RegJrnID INTEGER, InvoiceDate VARCHAR(8), InvoiceNumberVARCHAR(20), TaxableAmount DECIMAL(32,8))/* Example data */INSERT INTO t1 VALUES (1, '20060101', '2321323', 100.00)INSERT INTO t1 VALUES (9, '20060213', '2130009', 40.01)INSERT INTO t1 VALUES (3, '20060101', '9402293', 512.44)INSERT INTO t1 VALUES (1, '20060104', '2321323', -100.00)INSERT INTO t1 VALUES (4, '20060105', '9302221', 612.12)INSERT INTO t1 VALUES (5, '20060105', '0003235', 18.11)INSERT INTO t1 VALUES (6, '20060111', '5953432', 2101.21)INSERT INTO t1 VALUES (3, '20060111', '9402293', -512.44)INSERT INTO t1 VALUES (7, '20060115', '4234444', 44.52)INSERT INTO t1 VALUES (8, '20060115', '0342222', 95.21)INSERT INTO t1 VALUES (6, '20060119', '5953432', -2101.21)INSERT INTO t1 VALUES (2, '20060101', '5440033', 231.01)/* Show what's in the table - just because */SELECT * FROM t1 ORDER BY RegJrnID, InvoiceDate/* Query for records to reverse */SELECT *FROM t1 a/* Ignore records that have already been reversed */WHERE a.RegJrnID != ALL/* This subselect finds reversed records (i.e. those that have aduplicate RegJrnID) */(SELECT b.RegJrnIDFROM t1 bGROUP BY b.RegJrnIDHAVING COUNT(*) > 1)/* User selection criteria are appended here *//* AND InvoiceNumber >= '5000000' AND InvoiceNumber <= '7500000' *//* Make the results look pretty (optional) */ORDER BY RegJrnID/* Housekeeping */DROP TABLE t1
View 19 Replies
View Related
Nov 17, 2006
Hello,I am trying to find all records that have matching detail_1 or detail_2columns. This is what I have now:select t1.id, t1.detail_1, t1.detail_2, t2.id from user_details t1,user_details t2where t1.detail_1 = t2.detail_1 or t1.detail_2 = t2.detail_2;Using smaller tables of around 1000 records, I have verified that thisindeed does the job. However, when I apply it to the real table thathas around 40,000 records, it takes so long that I have to stop thequery.The table in question is quite simple and is created as follows. Thereare no indexes on the table:create table user_details (id integer, detail_1 varchar(50), detail_2varchar(50)Is there a way to make it go faster?
View 4 Replies
View Related
Mar 3, 2008
Hi! I have an optimisation problem. I've currently got a query that retrieve about 10K records according the mode that user difined earlier.
At the very beginning I've wrote the same code through the user-defined functions, but I found in publications that it's a wrong way to develop with MSSQL, but it worked more and more faster (3 sec). Now I rewrite my query with inline table functions...CREATE FUNCTION ans_app_name1()
RETURNS TABLE AS RETURN
(SELECT CASE WHEN OBJECT_ID('app_name') > 0 THEN
(SELECT TOP(1) app_name FROM app_name WHERE spid = @@spid)
ELSE NULL END AS app_name )
GO
CREATE FUNCTION ans_get_user_property1(
@property_name VARCHAR(255))
RETURNS TABLE AS RETURN
(SELECT u.value AS user_property FROM user_property u
JOIN ans_app_name1() a ON a.app_name = u.app_name
WHERE property = @property_name AND
username = system_user)
GO
CREATE FUNCTION ans_get_mode()
RETURNS TABLE AS RETURN
(SELECT CAST(user_property AS INT) AS mode FROM ans_get_user_property1('mode'))
GO
CREATE FUNCTION ans_get_state_date()
RETURNS TABLE AS RETURN
SELECT
CASE
WHEN ((SELECT mode FROM ans_get_mode()) IN (3,6)) THEN
CAST(user_property AS DATETIME) END AS state_date
FROM ans_get_user_property1('state_date')
GO
ALTER FUNCTION is_real1(
@mode INT,
@state_date DATETIME)
RETURNS TABLE AS
RETURN (
SELECT uid,
CASE
WHEN (begin_date = 0) or (end_date = 0) THEN 0
WHEN
((@mode IN (1,5)) AND (begin_date <= CURRENT_TIMESTAMP) AND ((end_date IS NULL) OR (end_date > CURRENT_TIMESTAMP)))
OR ((@mode IN (2,5,6)) AND (begin_date IS NULL) AND (end_date IS NULL) AND ISNULL(is_committed, 0) = 0)
OR ((@mode IN (3,6)) AND (begin_date <= @state_date) AND ((end_date IS NULL) OR (end_date > @state_date)))
OR ((@mode = 4) AND (begin_date <= CURRENT_TIMESTAMP) AND ((end_date IS NULL) OR (end_date > CURRENT_TIMESTAMP)))
THEN 1
ELSE 0
END AS is_real
FROM host
)
GO
ALTER view v_host as
SELECT
h.uid,
h.id_histland,
h.id_group,
h.numporyadok,
h.numgroup,
h.objidentnum,
h.egrokn_reg_num,
h.kad_num,
h.ansname,
h.objname,
h.id_objprotstatus,
h.id_kategory,
h.id_objkind,
h.id_histcultvalue,
h.uid_objautor,
h.createdate,
h.id_scientifictype,
h.obj_description,
h.id_admdistrict,
h.cityblock,
h.id_settlement,
h.id_admdistrict2,
h.p_index,
h.id_street,
h.housenum,
h.housenumpart,
h.houseliteranum,
h.buildingnum,
h.corneradress,
h.appartmentnum,
h.id_urbanblock,
h.id_microdistrict,
h.kadblock,
h.end_date,
h.objlocation,
h.parent_uid,
h.kaddistrict,
h.mundistrict,
h.is_committed,
h.order_uid,
h.begin_date,
h.cr_date,
h.order_uid_end,
h.uid_parent_host,
h.is_host_object,
h.id_funcassign,
h.id_immovobjtype,
h.hist_data,
CASE WHEN h.obj_code is NULL THEN
'<object registered>'
ELSE h.obj_code
END AS obj_code,
s.sname AS sname_objprotstatus,
k.sname AS sname_kategory,
ok.sname AS sname_objkind,
p.sname AS name_property,
ISNULL(h.ansname + ' ', '') + h.objname as fullname,
h.pre_material,
h.hist_view_change,
h.adr_id,
h.street_id,
h.cod_mun,
h.hist_object_id,
h.scientifictype
FROM
host h
LEFT JOIN sl_objprotstatus s ON h.id_objprotstatus = s.id
LEFT JOIN sl_kategory k ON h.id_kategory = k.id
LEFT JOIN sl_objkind ok ON h.id_objkind = ok.id
LEFT JOIN sl_property p ON h.id_property = p.id
--JOIN is_real1((SELECT mode FROM ans_get_mode()), (SELECT state_date FROM ans_get_state_date())) r ON r.uid = h.uid
WHERE
(CASE
WHEN (begin_date = 0) or (end_date = 0) THEN 0
WHEN
(((SELECT mode FROM ans_get_mode()) IN (1,5)) AND (begin_date <= CURRENT_TIMESTAMP) AND ((end_date IS NULL) OR (end_date > CURRENT_TIMESTAMP)))
OR (((SELECT mode FROM ans_get_mode()) IN (2,5,6)) AND (begin_date IS NULL) AND (end_date IS NULL) AND ISNULL(is_committed, 0) = 0)
OR (((SELECT mode FROM ans_get_mode()) IN (3,6)) AND (begin_date <= (SELECT state_date FROM ans_get_state_date())) AND ((end_date IS NULL) OR (end_date > (SELECT state_date FROM ans_get_state_date()))))
OR (((SELECT mode FROM ans_get_mode()) = 4) AND (begin_date <= CURRENT_TIMESTAMP) AND ((end_date IS NULL) OR (end_date > CURRENT_TIMESTAMP)))
THEN 1
ELSE 0
END) = 1
GO
set statistics time on
-- cleaning cache
dbcc dropcleanbuffers
dbcc freeproccache
SELECT * FROM v_host
/*
(9777 row(s) affected)
SQL Server Execution Times:
CPU time = 9718 ms, elapsed time = 10834 ms.
*/
View 37 Replies
View Related
May 1, 2007
Hello all!
I've currently got a query that takes two tables with the same structure, and spits out the differences in the second table - this is done to help me see what new records have been added in the second table that don't exist in the first. The only catch is that I have to compare two columns in sort of a pseudo-composite-key. Here's the gist of the table structure:
Table A:
ID (Autonumber),
Firstname,
Lastname,
OS,
PC Manufacturer
Table B (The new one):
Firstname,
Lastname,
OS,
PC Manufacturer
The two columns I have to compare are firstname and lastname, as there aren't any other unique identifiers that match between the two tables. Right now, I'm using some really convoluted WHERE statements, and I suspect that it could be done more cleanly. Does anyone have any suggestions?
Thanks in advance...
View 3 Replies
View Related
Mar 26, 2006
What are the best practices to avoid CPU load in SQL Server?
View 4 Replies
View Related
Jul 2, 2007
We have 4 phases(scenarios), the no of attributes in each scenario is unknown(dynamic)
ex: Scenario1 & case1 = 10,Scenario1 & case2 = 50,Scenario2 & case1 = 70,Scenario1 & case6 = 100
We want to give preference in this sequence...
1. Performace
2. More customization
3. HDD Space
1) For more customization, we need to maintain all the phases' details in same table like id,name,description,textfield. In case of more fields, copy all the values to text fields by seperating with delemeters(,) like: 10,'Name1','Description1','110,abc,xyz,nmo,pqz'. In this approach we can save n number of values. Out of 10 lakh records, 600-900 records will be filtered by id and name in average. Will it cause any performance problem while make the text and extract the individual values? What about searching?
2) For performace, we will have a big table with fixed no of tables say 300(no exendability beyond this & most of the columns will be NULL if values are less).
3) Create seperate tables dynamically for each schenario and case, save the values in curresponding tables.(can be a chance of getting 300-700 tables)
Please suggest me best among all these. Thanks in advance...
Loka
View 4 Replies
View Related
Jan 16, 2008
Below you see a query I run.
The result of this query is shown below...
The result shows fields such as Bid_Price_Best_Latest and Ask_Price_Best_Latest may or may not have null values.
At present, I use a cursor to get the values of these two fields if any of them is null from the table tblPricesSourcesImportHistory for the latest Import_Date on each record.
These two fields are referred to as Bid_Price, Ask_Price in tblPricesSourcesImportHistory.
Now I would like to find out how to do this without the use of a cursor, so that I can speed up this query.
Thanks
select
fp.Security_ID,
fp.Security_Name,
fp.Bid_Price_Best_Latest,
fp.Bid_Size_Best_Latest,
fp.Bid_Source,
fp.Ask_Price_Best_Latest,
fp.Ask_Size_Best_Latest,
fp.Ask_Source
from
tblTEST fp
left join tblSources s on fp.Security_ID = s.Security_ID
inner join tblPricesSourcesImportHistory h on
h.Source_ID = s.Source_ID
where
(fp.Bid_Price_Best_Latest is null) or (fp.Ask_Price_Best_Latest is null)
group by
fp.Security_ID,
fp.Security_Name,
fp.Bid_Price_Best_Latest,
fp.Bid_Size_Best_Latest,
fp.Bid_Source,
fp.Ask_Price_Best_Latest,
fp.Ask_Size_Best_Latest,
fp.Ask_Source
order by
fp.Security_ID
fp.Security_ID Security_Name Bid_Price_Best_Latest Bid_Size_Best_Lates Bid_Source Ask_Price_Best_Latest Ask_Size_Best_Latest Ask_Source
1 Alli 84.0000 0.50 G NULL NULL NULL
2 bow 82.5000 0.50 G NULL NULL NULL
4 xte NULL NULL NULL 90.0000 0.50 G
6 Wqu 84.5000 0.50 I NULL NULL NULL
...
...
...
Not sure if it helps but here are the structures of the tables you may need...
CREATE TABLE [dbo].[tblSources](
[Source_ID] [int] IDENTITY(1,1) NOT NULL,
[Security_ID] [smallint] NOT NULL,
[Source_Parent] [varchar](50) NOT NULL,
[Source_Code] [varchar](20) NOT NULL,
[Source_Security_Name] [varchar](50) NOT NULL)
CREATE TABLE [dbo].[tblPricesSourcesImportHistory](
[Price_Source_Import_History_ID] [int] IDENTITY(1,1) NOT NULL,
[Source_ID] [smallint] NULL,
[Source_Parent] [varchar](255) NULL,
[Source_Code] [varchar](255) NULL,
[Security_Name] [varchar](255) NULL,
[Source_Security_Name] [varchar](255) NULL,
[Bid_Price] [varchar](255) NULL,
[Bid_Size] [varchar](255) NULL,
[Ask_Price] [varchar](255) NULL,
[Ask_Size] [varchar](255) NULL,
[Import_Date] [smalldatetime] NOT NULL)
View 4 Replies
View Related
Jul 13, 2007
Hi,
I'm kind of rusty when it comes stored procedures and I was wondering if someone could help me with the following SP. What I need to do is to query the database with a certain parameter (ex. Culture='fr-FR' and MsgKey='NoUpdateFound') and when that returns no rows I want it to default back to Culture 'en-US' with the same MsgKey='NoUpdateFound'). This way our users will always get an error message back in English if their own language is not defined. Currently I'm doing the following,
- Check row count on the parameter passed to sql and if found again do a select returning the values- else do a select on the SQL with the default parameters.
I'm guessing there is a way to run a SQL and return it if it contains rows otherwise issue another query to return the default.
How can I improve this query. Your help is greatly appreciated. Also, if I do any assignment since 'Message' is defined as text I get the following error message
Server: Msg 279, Level 16, State 3, Line 1 The text, ntext, and image data types are invalid in this subquery or aggregate expression.
CREATE PROCEDURE GetMessageByCulture( @Culture varchar(25), @MsgKey varchar(50) )AS SET NOCOUNT ON;
IF ( (SELECT COUNT(1) FROM ProductUpdateMsg WHERE Culture=@Culture AND MsgKey='NoUpdateFound') > 0) SELECT Message FROM ProductUpdateMsg WHERE Culture=@Culture AND MsgKey=@MsgKey ELSE SELECT Message FROM ProductUpdateMsg WHERE Culture='en-US' AND MsgKey=@MsgKeyGO
Thank youM.
View 2 Replies
View Related
Jul 28, 2005
Hi ,I have this complex stored procedure, that I was happy with but unfortunately the perfoermance fo this SP are very poor.Do you have any idea of how I can optimize it?ThanksWattierSELECT DISTINCT dateend,active,Tparts.idpart,TParts.Namepart, COALESCE( (SELECT SUM(nb) FROM TEChart WHERE TEChart.part = TParts.idpart and qc=0 and DateInsp>=@start and DateInsp<=@end),0) AS review ,COALESCE((SELECT SUM(nb) FROM TEChart WHERE TEChart.part = TParts.idpart and TEchart.qc = @concern and DateInsp>=@start and DateInsp<=@end),0) AS reject, COALESCE ((SELECT - DATEDIFF(day, GETDATE(), MAX(DateInsp)) FROM TEchart WHERE TEchart.QC <> 0 AND TEchart.part = TParts.idPart GROUP BY TEchart.part), (SELECT - DATEDIFF(day, GETDATE(), (SELECT datestart FROM Tproject WHERE idproject= @project)))) AS jourFROM Tparts,TPartQCWHERE Tparts.project=@project and TPartQC.qc=@concern and TPartQC.part=Tparts.idpart order by reject DESC,active DESC,review DESC
View 2 Replies
View Related