Table Filter Operation Optimization
Aug 27, 2007
Table filter operation where I want to filter against any combination from any of the available fields:
table a(f1 int, f2 int)
2 indexes - first by field f1, second by field f2
data - many records like (f1=1, f2=2), and many records like (f1=3, f2=4)
sql: select * from a where f1=1 and f2=4
result will be - no records, but full table scan is required, indexes are not used
question - how to optimize above structure, but do not create compound index by (f1, f2)?
if we will have many fields (lets say 10), which can be used in filter in any combination we will need to create 10! Indexes. Any way to avoid or optimize it for a high volume searches?
View 1 Replies
ADVERTISEMENT
Apr 10, 2008
Hello Everybody,
I have a small tricky problem here...need help of all you experts.
Let me explain in detail. I have three tables
1. Emp Table: Columns-> EMPID and DeptID
2. Dept Table: Columns-> DeptName and DeptID
3. Team table : Columns -> Date, EmpID1, EmpID2, DeptNo.
There is a stored procedure which runs every day, and for "EVERY" deptID that exists in the dept table, selects two employee from emp table and puts them in the team table. Now assuming that there are several thousands of departments in the dept table, the amount of data entered in Team table is tremendous every day.
If I continue to run the stored proc for 1 month, the team table will have lots of rows in it and I have to retain all the records.
The real problem is when I want to retrive data for a employee(empid1 or empid2) from Team table and view the related details like date, deptno and empid1 or empid2 from emp table.
HOw do we optimise the data retrieval and storage for the table Team. I cannot use partitions as I have SQL server 2005 standard edition.
Please help me to optimize the query and data retrieval time from Team table.
Thanks,
Ganesh
View 4 Replies
View Related
Apr 15, 2008
Hello,
Here is my problem:
I use SQL Server 2005. I have approx. 50 tables in my database and 30 of them have a filed named "CompanyID". Example:
create table A (ID int identity, NAME varchar(100), COMPANYID int)create table A (ID int identity, REF_ID int, FIELD1 varchar(100), FIELD2 varchar(100), COMPANYID int)
Also there are nearly 200 stored procedures that read data from these tables. Example:
create procedure ABCasbegin /* some checks and expressions here ... */ select ... from A inner join B on (A.ID = B.REF_ID) where ... /* ... */end;
All my queries in the Stored procedure does not filter the tables by CompanyID, so they process the entire data.
However, now we have a requirement to separate the data for each company. That means that we have to put a filter by CompanyID to each of those 20 tables in each query where the tables appear.
Firstly, I put the CompanyID in the context so now its value is accessible through the context_info() function. Thus I do not need now to pass it as a parameter to the stored procedures.
However, I don't know what is the easiest and fastest way to filter the tables. Example:
I modified the above mentioned procedure in the following way:
create procedure ABCasbegin /* some checks and expressions here ... */
-- gets the CompanyID from the context: DECLARE @CompanyID int; SELECT @CompanyID = CONVERT(float, CONVERT(varchar(128), context_info()))
select ... from A inner join B on (A.ID = B.REF_ID) where ...
and A.COMPANYID = @CompanyID and B.COMPANYID = @CompanyID /* ... */end;
Now I have the desired filter by CompanyID. However, modifying over 200 stored procedures is rather tedious work and I don't think that this is the best approach. Is there any functionality in SQL Server that can provide the possibility to put an automatic filter to the tables.
For example: when I wrote "SELECT * FROM A", the actual statements to be executed would be "SELECT * FROM A WHERE CompanyID = CONVERT(float, CONVERT(varchar(128), context_info()))".
I was looking for something like "INSTEAD OF SELECT" triggers but I didn't manage to find any answer.
I would very grateful is someone suggests a solution for something like "global table filter" (that will help me make an easy refactoring)?
Thanks in advance.
Best regards,
Beroetz
View 5 Replies
View Related
Mar 7, 2014
Why the Indexes on table slow down the DML operation on table, what is the exact reason?
View 5 Replies
View Related
Dec 10, 2001
i have 6 read only tables that every night all the data gets dumped and a new updated copy of the data is copied over. the tables range from 500,000 rows to almost 4 million. i have indexes set up on the fields i use to query against. my questions are
1.since i dump all the data every night and replace it, do i need to rebuild the indexes every night or is that done after the data is reentered?
2.i want to use a fill factor on the table since it is read only, but will dumping the data every night and reinputting it have adverse affects with a fill factor?
3.should i be shrinking the database or defragging it everynight cause of my data dumps and reloads?
thanks in advance
spiral
View 1 Replies
View Related
Jul 26, 2006
Query #1:select <list of fields>from Cjoin B on C.b_key = B.b_keyjoin A on B.a_key = A.a_keywhere A.o_key = <some value>Query #2:select <list of fields>from Cwhere b_key in (select b_keyfrom Bwhere a_key in (select a_keyfrom Awhere o_key = <some value>))#1 (and other things with the same general pattern) are used inliterally thousands of places in this one client's system, and ismuch nicer to write, but seems to be rather slower than #2. Isthere any way to tweak the tables to tell the system somethinglike "hey, B, whenever you're joined to A, you should seriouslyconsider waiting for A to be filtered down to a manageable levelfirst"? And similarly for C/B.MS SQL 2000, SP3, 6.5 compat mode. These are set in stone untilwe upgrade the accounting software (highly non-trivial).
View 1 Replies
View Related
Apr 5, 2006
I am having the following situation - there is a view that aggregates and computes some values and a table that I need the details from so I join them filtering on the primary key of the table. The execution plan shows that the view is executed without any filtering so it returns 140 000 rows which are later filtered by the join operation a hash table match. This hash table match takes 47% of the query cost. I tried selecting the same view but directly giving a where clause without the join it gave a completely different execution plan. Using the second method is in at least 4 folds faster and is going only through Index Seeks and nested loops.
So I tried modifying the query with third version. It gave almost the same execution plan as the version 1 with the join operation.
It seams that by giving the where clause directly the execution plan chosen by the query optimizer is completely different it filters the view and the results from it and returns it at the same time, in contrast to the first version where the view is executed and return and later filtered. Is it possible to change the query some how so that it filters the view before been joined to the table.
Any suggestions will be appreciated greatly
Stoil Pankov
"vHCItemLimitUsed" - this is the view
"tHCContractInsured" - this is the table
"ixHCContractInsuredID" - is the primary key of the table
Here is a simple representation of the effect:
Version 1:
select *
from dbo.vHCItemLimitUsed
inner join tHCContractInsured on
vHCItemLimitUsed.ixHCContractInsuredID = tHCContractInsured.ixHCContractInsuredID
where tHCContractInsured.ixHCContractInsuredID in (9012,9013,9014,9015)
Version 2:
select *
from vHCItemLimitUsed
where ixHCContractInsuredID in (9012,9013,9014,9015)
Version 3:
select *
from dbo.vHCItemLimitUsed
where ixHCContractInsuredID in
(select ixHCContractInsuredID
from tHCContractInsured
where ixHCContractInsuredID in (9012,9013,9014,9015))
View 1 Replies
View Related
Nov 20, 2013
I'm using SQL Server 2012 Analysis services in Tabular mode and connected to Oracle Database and while importing, I'm getting below error after importing some rows.
OLE DB or ODBC error: Accessor is not a parameter accessor.. The current operation was cancelled because another operation in the transaction failed.
View 1 Replies
View Related
May 30, 2006
Microsoft SQL Server 2000 - 8.00.2039
Got this error:
Could not complete cursor operation because the table schema changed after the cursor was declared. SQLCode: 16943 SQLState: HY000
Is this a known issue? I suspect the application logic may cause this error. Please advise.
Thanks a lot!
View 16 Replies
View Related
Oct 22, 2015
I have a job which executes hourly.
Essentially:
Begin
Truncate table A
Insert into A
(Col1,
Col2,
Col3...
)
Select Value1,
Value2,
Value3...
From Table B
End
The insert operation query takes approximately 3.5 minutes to execute. What's occurring is the Table is immediately truncated, and there are no rows in the table for those 3.5 minutes.
How can I avoid having this gap - where there are no rows in the table for that period of time during the job execution ? The table could be locked, but that doesn't seem like the best solution.
View 8 Replies
View Related
Apr 15, 2008
Hello,
Here is my problem:
I use SQL Server 2005. I have approx. 50 tables in my database and 30 of them have a filed named "CompanyID". Example:
create table A (ID int identity, NAME varchar(100), COMPANYID int)
create table A (ID int identity, REF_ID int, FIELD1 varchar(100), FIELD2 varchar(100), COMPANYID int)
Also there are nearly 200 stored procedures that read data from these tables. Example:
create procedure ABC
as
begin
/*
some checks and expressions here
...
*/
select ...
from A inner join
B on (A.ID = B.REF_ID)
where ...
/* ... */
end;
All my queries in the Stored procedure does not filter the tables by CompanyID, so they process the entire data.
However, now we have a requirement to separate the data for each company. That means that we have to put a filter by CompanyID to each of those 20 tables in each query where the tables appear.
Firstly, I put the CompanyID in the context so now its value is accessible through the context_info() function. Thus I do not need now to pass it as a parameter to the stored procedures.
However, I don't know what is the easiest and fastest way to filter the tables. Example:
I modified the above mentioned procedure in the following way:
create procedure ABC
as
begin
/*
some checks and expressions here
...
*/
-- gets the CompanyID from the context:
DECLARE @CompanyID int;
SELECT @CompanyID = CONVERT(float, CONVERT(varchar(128), context_info()))
select ...
from A inner join
B on (A.ID = B.REF_ID)
where ...
and A.COMPANYID = @CompanyID
and B.COMPANYID = @CompanyID
/* ... */
end;
Now I have the desired filter by CompanyID. However, modifying over 200 stored procedures is rather tedious work and I don't think that this is the best approach. Is there any functionality in SQL Server that can provide the possibility to put an automatic filter to the tables.
For example: when I wrote "SELECT * FROM A", the actual statements to be executed would be "SELECT * FROM A WHERE CompanyID = CONVERT(float, CONVERT(varchar(128), context_info()))".
I was looking for something like "INSTEAD OF SELECT" triggers but I didn't manage to find any answer.
I would very grateful is someone suggests a solution for something like "global table filter" (that will help me make an easy refactoring)?
Thanks in advance.
Best regards,
Beroetz
View 3 Replies
View Related
Sep 21, 2006
I have several reports where I am trying to use a single sp, and filter some of the paramater selections in reporting services.
I can get a single item to work in the filters
ie: state.value = SC
but i have not been able to get a multivalue scenario to work.
ie: State.value in SC,GA,TN
or State.value in ('SC','GA','TN')
or State.value in ("SC","GA","TN")
or State.value in 'SC','GA','TN'
or State.value in "SC","GA","TN"
I am guessing it is a simple syntax thing, but well obviously I havent found it.
Any help would be appreciated
Rick
View 5 Replies
View Related
Nov 3, 2006
Hi All,
How do I filter an EXEC table as to put the returned value into a variable?
I have to filter an EXEC table because I am using a table variable to define which table tto select.
Help appreciated.
View 6 Replies
View Related
Jun 7, 2007
I'd like to set the Filters in the Filters tab of the Table Properties dialog to say:
=Fields!WT_TO.Value > 0 OR
=Fields!WT_TO_PREV.Value > 0
but teh And/Or column is permanently disabled, and its sticking in a default value of AND
what's up with that?
View 6 Replies
View Related
Mar 26, 2008
Some tables i have in a report. A couple of them work, but 2 others dont. I would do it in the query, but then i wouldnt be able to group them in a list.
So my question is, how come i can filter out my BillcustomerCode, but when i want to filter out my subbrand_id (which is an int) or subbran_ Descrition(which is a varchar). how would i filter out these types in the table filter.
Also, how would i do an "Or" instead of an "and" in my filter too.
I have one table where i need to have 3 things not in the table:
Bill_Cus_code !=RNPROF
Bill_Cus_code !=RNPROC
Sub_Brand_Id ! =65 (or Sub_Brand_Discription != "Pro Club")
how do i do this in the expression. It seems to take out "RNPROF and RNPROC" but not the third one. Why is this?
View 1 Replies
View Related
Oct 26, 2005
I am using RS 2005. I am setting up a Model for use within Report Builder so our clients can write their own reports.
View 12 Replies
View Related
Jan 13, 2006
Dear all,
My question about SQL statement:
Table1 - Fields: T1PK, T1Field
Records:
1, ABC
2, DEF
3, GHI
Table2 - Fields: T2PK, FK (join T1PK), T2Field
Records:
101, 1, XXX
102, 1, YYY
103, 1, XXX
104, 2, XXX
I want to write a SQL statement that join Table1 and Table2 together, T2Field equal to XXX, and list all records from Table1.
That's the result columns: T1PK, T1Field, T2PK, T2Field
Result records:
1, ABC, 101, XXX
1, ABC, 103, XXX
2, DEF, 104, XXX
3, GHI, NULL, NULL
Is it possible? Please help!!
Joey
View 2 Replies
View Related
Mar 6, 2014
I have a table that has multiple records as illustrated in the simple list below. The real list is much longer.
MachineA 1/1/2008
MachineA 1/3/2008
MachineB 1/7/2008
MachineB 1/8/2009
MachineB 5/5/2010
MachineA 5/7/2011
MachineA 4/2/2013
I need to query to return a result for each unique machine with the latest date. The example result below would be returned because they have the latest date.
MachineA 5/7/2011
MachineB 5/5/2010
Select Distinct would almost do it, but I need each unique machine that has the latest date.
View 9 Replies
View Related
Feb 11, 2008
Hai guys,
In my report I want to use OR condition instead of AND in Table Filter Expression. By default this option is disabled in VS2005 Pro. How do I enable this? or is there any other way to do this? Thanks in advance.
View 7 Replies
View Related
May 20, 2008
We have a fact table where one of the measures in that fact table is invoice balance. Some of our reports requirements list that they only want to see the sum of those invoices where the balance is greater than X dollars. How do I go about filtering calculations based on a measures value?
View 3 Replies
View Related
Oct 6, 2004
I have a big table with several types of transsactions: PO (Puchase Orders, SO,( Sales orders), INV (Invoices) ....
I want to create cubes with only one type of transactions (1 cube for PO,...)
Where and how can I filter the rows I want to use in my cube ?
Thanks
View 2 Replies
View Related
Apr 8, 2004
i am having 3000 records in table. now i want take out the duplicate from that table.
for example: i want to find out the duplicates of 'CompanyNames'.
help needed to write query for this operation.
View 2 Replies
View Related
Apr 21, 2008
H, need help please!
I have two tables on a reports, each having it's own dataset.
I want to set the visibility of the table depending on the filter selection.
So if the user selects: show open then show only the table with the open dataset.
I don't now how to this using the expression in the visibility expression.
Please Assist!
Regards
View 2 Replies
View Related
Mar 15, 2007
I am using VS 2005/SQL 2005 and must use a stored procedure as provided. The stored procedure returns parent rows and child rows (same as parent with additional columns populated, like ChildID). I cannot change the stored procedure.
I have been unable to filter the data (either at the Dataset level or the Table Properties level) to only see the "parent" rows. On my report, I added one column to print the value for my test and a second to use it:
=Trim(CStr(Len(Fields!ChiIdID.Value)))
=IIf(Trim(CStr(Len(Fields!ChildId.Value)) = "0"),"PARENT","CHILD")
Both columns produce the values I expect. However, if I try a filter using the first expression, it fails! I.E. if I test = 0, it filters out ALL rows and if I test != 0, all rows are displayed?????
I am new to SSRS, so I would not be surprised if this is cockpit error. It seemed like a very simple requirement that I have used in othe report tools for years.
Any help would be appreciated.
View 3 Replies
View Related
Jan 26, 2006
Howdy,
I have a table that has a group. In this group, I want to filter by 2 different expressions, concatenated with an OR. BUT I can't change the "And/Or" column value for the first entry because it is grayed out. The column will automatically change to an OR value if both my expression column fields are the same (which I dont want) but if I put any other value in to the expression field of the second row, the "And/Or" field of the first row automatically changes to an AND.
PLEASE! How do I get the And/Or field "ungrayed" so I can change it to what I want?
The 2 filters I and using check the UserID = to the user, and the other is checking a count to get the Top N 1. (So just showing the current user and the top producer)
View 14 Replies
View Related
Jul 13, 2015
I am looking to test this feature - and the "Transaction Performance Collector" has recommended me a table to port to In-Memory OLTP.Β
I have now tried the "Table Memory Optimization Advisor" tool.
After a couple of tweaks to the table design - the tool is now passing validation but the tool is not allowing to progress to the next step:
Could it be down to not having enough memory? But would this not show in the advisor?
View 4 Replies
View Related
Jul 20, 2005
I have a set of udf's dealing that return a one column table of valuesparsed from a comma delimeted string.For example:CREATE FUNCTION [dbo].[udf_filter_patient](@patient_list varchar(2000))RETURNS @patient TABLE(patient_id int)ASBEGINinsert into @patientselect patient_id from patient-- parse @patient_list stuff excludedRETURNENDI have come up with the following two schemes to use these udfs. Theseexamples are obviously simplified, and I have a handful of stored proceduresthat will use between 10 or more of these filters. If the two areequivalent, I prefer Method 2 because it makes for much neater SQL whenusing many filter criteria.So my question is, will one noticebly outperform the other? Or is there abetter way in which to filter on a list of criteria?Method 1 :CREATE PROC sp__filter_open_bills@patient_list varchar(2000)ASCREATE TABLE #patient(patient_id int)INSERT INTO #patientSELECTpatient_idFROMdbo.udf_filter_patient( @patient_list )SELECT*FROMopen_billsINNER JOIN #patient on #patient.patient_id = open_bills.patient_idGOMethod 2 :CREATE PROC sp__filter_open_bills@patient_list varchar(2000)ASSELECT*FROMopen_billsWHEREopen_bills.patient_id IN ( SELECT patient_id FROMdbo.udf_filter_patient( @patient_list ) )GOThanks for the help!Chris G
View 4 Replies
View Related
Nov 12, 2015
how can I have a default filters in my Excel PowerPivot Table, without the user need to select them in Excel creating with DAX Language.
View 8 Replies
View Related
Feb 10, 2015
I have a table:
declare tableName table
(
uniqueid int identity(1,1),
id int,
starttime datetime2(0),
endtime datetime2(0),
parameter int
)
A stored procedure has new set of values for a given id. Sometimes the startime and endtime are the same, in which case I update the value of parameter. Sometimes I add a new time range (insert statement), and sometimes I delete a time range (delete statement).
I had a question on merge, with insert, delete and update and I got that resolved. However I have a different question regarding performance of the merge statement.
If my target table has hundreds of millions of records and I want to delete/update/insert a handful of records, will SQL server scan the entire target table? I can't have:
merge ( select * from tableName where id = 10 ) as target
using ...
and I can't have:
merge tableName as target
using [my query] as source on
source.id = target.id and
source.starttime = target.startime and
source.endtime = target.endtime
where target.id = 10
...
This means I cannot filter the set of rows in the target table to a handful of records where id = 10.
View 1 Replies
View Related
Mar 26, 2008
Hi All,
I am working on SQL server 2005 Reports.
I have one report, one dataset is assigned to it, and one table which displays it.
Now I come accros requirement that, the column value in the filter condition for the table is present in one textbox.
I can not use textbox i.e. reportItems in filter condition. Can someone suggest me how to use textbox value in filters?
I want to display parent/child records on report. I am not getting the proper solution.
The data is like this:
Sequence ItemCode IsParent
1 XYZ 0 'do not have child record
2 PQR 1 'have child records with sequence no 3
3 ASD 0
3 AFDGE 0
3 VDC 0
4 ASR 1 'have child records with sequence no 5
5 ASR 0
If IsParent = 1, that record has child records with sequence = parent sequenece + 1
I think u can understand the data I need to bind, and it is like:
XYZ
+ PQR
ASD
AFDGE
VDC
ASR
On + click we can do show/hide of child records.
I m not getting how to achive this in SQL server report. Can u give some hint?
Thanks in advance
Pravin
View 1 Replies
View Related
Oct 14, 2015
I have a simple pivot table (screenshot below) that has two variables on it: one for entry year and another for 6 month time intervals. I have very simple DAX functions that count rows to determine the population N (denominator), the number of records in the time intervals (numerator) and the simple percent of those two numbers.
The problem that I am having is that the function for the population N is not overriding the time interval on the pivot table when I use an ALL function to do so. I use ALL in other very simple pivot tables to do the same thing and it works fine.
The formula for all three are below, but the one that is the issue is the population N formula. Why ALL would not work, any other way to override the time period variable on the pivot table.
Population N (denominator):
=CALCULATE(COUNTROWS(analyticJudConsist),ALL(analyticJudConsist[CurrentTimeInCare1]))
Records in time interval (numerator):
=COUNTROWS(analyticJudConsist)
Percent:
=[countrows]/[denominatorCare]
View 13 Replies
View Related
Jun 6, 2007
Dear Advance,I used one stored procedure to retrive 3 different result set. and in the codebehind i seperate it. means from the dataset i seperate three different datatable and then show my data as my need.but the main problem is ... after retriving the datafrom the database i have to user foreach loop to bind the coulmns data to my different custom class.example: foreach (DataRow oDrow in MyDataTable.Rows) {oClass=new Class();oClass.Name1=oDrow["Name1"] .toString();oClass.Name2=oDrow["Name2"] .toString();.... } 1. so my first question is there any optimization possible ?2. my result set is too loong ... so should keep just one hit to database or hit more than one time Currently i am optimizing my web application. in the previous version 1 have to hit the database 3/4 times for different purposes. but now it hits only one time... but it takes time in the codebehind to perform different operation.Any Suggestion
View 1 Replies
View Related
Dec 11, 2001
I have a SP that calls about 10 stored procedures sequentially. The 10 SP's are basically complex update statements, each one individual. Is there any way to optimize this?
I know putting the 10 into 1 SP would make it compile faster but thats about it. Are there any execution tricks of Stored Procedures firing off sequentially?..or anything I should know?
-thanks
Bartman
View 2 Replies
View Related
