It's often said or done that when inserting or updating into a 'large' table that disabling the non-clustered indexes can is needed for performance.
Now I know the obvious way to find out if this is best or not is by testing the different options. I was wondering if there was a rule of thumb to this?
Say you have a table with half a billion rows and 4 non-clustered indexes and are only updating half a million rows then sometimes disabling every night and re-enabling can take way more time than the actual update. Haven't found an articles advising to disable them when a table is over X rows and you are updating Y% of them...
This is a general question about data modeling. I'm more curious than anything else.
There is much talk about over-training data model, and I'm sure there are under training as well. As a rule of thumb, depending on the algorithm, what is a good ratio of attributes vs data points?
I've been fixing some issues lately where weekly maintenance has been causing logs to grow and filling disks.
Is there any rule of thumb for allocating log space for doing reorgs and rebuilds in a worst case scenario? I'm thinking 3x the largest database size?
I've been watching them run on databases in the range of 50GB where the logs are growing well over that for rebuilds or even reorgs. Once you have a few databases like this on a server, you can suddenly eat through a lot of disk space just for holding logs during maintenance.
Using SQL Server 2008, we would like propose mirroring between two servers of a critical database. Since we initiate, may require to clarify on its purpose and also required changes from application end.Any changes required from OS Level? (I believe both servers IP or Host name should be added in host entries. Mirroring ports should be allowed/open including Principal and mirror server IP Addresses): Windows Team.Any changes required from Application? (Instance name, authentication: user name and its password should be added in web config files): Application Team.Any changes required from Network Team?Also for mirroring both the principal and mirror servers should be with same version, does it only mean SQL Server 2008 versions are enough or does it also mean to say build numbers 10.00.4000 should also be same.URL....
I need to set up asyncronous data replication across two clustered instances of SQL 2012 across 2 Datacenters. Both the datacenters have a common domain however the vlans are different. There are only 3 small databases on the primary instance.
any issue in setting up mirroring in this case as vlans are different.
Please forgive my ignorance, however I am 2 months new to SQL-Server 2000.
Is there a way to create a rule specific to a user such that he can only SELECT from a table where a field equals a specific value.
In other words user Joe types in: SELECT * FROM SALES_BY_LOCATION. Because Joe's territory is Los Angeles, I want a rule that only retrieves sales data from Los Angeles.
In My table i'm collecting information about our customers.In that i have fields zip and phone.So i wanted to implemement a rule that zip should be atlaest of of 5 characters and phone should be 13 characters Including '-'s(333-333-3333).How can i implement these two rules on my table.
Not long ago I accountered this situation: I had two databases on "MS SQL Server". In one of the databases there was a nomenclature with very large primary key. I had to transport that nomenclature and transform the wide PK into single identity column into the other database.
I decided to use a function for that transformation. BUT that function had to mark somewhere which combination of the PK columns is relative to which identity value. BUT functions CAN'T WRITE under MSSQL. So I took the challenge and mine all sources of information. The result was a function "Exec4Fun" that breaks the rule. I suppose that with this function it's possible to avoid the restriction for triggers, which prevents writing in the triggering table? (not tested yet)
If someone needs such tools, just write back your e-mail and I'll send some code.
When i use the MS association rules ,i don't know how it is worked on the background .I stuy the Fp-Growth algorithm , but there're some questions , I don't kown what's the meaning of transcation database. who can give me one example ? thanks .I know we can store the data in relation database,but in basket Analysis ,how a transaction stroed in relation database?
In my package i have column called "optinout" coming from source file which has the value "start" and "stop", now in my destination table i have a column called "contact permission code" to which i need to apply a business rule based on the value in column "optinout" the rule is as stated "Contact Permission Code" is OptOut if "OptInOut" value is "STOP" else OptIn.
Hi there, it has been a long i'm trying to execute Microsoft Association Rules on my database.
I solved memory leak problem now, but i still can't understand output rules.
Database contain all the italian student who took a degree last year. Here in Italy, they have to compile a summary where they speak about universitary experience. ie: they talk about experience with teachers (pointage from 1 to 5); they says if they want to continue in the universitary field or not, and so on.
Most of the rules, says: Int_Stud=1-2, RapDoc>4
Int_Stud is the column where i store student intention to continue university. 1 means they want to go on, 2 means they do not want to continue to study. So, this rules has no sense, because it relates all the student (in my mind): the one who wants to continue university and the one who do not want to.
I think problem is that visual studio 2005 and analysis service has no understanding of Int_Stud world, they've no idea that Int_Stud can have just 2 values and that they're opposite each other. Is there a solution to this problem? Can i discretize this column?
Even if I know not to have perfect english, I hope to be understandable
I used the MS clustering method to run on my sample data, and I would like to apply the rules built to score/predict on another data set. But I am not able to find out where the rules are. I am wondering whether there are sql codes available that can be directly applied on the new data set. cheers
I note that there exist three web viewers for data mining algorithms, namely, DMNaiveBayesViewer, DMDecisionTreeViewer and DMClusterViewer. How come there are no viewers for association rules (itemsets, rules, dependency network)? Can you suggest any alternative way of showing such valuable information in a web application?
I understand Mr. MacLennan's explanation provided at http://forums.microsoft.com/MSDN/ShowPost.aspx?PostID=282651&SiteID=1 and appreciate the time he took to explain how importance works. However, like the user with username "sang", I also ran the data in BI 2005 and got the same results listed by the aforementioned user. I did this using the following data:
donut muffin
y y
y y
y y
y y
y y
y y
y y
y y
y y
y y
y y
y y
y y
y y
y y
n y
n y
n y
n y
n y
etc.
The rule muffin -> donut has an importance of -0.105302438, which is not the same as Mr. MacLennan's results. I tried switching the roles of a and b in a -> b and using different bases on the logarithms. I don't get the result of -0.105302438 with any of these. I also tried to calculate importance with a small data set I have and can't get the results using Mr. MacLennan's explanation with that data set either. Any thoughts on the descrepancy?
I am doing the Market basket analysis for a retailer using association rule. The whole data set is huge which contains grocery, clothes and books etc. If I want to check out the relationship between several different clothes brands, (e.g. LEVI'S and adidas), should I just remove all the grocery and books transactions, use the subset which only contains clothes transactions to re-run the association rules? Is this gonna work?
I got a question about the data preparation of market basket analysis.
There are always some transactions with only one single SKU product. It seems that these kind of transactions have nothing to do with association. Shall I just exclude them or what?
I want to score my data by only the assoziation rules I filtered in Mining-Model-Viewer.
Is this possible?
I recognized that MiningModel Predictionquery uses the generated Model (all rules).
Is there any way to influence the model at generating time.
Thanks a lot.
Hi,
I specified the point to solve my problem but I did not understand why MS SQL Server didn't recognize the Association parameters I but in. It uses the Minimum_Probability I put in, but the Minimum_Importance i wrote didn't care it, it use the default value.
I want to paste a screenshot here but it isn't possible. By the way I'm using the evaluation version, which should be same as the enterprise edition.
I need to create a set of cases for a project that uses the Microsoft Association Rules algorithm to make recommendations for products to customers. My question is: the set of scenarios must include all transactions of customers for training?. or is it sufficient some percentage of total transactions? If i do not use all transactions of customers, could be that the algorithm does not consider some products in their groups or rules and could not make recommendations about these?
I need to implement one logic similar to rule engine. Below is the example.how to execute all queries in second table order?? what is the best way to implements this
declare @tblRules AS Table (RuleNo INT, RuleDesc NVARCHAR(500), RuleQuery NVARCHAR(MAX), QueryExecutionInterval NVARCHAR(50)) declare @tblRuleResults AS Table (RuleResultID INT, RuleNo INT, ExecuteTime DateTime, NextExecutionTime DateTime, Result NVARCHAR(10)) INSERT INTO @tblRules VALUES ('1','Fail - 2 times within 1 Hour','XXX','Every 15 Minutes') INSERT INTO @tblRules VALUES ('2','Fail- 2 times within 2 Hour','YYY','Every 30 Minutes')
The problem is that I can't seem to reduce the minimum probability below .42 to view more rules.
I've considered that it might be becuase these are the only rules discovered, however I know quite a bit about the data and I would excpect many more associations.
I'm new to analysis services and hopefully this is a quick & easy question. I have a couple of quite large (163,000 tuple) tables with columns essentially representing a bit vector. I would like to mine for association rules but the number of '1' values are very, very sparse and they are the only objects of interest. How can I get more control over the algorithm---that is, how can I stipulate that the state of the column must be '1' to be considered? Any help or direction to the proper documentation would be great.
I'm building a mining model wiht MS Association Rules. After processing this model, the result includes some rules(example):
E = Existing, C = Existing -> B = Existing F = Existing -> E = Existing C = Existing, B = Existing -> E = Existing F = Existing -> B = Existing B = Existing, A = Existing -> C = Existing F = Existing, B = Existing -> E = Existing F = Existing, E = Existing -> B = Existing D = Existing -> A = Existing C = Existing -> A = Existing E = Existing, A = Existing -> B = Existing
I want to buid a query that has two or more items on the left of the rules, example: E = Existing, C = Existing -> B = Existing ->I want to buid a query to predict that: when a customer buy 'E' and 'C' then he likely buys 'B'
I read the paper of sequence clustering. It seems that the major application of the algorithm is for the web site. I was just thinking that can I apply this algorithm on the purchase sequences of credit card data?
If so,please also tell me the difference between sequence clustering and association rules on credit card data application. Although I realize that sequence clustering is a fully probabilistic model and it has the capability of prediction, association rules also give the probabilities of purchasing the other products.
I would really appreciate if you could help me out. I am trying to create a taxonomy to be taken into account into the association rules algorithm. For example, if my data is a group of purchases from a supermarket I could have one client who bought milk, cookies and shampoo, and another who bough cheese, cookies and soap.
I would like to specify that milk and cheese belong to the category "dairy" and shampoo and soap belong to "personal hygene". If there are interesting rules regarding the categories I would like them to be taken into account. Additionally, I would not like to have rules like "milk -> dairy". If one specific object appers in a rule, its corresponding category should not. In this scenario I could have milk and "personal hygene" in the same rule, but not shampoo and "personal hygene".
I have seen this done by other mining tools but I've been having trouble finding a way to make this possible in Analysis Services.