Setting A Minimum Support Threshold For Association Rule Mining
Jul 11, 2006What is the best practice in setting a minimum support threshold for market basket analysis? Is there a formula? Does it depend on ROI you predict?
View 4 Replies
ADVERTISEMENT
PredictProbability With Association Rule Model..
Nov 20, 2006I have run into a .. somewhat of a "duh" question. I'm running association rule to run a basket analysis, and I'm trying to get probability of each prediction. I know this is wrong, but how do I go about running PredictProbability on each ProductPurchase prediction?
When I run the below DMX query, I get this error message...
Error (Data mining): the dot expression is not allowed in the context at line 5, column 25. Use sub-SELECT instead.
Thanks in advance...
-Young K
SELECT
t.[AgeGroupName]
, t.[ChildrenStatusName]
, (Predict([Basket Analysis AR].[Training Product], 3)) as [ProductPurchases]
, (PredictProbability([Basket Analysis AR].[Training Product].[ProductName])) as [ProductPurchases]
From
[Basket Analysis AR]
PREDICTION JOIN
OPENQUERY([DM Reports DM],
'SELECT
[AgeGroupName]
, [ChildrenStatusName]
FROM
[dbo].[DM.BasketAnalysis.Contact]
WHERE isTrainingData = 0
') AS t
ON
[Basket Analysis AR].[Age Group Name] = t.[AgeGroupName]
AND [Basket Analysis AR].[Children Status Name] = t.[ChildrenStatusName]
Cases Supporting A Particular Association Rule
Jun 18, 2007I haven't been able to find a DMX query which will spit out the cases which support a particular association rule. I was hoping it would work sort of like drillthrough but show only the cases supporting a particular rule. Am I missing something?
What I ended up doing was extracting the itemsets of the rule from the model's content then running a SQL query to retrieve the cases that contain both the left-hand and right-hand itemset of the rule. I'm hoping there's a better way.
Association Algorithm - Importance Of A Rule
Mar 6, 2006Can anyone tell me, how the Business Ã?ntelligence Studio calculates the importance of a rule. I can't find the formula. I know some formulas, but the result in SQL Server is completly different.
Thanks!
Horizontal Variety Seeking Association Rule Problem
Jul 13, 2006I read somewhere that market basket analysis finds rules with substitutes as likely as rules with complements due to a consumer behavior called "horizontal variety seeking". This is when customers buy more than one product in the same category even though they are subsitutes. For example, when people go to the grocery store and buy soda, they buy coke and sprite at the same time even though they are substitutes of each other. I was wondering if anyone has experience with this anomaly and how they solved it. I found a time series model called the vector autoregressive model which is used to find the elasticity of prices over a time period. Does anyone have experience working with the VAR model? I am having trouble figuring out what some of the variables in the model are.
Below is the paper
http://www.feb.ugent.be/fac/research/WP/Papers/wp_04_262.pdf#search='VAR%20model%20market%20basket%20analysis'
Mining Association Rules Using SQL
Nov 26, 2006Hello everybody,
I am interested in mining association rules field
I have found some posted statements but I didn't undertand them. I need your help badly.
Just tell from where should I start?
Thanks alot
Differences Between Mined Data And Queried Data : Association Rule
Sep 22, 2006Dear all,
I have a table containing call records, and made a mining model from that table only. The model has 3 columns : calling_number, called_number, and target_operator, using Association Rule algorithm. The key is calling_number, input was operator, and predicted column called_number.
The result shows no rule, but there are results with item-set size of 1 (column) and 2 (column). On the top record of the result, SQL Server says there are 1891 support for called_number = 1891 and operator = 'INDOSAT'.
I queried the table with this query
SELECT DISTINCT calling_number
FROM call_records
WHERE called_number = '07786000815'
AND target_operator = 'INDOSAT';
It returns 2162 records instead of 1891. If I removed the DISTINCT qualifier, SQL Server returns 2159 records. Why is this differences with the result of mining?
Thank you,
Bernaridho
Microsoft Association Mining Algorithem + ASP.NET 2.0
Aug 19, 2006Hi
I am working on an academic medical project. I have created PATIENT table
PATIENT_ID
NAME |
CITY
DISEASE table
DISEASE_ID
NAME |
and Relationship table[FOREIGN KEYS]PATIENT_DISEASE
PATIENT_ID
DISEASE_ID
I am using Microsoft association algorithm [SQL Server 2005 BI Studio] to find association between PATIENT CITY --> Associated DISEASE. I will be entering dummy data into these tables as this is academic project. I like to know can i be able to find the associated Disease/s with the PATIENT City with this algorithm ? such a way as soon as the user enters PATIENT City, assocaited Disease will be selected from the diseases table? on web interface [asp.net].
i like to know after building this association model, can i use it on my web page to prompt the user associated disease with patient city? or building this model only gives association rules, i need to write a procedure or t-sql statements to implement the association rules?
I am using SQLSERVERDATAMINING.COM tutorials to build the model.
Your information and help is highly appreciated.
regards
Basket Analysis && Association Mining
Jul 6, 2007I'm looking for suggestions on the right design approach in relation to a problem that resembles Basket analysis. The data to be analyzed is a dimension Attribute_DIM and contains an ID, Attribute and Attribute_Value. Some examples of the data are :
ID Attribute Attribute_Value
1 Color Black
1 Movie Men in Black
1 Book Of Human Bondage
2 Color White
2 Movie Men in Black
2 Book Grapes of Wrath
We need to be able to analyze multiple selections of the dimension. For example,
Men In Black
Grapes Of Wrath Of Human Bondage
Men In Black Black 1 1
White 1 0
I have had some success using the Association Algorithm Mining Model. I think It is an overkill since I only need descriptive and no predictive analysis.
I'm looking for some ideas on the right approach to this problem. Ideally, we need to present the data in a cube and have the possibility to perform member analysis of the dimension.
I have looked at several articles (including http://msdn2.microsoft.com/en-us/library/aa902637(sql.80).aspx and http://www.aspnetpro.net/newsletterarticle/2004/10/asp200410ri_l/asp200410ri_l.asp). I'm not convinced those are the solutions and would appreciate any insight into this problem.
Thank you,
Anna.
Association Mining Model Predictions
May 17, 2007Hi,
I've been playing around with the association mining model in SQL server 2005 and built a market-basket analysis of my data that I'm pretty happy with. The next task for me is figuring out how to run DMX queries against the data that I've just mined, so we may possibly use it in a web based application. This wouldn't necessarily be a difficult problem (and still may not be), but every example I've seen for the Mining Model Prediction Designer uses relational databases and I built my mining model off OLAP. Therefore, my predictable attribute is nested and when relating the mining model structure to the relational database that the cube was built off always gives me an error:
"Errors in the high-level relational engine. The 'CompanyName' column could not be found in the top-level clause of the SHAPE statement."
What I would like to do, and I'm not really even sure how I should structure any of my queries, is feed the model a product and have it return a listing of all the products it predicts. Currently, I've only been able to get the designer mode to process a singleton query, and even that didn't return any useful data. I know that this probably can be done pretty easily so any advice you may be able to offer would be greatly appreciated!!
So you may better understand my question, my association mining structure hierarchy looks as this..
[Model] ProductRecommend
[Case][Key]CustomerList
[Case][Attribute]CompanyName
[NestedTable]Product
[Nested][Key]PRODUCTCLASSID
[Nested][Attribute]PRODUCT
With that in mind, I'm trying to perform a query simliar to this:
SELECT
PredictProbability([ProductRecommend].[Product].[PRODUCTCLASSID]), <---- Throws Error for PredictProbability syntax no matter what I try to get to [PRODUCTCLASSID]
(SELECT [PRODUCT] FROM [ProductRecommend].[Product])
From
[ProductRecommend]
NATURAL PREDICTION JOIN
(SELECT 'test' AS [COMPANYNAME],
(SELECT '1234' AS [PRODUCTCLASSID],
'ProductA' AS [PRODUCT]) AS [Product]) AS t
Thanks again for any help!
Association Rules Algorithm [support,confidence]
Dec 16, 2006In assotiation rules each rule has a [support, confidence] part. In Microsoft Association Rules there is a [probability,importance] measure in each rule and importance can be greater that 1.
I found the following in msdn but i'm not sure if i understood correctly.
MINIMUM_PROBABILITY: Specifies the minimum probability that a rule is true. For example, setting this value to 0.5 specifies that no rule with less than fifty percent probability is generated.
The default is 0.4.
MAXIMUM_SUPPORT: Specifies the maximum number of cases in which an itemset can have support. If this value is less than 1, the value represents a percentage of the total cases. Values greater than 1 represent the absolute number of cases that can contain the itemset.
The default is 1.
My questions are
1) Can i explain the [probability,importance] in [support,confidence]? If yes, how?
2) What importance>1 means?
Thank you in advance.
Plug-in Algorithm In Data Mining Using Sql Server 2005-- Modification For Association , Classification??
Sep 7, 2006managed plug-in framework that's available for download here: http://www.microsoft.com/downloads/details.aspx?familyid=DF0BA5AA-B4BD-4705-AA0A-B477BA72A9CB&displaylang=en#DMAPI.
This package includes the source code for a sample plug-in algorithm written in C#.
in this source code all .cs files are modified for clustering algorithm
if my plugin algorithm is of association or classification type then what modifications are requried in source code???
Data Mining :: Implementing Excel Data Mining In A Classroom Setting?
Jun 15, 2015Implementing data mining Add-in in an academic setting? We need to handle over 150 new students a semester and have their connection to Analysis Services survive for their four years at the college. We are introducing data mining to every freshman business student as a unit within their Intro to Excel class (close to a month of work to give them a sense of what is possible). Other courses later in their curriculum will expand on that introduction.Â
Once implemented, we would have as many as 900 connections to manage (four years from now). It is possible that multiple sections will be running at the same time, so 40 students may be accessing the data mining tools concurrently. Â
Is there a way to "bulk establish" the access credentials and establish those databases?
Where Can I Store The Mining Results From Mining Models In SQL Server 2005 Data Mining Engine?
Apr 26, 2006Hi, all here,
I am wondering where can I store my mining results in data mining engine? For example, I got mining results like accuracy chart, decision trees, and other formats of results based on different mining algorithms I used for my data mining, so where can I actually store the results for reporting service use later? Is it possible to do that in SQL Server 2005?
Thanks a lot for any help and guidance in advance.
Threshold
Nov 3, 1998is there any threshold manager in the ms sql server? like in the
transaction log, where you can add a stored proc that
will dump the tran log everytime a threshold is hit.
thnks.
Le Threshold Maximum
Feb 9, 1999Hi,
I have a table with 188376 rows and the data size = 3012 KB, index size = 5884 KB .
LE threshold max is set to 2000 and LE threshol percent to 20%
I have an index on that table and observed that it is not getting used.
I would like to know whether sql optimizer uses the index based on the cost
of the query plan or does the table scan once the LE thresholdlimit is reached
overriding the optimized plan.
thanks,
MMS
INNER JOIN/Index Threshold?
Jan 4, 2007Guys I am really stuck on this one. Any help or suggestions would beappreciated.We have a large table which seemed to just hit some kind of threshold.They query is somewhat responsive when there are NO indexes on thetable. However, when we index email the query takes forever.FACTS- The problem is very "data specific". I can not recreate theproblem using different data.- There is only a problem when I index email on the base table.- The problem goes away when I add "AND b.email IS NOT NULL" to theinner join condition. It does not help when I add the logic to the"WHERE" clause.DDLCREATE TABLE base (bk char(25), email varchar(100))create clustered index icx on base(bk)create index ix_email on base(email)CREATE TABLE filter (bk char(25), email varchar(100))create clustered index icx on filter (bk)create index ix_email on filter (email)QuerySELECT b.bk, b.emailFROM base b WITH(NOLOCK)INNER JOIN filter f ON f.email = b.email--and f.email is not nullData Profile--35120500, 35120491, 14221553SELECT COUNT(*) ,COUNT(DISTINCT bk), COUNT(DISTINCT email)FROM base--16796199, 16796192, 14221553SELECT COUNT(*) ,COUNT(DISTINCT bk), COUNT(DISTINCT email)FROM baseWHERE email IS NOT NULL--250552, 250552, 250205SELECT COUNT(*) ,COUNT(DISTINCT bk), COUNT(DISTINCT email)FROM filter--250208, 250208, 250205SELECT COUNT(*) ,COUNT(DISTINCT bk), COUNT(DISTINCT email)FROM filterWHERE email IS NOT NULL
View 18 Replies View RelatedSupport For SQL Express2005 My Host Provider Does Not Provide Support?
Dec 5, 2005Reader Community
I've just started hosting my newly created Microsoft Visual Web Developer 2005 Express Edition web site. Unfortunately the Login group membership functions will not function correctly. Having contacted the web service hosting provider, They replied: "We do not support SQL express2005. The only way to use the extra functions of ASP.NET2 such as group membership is if it is using an SQL 2000 database to connect to. "
Is it possible to design web sites with Microsoft Visual Web Developer 2005 Express Edition that store membership details on an SQL 2000 database?
I've just paid £88 approx. $140 for a years subscription, have I chosen the wrong web service hosting provider?
Should I have designed the web site with a better web site design software tool that also makes designing membership login functionality easy, just as Microsoft Visual Web developer 2005 express edition?
Look forward to all comments?
Regards
Philip
How Is The 'Score' Value Derived In The Lift Chart/Mining Legend For Data Mining Models?
Sep 26, 2006Hi,
I have just run a simple data set through a model to predict a simple true or false value (i.e. binary output)
The Lift Chart/Mining Legend in Analysis Services shows three results €“ Score, Population Correct (%), and Predict Probability (%)
Population Correct I beleive is the percentage of predictions it got right out of the total number of predictions it tried to make. Is this correct?
However, I can€™t work out how the other two are derived in particular the 'SCORE'. To give a live example the scores were as follows:
Model Score Pop Correct Pred Probability
Decision Trees 0.83 76.59% 54.28%
Neural Network 0.75 67.63% 50.05%
Ideal Model 100.00%
Can anyone help with this and give a detailed explanation?
Many thanks,
S Rajput
Excel 2007 Data Mining Add-in Advance Create Mining Model Question
Apr 11, 2007Hi,
I am trying to model data in analysis services with the Advance Create Mining Model function in the excel addin. I am having trouble creating an association model that works like the Associate button above the Advanced button.
The format of my data is like this
OrderID Product
100 Bike
100 Helmet
100 Shoes
200 Helmet
200 basketball
200 Bat
300 Shoes
300 Socks
The associate button works perfectly since it asks me which column is the transaction id (orderid) and which column I am trying to predict (product). The advanced create mining model asks me to determine what the columns are...
OrderID=key Product=Input+Predict?
When I run the advance create mining model associate, I get a browser that gives me no rules and the support for only one item itemset (each product but no combination of products).
Does anyone know what I have to do to get it to work like the associate button?
Locating Set Of Points Close To One Another (within A Threshold)
May 6, 2006Hi allI have a large data set of points situated in 3d space. I have a simpleprimary key and an x, y and z value.What I would like is an efficient method for finding the group ofpoints within a threshold.So far I have tested the following however it is very slow.---------------select *from locations a full outer join locations bon a.ID < b.ID and a.X-b.X<2 and a.Y-b.Y<2 and a.Z-b.Z<2where a.ID is not null and b.ID is not null---------------If anyone knows of a more efficient method to arrive at this results itwould be most appreciated.Thanks in advanceBevan
View 8 Replies View RelatedAnalysis :: Threshold File For SSAS - PAL
Aug 7, 2015I am doing workload analysis on SSAS - Tabular (2012), I have perfmon logs captured and want to run through PAL. I am looking out for threshold file for SSAS tabular 2012/2014.Â
View 2 Replies View RelatedCreate Mining Structure And Mining Model With Code
Oct 18, 2006Dear friends,
I encounter a serious problem.
I would like to develop an application that can create Data Mining structures and a mining model in SQL Server 2005 with VB.NET. I tried the code from book Data Mining with SQL server 2005 in chapter 14 but did not work. Any good idea?
Please help me.
Best regards,
Manolis
Create Mining Structure And Mining Model With Code
Oct 20, 2006
Thank you very much for your help.
The errors that I can see in the code that you gave in your answer are the following and they are more or less the same as I had previously
I tried the code but initially I have encounter the following problems.
1. In any line that have the declaration As Server, As Database like in
Public Function CreateDatabase(ByVal srv As Server, ByVal databaseName As String) As Database gives me the problem that type Database is not declared the same type Server is not declared and it does not give me any option.
2. In addition to that for As DataSource, As RelationalDataSource, As RelationalDataSourceView, As ScalarMiningStructureColumn, As DataSourceViewBinding, gives me the problem that type is not declared.
3. Finally in mc = New MiningModelColumn("Yearly income", Utils.GetSyntacticallyValidID("Yearly income", Type.GetType(MiningModelColumn))) is not accesible in this context because it is 'Private'.
I have some more problems but I thing that by solving the above that I referred I will solve the rest.
Thank you any way.
Best regards,
Manolis
PhD student
Can I Filter The Data On Mining Structure, Mining Model?
Jul 18, 2006I perform data mining on all products and a specific product category.
Do I need to create 2 data source views, one for all products and the other one for the specific product category?
Afterward, I run the Data Mining Wizard 2 times to create 2 mining structures.
I also need to add the same mining model (e.g. Bayes, Cluster) to each of these mining structures.
Is there any simple way to do it?
Thanks.
Joe.
Mining Structure Has To Contain At Least 2 Mining Models To See Their Accuracy Charts?
Nov 27, 2006Hi ,all here,
Thank you very much for your kind attention.
I just found that I am not able to view the accuracy chart for my mining model. The error message is: no mining models are selected for comparision. Which is quite strange.
Any guidance? thank you very much.
With best regards,
Yours sincerely,
Show Values For Temperatures Not Equal To Or Above Threshold?
Feb 10, 2014The below stored procedure is used to create a vertical benchmark line on the X-Axis which has a hour scale. I use the stored procedure to find out which temperature crosses or equals the threshold temperature (340), then plot the vertical benchmark line at the hour the first temperature is equal to or greater than 340 degrees and less than 1000 degrees.
The logic below works if the temperature is equal to or greater than 340 degrees and less than 1000 degrees. THE ISSUE is I have 8 temperatures if they don't cross the threshold of 340 degrees I need to set a default value for my vertical line. In other words if the temperature is 180 and my threshold is 340 then set my vertical line on the highest temperature close to 340.
I tried removing my Where clause (but then it breaks the logic for those temperatures that are equal to or greater than 340). I tried using Case When but this didn't give me what I want either. I tried UNION as well. All giving me results I don't want.
Here is what I am looking for:
This first example is one where there was a temperature that was equal to or greater than the threshold of 340 degrees. This is CORRECT
Code:
first_to_cross_thresholdAgeSampleDateDiffAgeovenStartTimestampAgeovenCompleteTimestamp
2014-02-04 19:24:003.3500002014-02-04 16:02:13.0002014-02-05 05:02:13.000
If 8 temperatures did not equal or cross the threshold then give me the hour of the highest temperature close to the threshold but do not return 0.
For Example:
temp1 92
temp2 108
temp3 0
temp4 284 <<< this is the closest to the threshold so give me the hour when this occurred.
temp5 2192 *Remember I can only count temperatures less than 1000 degrees. Anything above 1000 degrees mean there is nothing in the oven. So it is false/positive.
temp6 102
temp7 0
temp8 12
Code:
first_to_cross_thresholdAgeSampleDateDiffAgeovenStartTimestampAgeovenCompleteTimestamp
2014-02-05 00:30:001.3000002014-02-05 02:00:13.0002014-02-05 02:00:13.000
Code:
CREATE PROCEDURE [dbo].[AgeScoreCardThreshold_JJ_12232013]
-- Add the parameters for the stored procedure here
@LicenseNumber int = NULL,
@Lot varchar(50) = NULL
[code].....
Creating An Alert For Blocked Process Threshold
Jul 18, 2007According to BOL you can configure an Alert to notify you when the blocked process threshold has been exceeded:
SQL Server 2005 Books Online
blocked process threshold Option
Use the blocked process threshold option to specify the threshold, in seconds, at which blocked process reports are generated. The threshold can be set from 0 to 86,400. By default, no blocked process reports are produced. This event is not generated for system tasks or for tasks that are waiting on resources that do not generate detectable deadlocks. For more information about deadlock detection, see Detecting and Ending Deadlocks.
You can define an alert to be executed when this event is generated. So for example, you can choose to page the administrator to take appropriate action to handle the blocking situation.
Can someone provide some direction on exactly how this is done? Does it require a Service Broker and queue?
SQL Server 2008 :: MAXDOP And Cost Threshold For Parallelism Settings?
Jul 2, 2015Referencing an article regarding MAXDOP and cost threshold for parallelism from Brent Ozar's website: [URL] .....
We have a 2 physical CPUs that are 4 cores each with hyper threading enabled. When looking through the task manager, under the performance tab, I see 16 CPU threads.We have set the MAXDOP value is set at 4.
Reading further, cost threshold for parallelism setting is recommended at 50 to start with.
Our setting is at the default 5.
Find Instances When Sum Of Three Claims Types Greater Than Predetermined Threshold
Jan 30, 2015I have a table that is made up of the sum of medical, mental health and pharmacy claims. I would like to query that to find instances when the sum of the three claims types are greater than a predetermined threshold.
For example:
Patient 1 Medical = 10,000 (could be 10 records at 1,000 each)
Patient 1 Mental Health = 5,000
Patient 1 Pharmacy = 15,000
Patient 2 Medical = 1,000
Patient 2 Mental Health = 0
Patient 2 Pharmacy = 500
Threshold is 25,000
If I queried the above sample table I would get one record:
Patient 1 30,000 - because 10,000+5,000+15,000 = 30,000 and is greater than the threshold.
I am not sure that a having clause would work though.
Setup And Upgrade :: Creating Alert Which Notifies If Logspace Reaches A Threshold Percentage?
Aug 20, 2015I want to configure an alert which notifies me if any of my database log files size exceeds a certain threshold percentage.
View 6 Replies View RelatedNested CASE WHEN - Create A Field To Show If Measure Hits Threshold Or Not To Be Later Picked Up In SSRS
Oct 28, 2015I am trying to script a case when to achieve the following.
I have a table of measures with certain threshold. The threshold direction can either be > or < so I want to create a field that shows if the measure hits that threshold or not to be later picked up in SSRS. So a nested case when?
CASE WHEN M.[Threshold Direction] = '>' THEN
CASE WHEN A.[Value] > M.[Threshold] THEN 'GREEN'
CASE WHEN A.[Value] < M.[Threshold] THEN 'RED'
ELSE ''
END END END AS 'Condition'Is this at all possible?
How Get The Lift Chart In Datamining.i Am Not Geeting Mining Accuracy Chart And Mining Model Prediction
Sep 14, 2007
Hi,
I am not getting Mining Accuracy Chart and Min ing Model Prediction
Plz tel me how to do.And how to use the filter input data used to generate the lift chart and
select predictable mining model columns to show in the lift chart