Can anyone tell me, how the Business Ã?ntelligence Studio calculates the importance of a rule. I can't find the formula. I know some formulas, but the result in SQL Server is completly different.
Thanks!
I understand Mr. MacLennan's explanation provided at http://forums.microsoft.com/MSDN/ShowPost.aspx?PostID=282651&SiteID=1 and appreciate the time he took to explain how importance works. However, like the user with username "sang", I also ran the data in BI 2005 and got the same results listed by the aforementioned user. I did this using the following data:
donut
muffin
y
y
y
y
y
y
y
y
y
y
y
y
y
y
y
y
y
y
y
y
y
y
y
y
y
y
y
y
y
y
n
y
n
y
n
y
n
y
n
y
etc.
The rule muffin -> donut has an importance of -0.105302438, which is not the same as Mr. MacLennan's results. I tried switching the roles of a and b in a -> b and using different bases on the logarithms. I don't get the result of -0.105302438 with any of these. I also tried to calculate importance with a small data set I have and can't get the results using Mr. MacLennan's explanation with that data set either. Any thoughts on the descrepancy?
hi,
i have a exercise using association datamining
my database have 350 records,
i use 90 records for datamining and it release some rules which i choose on top of mSOLAP_NODE_SCORE,
but when i use select statement to check my result i have 1 records, the same as my result, and 5 records not true;
for example:
rules A=a,B=b-> C=c
select * from <my_table> where A='a' and B='b' and C='c'; ==>1 record return
select * from <my_table> where A='a' and B='b' and C<>'c'; ==>5 records return
C with 3 values c1,c2,c
with the second statement C includes 2 c1 and 3 c2
i don't understand how they work.
i want to choose some best rules can present my database.
how can i choose importance and probability to get best rules.
with database have 90 records and a database have 350 records which values i should use for minimum_probability, Minimum_Support, Minimum_importance...
when i choose rules i should choose on importance or probability.
thanks for your help
I have run into a .. somewhat of a "duh" question. I'm running association rule to run a basket analysis, and I'm trying to get probability of each prediction. I know this is wrong, but how do I go about running PredictProbability on each ProductPurchase prediction?
When I run the below DMX query, I get this error message...
Error (Data mining): the dot expression is not allowed in the context at line 5, column 25. Use sub-SELECT instead.
Thanks in advance...
-Young K
SELECT
t.[AgeGroupName]
, t.[ChildrenStatusName]
, (Predict([Basket Analysis AR].[Training Product], 3)) as [ProductPurchases]
, (PredictProbability([Basket Analysis AR].[Training Product].[ProductName])) as [ProductPurchases]
From
[Basket Analysis AR]
PREDICTION JOIN
OPENQUERY([DM Reports DM],
'SELECT
[AgeGroupName]
, [ChildrenStatusName]
FROM
[dbo].[DM.BasketAnalysis.Contact]
WHERE isTrainingData = 0
') AS t
ON
[Basket Analysis AR].[Age Group Name] = t.[AgeGroupName]
AND [Basket Analysis AR].[Children Status Name] = t.[ChildrenStatusName]
I haven't been able to find a DMX query which will spit out the cases which support a particular association rule. I was hoping it would work sort of like drillthrough but show only the cases supporting a particular rule. Am I missing something?
What I ended up doing was extracting the itemsets of the rule from the model's content then running a SQL query to retrieve the cases that contain both the left-hand and right-hand itemset of the rule. I'm hoping there's a better way.
I read somewhere that market basket analysis finds rules with substitutes as likely as rules with complements due to a consumer behavior called "horizontal variety seeking". This is when customers buy more than one product in the same category even though they are subsitutes. For example, when people go to the grocery store and buy soda, they buy coke and sprite at the same time even though they are substitutes of each other. I was wondering if anyone has experience with this anomaly and how they solved it. I found a time series model called the vector autoregressive model which is used to find the elasticity of prices over a time period. Does anyone have experience working with the VAR model? I am having trouble figuring out what some of the variables in the model are.
Below is the paper
http://www.feb.ugent.be/fac/research/WP/Papers/wp_04_262.pdf#search='VAR%20model%20market%20basket%20analysis'
What is the best practice in setting a minimum support threshold for market basket analysis? Is there a formula? Does it depend on ROI you predict?
View 4 Replies View RelatedWhat is the algorithm that generates the itemsets in the Association model? I'm looking to possibly use this part of the Association algorithm (i.e. the grouping into itemsets) in a separate plug-in algorithm.
View 1 Replies View Related
Hi!
I need to deploy several Association algorithms, so I want to do it using ISS. Can anyone help me telling me which task should I have to use to do it?
Thanks!
Ezequiel
I need to create a set of cases for a project that uses the Microsoft Association Rules algorithm to make recommendations for products to customers. My question is: the set of scenarios must include all transactions of customers for training?. or is it sufficient some percentage of total transactions? If i do not use all transactions of customers, could be that the algorithm does not consider some products in their groups or rules and could not make recommendations about these?
thanx
Diego B.
MS uses the a priori algorithm in Association Rules, while other DM software have gone to the Novel Algorithm. Can you tell us why MS decided to stay with the a priori? Did you overcome the limitations that it's accused of having? Thanks!
View 5 Replies View RelatedIn assotiation rules each rule has a [support, confidence] part. In Microsoft Association Rules there is a [probability,importance] measure in each rule and importance can be greater that 1.
I found the following in msdn but i'm not sure if i understood correctly.
MINIMUM_PROBABILITY: Specifies the minimum probability that a rule is true. For example, setting this value to 0.5 specifies that no rule with less than fifty percent probability is generated.
The default is 0.4.
MAXIMUM_SUPPORT: Specifies the maximum number of cases in which an itemset can have support. If this value is less than 1, the value represents a percentage of the total cases. Values greater than 1 represent the absolute number of cases that can contain the itemset.
The default is 1.
My questions are
1) Can i explain the [probability,importance] in [support,confidence]? If yes, how?
2) What importance>1 means?
Thank you in advance.
hi
i m trying to build microsoft association model using Microsoft association algorithm. i got
1) patient table(patientid, name, city)
2) diseases(diseaseid, dieseasename)
It is M:N [many to many] relationship between above tables, so
3)Patient_diseases(patientid,disease_id). [RELATIONSHIP TABLE]
i am trying to associate city in patient table --> disease in diseases table. I want to build association data mining model and use it on web form, such a way when the user enters city associated disease will be displayed.
should i select all 3 table to build the model? could help me to decide what tables should i select as Case and what tables as Nested? what attributes from the table should i select as key, input, predictive ?
i am using data mining tutorials on sqlserverdatamining.com to build this model. is there anything further during my model building i get into confusion? please suggest me where i can find complete resource or inform here.
i appreciate Mr.Jamie for his guidance so far in my academic project. i do have the book 'Data mining with sql server 2005'. I left with just one day to do this and document.
hoping someone could suggest. your help is much appreciated.
regards
raju
managed plug-in framework that's available for download here: http://www.microsoft.com/downloads/details.aspx?familyid=DF0BA5AA-B4BD-4705-AA0A-B477BA72A9CB&displaylang=en#DMAPI.
This package includes the source code for a sample plug-in algorithm written in C#.
in this source code all .cs files are modified for clustering algorithm
if my plugin algorithm is of association or classification type then what modifications are requried in source code???
Dear all,
I have a table containing call records, and made a mining model from that table only. The model has 3 columns : calling_number, called_number, and target_operator, using Association Rule algorithm. The key is calling_number, input was operator, and predicted column called_number.
The result shows no rule, but there are results with item-set size of 1 (column) and 2 (column). On the top record of the result, SQL Server says there are 1891 support for called_number = 1891 and operator = 'INDOSAT'.
I queried the table with this query
SELECT DISTINCT calling_number
FROM call_records
WHERE called_number = '07786000815'
AND target_operator = 'INDOSAT';
It returns 2162 records instead of 1891. If I removed the DISTINCT qualifier, SQL Server returns 2159 records. Why is this differences with the result of mining?
Thank you,
Bernaridho
Hi,
I am using Sql 7.0 with sp2. I just started as a sql dba. I have a question here, What is the importance of SID's ? When we are mapping to sql logins and user_id 's how we have to give importance regarding SID's.
Pls suggest me a good article or some suggestions...
Thanks!
hi i have an association rule mining model and i want to order the output by the importance here is the select statment:
SELECT
[RelatedOrder].[Order Line]
From
[RelatedOrder]
NATURAL PREDICTION JOIN
(SELECT (SELECT 888 AS [Product ID]) AS [Order Line]) AS t
thanx
Hi all-- there is a file in C:Program FilesMicrosoft SQL ServerMSSQLBinn direcetory called sqlctr.h which contains a lot of counter parametres..could any one tell me having its importance and can we change any of parametres to gain performance..
Thanks in advance..
// This file is generated by the description file processor.
// Please do not edit.
#defineBUFMGR_OBJECT0
#define BUF_RESERVED_PAGE_COUNT2
#define BUF_CHECKPOINT_WRITES4
#define BUF_AWE_LOOKUP_MAPS6
#define BUF_BLOCK_WRITES8
#define BUF_COMMITTED_PAGE_COUNT10
#define BUF_AWE_UNMAP_CALLS12
#define BUF_TARGET_PAGE_COUNT14
#define BUF_AWE_UNMAP_PAGES16
#define BUF_CACHE_RATIO_BASE18
#define BUF_FREELIST_STALLS20
#define BUF_HASHED_PAGE_COUNT22
#define BUF_LIFE_EXPECTANCY24
#define BUF_CACHE_HIT_RATIO26
#define BUF_AWE_WRITE_MAPS28
#define BUF_PAGE_REQUESTS30
#define BUF_STOLEN_PAGE_COUNT32
#define BUF_BLOCK_READS34
#define BUF_NUM_FREE_BUFFERS36
#define BUF_LAZY_WRITES38
#define BUF_READAHEAD_PAGES40
#define BUF_AWE_STOLEN_MAPS42
#define BUF_PROCCACHE_SIZE44
#defineBUFPART_OBJECT46
#define BUFPART_NUM_FREE_BUFFERS48
#define BUFPART_FREE_BUFFERS_USED50
#define BUFPART_FREE_BUFFERS_EMPTY52
#defineGENERAL_OBJECT54
#define GO_LOGINS56
#define GO_LOGOUTS58
#define GO_USER_CONNECTIONS60
#defineLOCKS_OBJECT62
#define LCK_TOTAL_WAITTIME64
#define LCK_NUM_WAITS66
#define LCK_AVERAGE_WAITTIME_BASE68
#define LCK_NUM_DEADLOCKS70
#define LCK_NUM_TIMEOUTS72
#define LCK_NUM_REQUESTS74
#define LCK_AVERAGE_WAITTIME76
#defineDBMGR_OBJECT78
#define DB_REPLTRANS80
#define DB_DBCC_SCANRATE82
#define DB_REPLCOUNT84
#define DB_LOG_SIZE86
#define DB_LOG_TRUNCS88
#define DB_LOG_USED_PERCENT90
#define DB_LOG_SHRINKS92
#define DB_BULK_KILOBYTES94
#define DB_FLUSH_WAIT_TIME96
#define DB_ACT_XTRAN98
#define DB_LOGCACHE_READS100
#define DB_FLUSH_WAITS102
#define DB_BCK_DB_THROUGHPUT104
#define DB_DBCC_MOVERATE106
#define DB_LOG_GROWTHS108
#define DB_TOTAL_XTRAN110
#define DB_LOGCACHE_BASE112
#define DB_BYTES_FLUSHED114
#define DB_LOG_USED116
#define DB_LOGCACHE_RATIO118
#define DB_DATA_SIZE120
#define DB_BULK_ROWS122
#define DB_FLUSHES124
#defineLATCH_OBJECT126
#define LATCH_TOTAL_WAIT_NP128
#define LATCH_WAITS_NP130
#define LATCH_AVG_WAIT_NP132
#define LATCH_AVG_WAIT_BASE134
#defineACCESS_METHODS_OBJECT136
#define AM_EXTENTS_ALLOCATED138
#define AM_WORKTABLES_CREATED140
#define AM_GHOSTED_SKIPS142
#define AM_FULL_SCAN144
#define AM_PAGES_ALLOCATED146
#define AM_PAGE_SPLITS148
#define AM_SINGLE_PAGE_ALLOCS150
#define AM_EXTENTS_DEALLOCATED152
#define AM_PROBE_SCAN154
#define AM_FREESPACE_PAGES156
#define AM_WORKTABLES_FROM_CACHE_BASE158
#define AM_LOCKESCALATIONS160
#define AM_PAGE_DEALLOCS162
#define AM_WORKTABLES_FROM_CACHE164
#define AM_INDEX_SEARCHES166
#define AM_FREESPACE_SCANS168
#define AM_FORWARDED_RECS170
#define AM_WORKFILES_CREATED172
#define AM_SCAN_REPOSITION174
#define AM_RANGE_SCAN176
#defineSQL_OBJECT178
#define SQL_AUTOPARAM_REQ180
#define SQL_BATCH_REQ182
#define SQL_RECOMPILES184
#define SQL_AUTOPARAM_UNSAFE186
#define SQL_COMPILES188
#define SQL_AUTOPARAM_FAIL190
#define SQL_AUTOPARAM_SAFE192
#defineCACHE_OBJECT194
#define CACHE_USE_COUNT196
#define CACHE_HIT_RATIO_BASE198
#define CACHE_OBJECT_COUNT200
#define CACHE_HIT_RATIO202
#define CACHE_PGS_IN_USE204
#defineMEMORY_OBJECT206
#define MEMORY_MEMGRANT_MAXIMUM208
#define MEMORY_CONNECTION_MEMORY210
#define MEMORY_MEMGRANT_WAITERS212
#define MEMORY_MEMGRANT_OUTSTANDING214
#define MEMORY_SQL_CACHE_MEMORY216
#define MEMORY_OPTIMIZER_MEMORY218
#define MEMORY_LOCKS220
#define MEMORY_SERVER_MEMORY222
#define MEMORY_LOCKOWNERS_ALLOCATED224
#define MEMORY_LOCK_MEMORY226
#define MEMORY_LOCKS_ALLOCATED228
#define MEMORY_SERVER_MEMORY_TARGET230
#define MEMORY_LOCKOWNERS232
#define MEMORY_MEMGRANT_ACQUIRES234
#defineUSER_QUERY_OBJECT236
#define QUERY_INSTANCE238
#defineREPLICATION_AGENT_OBJECT240
#define RUNNING_INSTANCE242
#defineMERGE_AGENT_OBJECT244
#define MERGE_CONFLICTS_INSTANCE246
#define UPLOAD_INSTANCE248
#define DOWNLOAD_INSTANCE250
#defineLOGREADER_AGENT_OBJECT252
#define LOGREADER_LATENCY_INSTANCE254
#define LOGREADER_TRANSACTIONS_INSTANCE256
#define LOGREADER_COMMANDS_INSTANCE258
#defineDISTRIBUTION_AGENT_OBJECT260
#define DISTRIBUTION_TRANS_INSTANCE262
#define DISTRIBUTION_LATENCY_INSTANCE264
#define DISTRIBUTION_COMMANDS_INSTANCE266
#defineSNAPSHOT_AGENT_OBJECT268
#define SNAPSHOT_TRANSACTIONS_BCPED270
#define SNAPSHOT_COMMANDS_BCPED272
#defineBACKUP_DEV_OBJECT274
#define DB_BCK_DEV_THROUGHPUT276
Hello Experts at Microsoft.
I am thinking of an easy way to explain importance to Marketers without going into the math. This is what i came up with so far. Does this sound correct to you guys?
Reasoning:
IMPORTANCE = Log(Improvement)
Improvement=P(X&Y)/(P(x)*P(y))
Improvement= (Probability 2 products are sold together)/(random chance 2 products are sold together)
If the (Probability 2 products are sold together) = (random chance 2 products are sold together) then Improvement=1. The log(1) = 0
IMPORTANCE SCORE
-2 to -1 10 to 100 times less likely than random chance
-1 to 0 0 to 10 times less likely than random chance
0 to 1 0 to 10 times more likely than random chance
1 to 2 10 to 100 times more likely than random chance
2 to 3 100 to 1000 times more likely than random chance
3 to 4 1000 to 10000 times more likely than random chance
4 to 5 10000 to 100000 times more likely than random chance
5 to 6 100000 to 1000000 times more likely than random chance
6 to 7 1000000 to 10000000 times more likely than random chance
WHY DO WE USE TRIGGERS IN SQL SERVER2005.
WAT IS ITS IMPORTANCE.
AND SOME SAMPLES
PLEASE GIVE ME SOLUTIONS
Is there a way to explicitly assign 'weights' or 'importance' factors to attributes and have that to be considered by the association rules and decision trees algorithms during training? I would like to do so without preprocessing the data (In any case, I can't think on a way to assign weight with preprocessing to boolean attributes like 'smoker')
thanks
Those of you who have installed SQL Server 2005 may have noticed that the installation creates several new Windows groups on the server. Do not underestimate the importance of these groups.
View 3 Replies View RelatedDoes anybody know how to send an email using xp_sendmail sp with HIGH importance setting for the message?
Thanks,
Dim
I try to search my data and sort the result by importance.
I'm using a MS Access database and my data (table1) looks like this:
Code:
ID NAME TEXT
1 Apples Good red apples
2 Bananas Fine yellow bananas
3 Yellow apples Great yellow apples
I want to search the data and get a result where the column "NAME" is more important than "TEXT". My SQL looks like this:
Code:
SELECT id,name,text,1 AS searchorder FROM table1 WHERE name LIKE '*yellow*'
UNION
SELECT id,name,text,2 AS searchorder FROM table1 WHERE text LIKE '*yellow*'
ORDER BY searchorder
The output is this:
Code:
ID NAME TEXT SEARCHORDER
3 Yellow apples Great yellow apples 1
2 Bananas Fine yellow bananas 2
3 Yellow apples Great yellow apples 2
So far so good - the order by importance works - but I do not get unique columns because of the searchorder column.
Can I fix my SQL so I get unique columns where the last line of "Yellow apples" does not appear or am I lost in space?
Best regards,
Peter from Denmark
During testing a package repetatively that deletes/inserts into several tables, over the course of several days, my package, which took 45 minutes to load 1700 XML files, began to take over 6 hours. Turns out it was an I/O bottleneck, and the Avg Disk Queue Length was around 200 and I was incurring many PAGEIOLATCH_EX. My devl machine uses a single local disk, no raid, so I had no options there, but I ran the maintenance wizard to recreate indexes/statistics and defraged the hard drive, and regained my original 45 minutes time. I guess I'll have to put a maintenance plan together to do this nightly.
-Kory
Hi
I came across something like 3-4-5 rule while going through datamining book....but couldn't get from where that rule has been generalized and how it really works....
can anyone explain this rule ?
Thank you
Hi,
How do I extract rules and it's value from a database?
I can extract the rules through view(sys.objects) but where can I get it's content?
Regards
Marcelo Gamba
How can I setup the dbs in sql server so that when I change the data in one table the changes will cascade down to the tables in my other dbs. Therefore, one database would hold a primary key table. If I had 15 other dbs, then I could somehow link them so the data changed in the primary key table of the 1st database would cascade down to the other dbs.
Thanks
I wonder about whether this rule is valid or invalid for nested BEGIN/END statement...
Code:
BEGIN
BEGIN
--Query #1 (blah)...
END
WHILE EXISTS (SELECT TOP 1 * FROM #tmpTblPurchaseRaw)
BEGIN
BEGIN
--Query #2 (blah)...
END
BEGIN
--Query #3 (blah)...
END
END
END
I have no idea if nested BEGIN/END is allowed or not...
I am getting some data from another database and only want to copy sensible data...
How do I write a validation rule in SQL (SQL SERVER 2000) for a fax number, so that it only contains numbers-- i.e digits 0-9
Thanks in advance
I have a query that maps all products to some customer levels. In this case levels 0,5,7 and 8
DELETE FROM ProductCustomerLevel
WHERE CustomerLevelID IN (0, 5, 7, 8)
INSERT ProductCustomerLevel
(
ProductID,
CustomerLevelID
[Code] ....
Basically this maps all products in a database to these customer levels so that they get a discount.
I know need to create a new customer level, example number 9. These will only have 1 or 2 products applied to it.
How can I change the SQL above so that it does not map those products already in Customer Level 9 to levels 0,5,7 and 8
Hi, I have a database which saves data about bus links. I want to provide a information to passenger about price of their journay. The price depends on three factors: starting busstop, ending busstop and type of ticket (full, part - for students and old people, ...).
So I created a table with three foreign key constraints (two for busstops and one for type).
When the busstop is deleted or type of ticket I want all data connected with it to be deleted automatically. I wanted to use cascade deleting.
But I receive a following exception: Introducing FOREIGN KEY constraint 'FK_TicketPrices_BusStops1' on table 'TicketPrices' may cause cycles or multiple cascade paths. Specify ON DELETE NO ACTION or ON UPDATE NO ACTION, or modify other FOREIGN KEY constraints.
How can I achieve my task? Why should it cause cycles or multiple cascade paths?
Hi,I have a table with the following columns:ID INTEGEDR,Name VARCHAR(32),Surname VARCHAR(32),GroupID INTEGER,SubGroupOneID INTEGER,SubGroupTwoID INTEGERHow can I create a rule/default/check which update SubGroupOneID &SubGroupTwoID columns when GroupID for example is equal 15 onMSSQL2000.It is imposible to make changes on client, so I need to checkinserted/updated value of GroupID column and automaticly updateSubGroupOneID & SubGroupTwoID columns.Sincerely,Rustam Bogubaev
View 4 Replies View Related