Quesion On Algorithm Settings Do Not Really Significantly Improve The Results Of The Mining Models?
Jul 16, 2007
Hi, all,
Another tricky confusion to me is that: many algorithms settings for the native algorithms in SQL Server 2005 Data Mining do not really significantly improve the results of those mining models with settings changes? (Apart from clustering algorithm setting of cluster number, by setting 0 as the number of clusters, the system will automatically cluster the data into clusters which I assume is the best way of mining the model with this method).
Any good advcies on this will be a lot appreciated.
I am looking forward to hearing from you shortly for this confusion and thanks a lot in advance.
With best regards,
Yours sincerely,
View 3 Replies
ADVERTISEMENT
Apr 26, 2006
Hi, all here,
I am wondering where can I store my mining results in data mining engine? For example, I got mining results like accuracy chart, decision trees, and other formats of results based on different mining algorithms I used for my data mining, so where can I actually store the results for reporting service use later? Is it possible to do that in SQL Server 2005?
Thanks a lot for any help and guidance in advance.
View 4 Replies
View Related
Sep 26, 2006
Hi,
I have just run a simple data set through a model to predict a simple true or false value (i.e. binary output)
The Lift Chart/Mining Legend in Analysis Services shows three results €“ Score, Population Correct (%), and Predict Probability (%)
Population Correct I beleive is the percentage of predictions it got right out of the total number of predictions it tried to make. Is this correct?
However, I can€™t work out how the other two are derived in particular the 'SCORE'. To give a live example the scores were as follows:
Model Score Pop Correct Pred Probability
Decision Trees 0.83 76.59% 54.28%
Neural Network 0.75 67.63% 50.05%
Ideal Model 100.00%
Can anyone help with this and give a detailed explanation?
Many thanks,
S Rajput
View 4 Replies
View Related
Nov 27, 2006
Hi ,all here,
Thank you very much for your kind attention.
I just found that I am not able to view the accuracy chart for my mining model. The error message is: no mining models are selected for comparision. Which is quite strange.
Any guidance? thank you very much.
With best regards,
Yours sincerely,
View 5 Replies
View Related
Dec 5, 2006
Hi, all here,
Since we are not able to use accuracy chart for Clustering algorithms there. So how can we verify the accuracy of clustering algorithm models here in terms of its classification and regression tasks?
Thank you very much in advance for your guidance and advices for that.
With best regards,
Yours sincerely,
View 12 Replies
View Related
May 10, 2007
Hello
I've created models with Decision Tree and Neural Network algorithms that predict continous target. But I don't know how to interpret scores that occure under scatter accuracy plot. How should I interpret scores under scatter accuracy plot?
How can I estimate occuracy of model created with Time Series? How can I compare accuracy of model created with Time Series with models created with Decision Trees and Neural Network algorithms?
Thanks in advance.
View 5 Replies
View Related
Jul 16, 2007
Hi, guys,
Thanks for your kind attention.
Just want to make things perfectly work and make the most of our fantastic SQL Server 2005 Data Mining Engine. Can any of you here give me some super advices on the validation of the mining models. As we always see, the 3 aspects of a mining model are: Score, Population correct, and Predict Probability. So the question is: How can we combine these three aspects to best judge the mining models by being able to tell which model is the best one? And to what extent can we really trust these mining models?
These are very important before we can actually bring the models into work to convince other people who have no ideas what are going on with these models. Yes, we just want to convince them with the results of these models and make the most of them and best help them getting the most from their business operations etc.
By the way please can you explain a bit details on each of these aspects? Thanks again.
I am looking forward to hearing from you shortly and thanks bunch for your help.
With best regards,
Yours sincerely,
View 3 Replies
View Related
Jun 12, 2006
Hi, all here,
I am having a question about automating data mining models managements. As we know in many businesses, patterns vary very frequently, therefore, the mining models created will need to be created again afterwards according to new rules appearing in the data. But can we make all these process automated like automatically assessing the mining model accuracy and automatically recreate the mining models based on predifined specifications? Would please any one here give me any idea about that?
Thanks a lot for any guidance and help for that.
With best regards,
View 3 Replies
View Related
Jul 16, 2007
Hi, all,
Again I am confused about the extent of being convinced by the mining models. We can validate the models with accuracy chart, but then to what extent can we trust that? (you never get 100% correct miming models) If we dont trust the results of the models, then the patterns the models discover are meaningless.
Just need some advices from you experts here to help me convince people on what I got from my mining models.
I am looking forward to hearing from you shortly and thank you very much.
With best regards,
Yours sincerely,
View 3 Replies
View Related
Jul 2, 2007
I am in the process of creating an Integration Services package to automate the process of training mining models and getting predictions. Until recently, I have been processing the models directly from Business Intelligence Studio without any problems. However, when I try to use the exact same training set as an input to the Data Mining Model Training destination, I get several errors. Here is the output:
[Mining Models [1]] Error: Parser: An error occurred during pipeline processing.
[Mining Models [1]] Error: Errors in the OLAP storage engine: The process operation ended because the number of errors encountered during processing reached the defined limit of allowable errors for the operation.
[Mining Models [1]] Error: Errors in the OLAP storage engine: An error occurred while the 'CPT MODIFIER' attribute of the 'BCCA DMS ~MC-CLAIM LIN~5' dimension from the 'BCCA LRG DMS TEST' database was being processed.
[Mining Models [1]] Error: File system error: The record ID is incorrect. Physical file: . Logical file: .
[Mining Models [1]] Error: Errors in the OLAP storage engine: The process operation ended because the number of errors encountered during processing reached the defined limit of allowable errors for the operation.
[Mining Models [1]] Error: Errors in the OLAP storage engine: An error occurred while the 'BILL TYPE' attribute of the 'BCCA DMS ~MC-CLAIM LIN~5' dimension from the 'BCCA LRG DMS TEST' database was being processed.
[Mining Models [1]] Error: File system error: The record ID is incorrect. Physical file: . Logical file: .
[DTS.Pipeline] Error: The ProcessInput method on component "Mining Models" (1) failed with error code 0x80004005. The identified component returned an error from the ProcessInput method. The error is specific to the component, but the error is fatal and will cause the Data Flow task to stop running.
I have not been able to find an answer as to why this is happening. I found a post regarding a similar problem with processing an OLAP cube in SSIS, but it seems that the author of that post never found an answer. Has anyone else here seen similar errors when processing mining models from Integration Services?
Also, if I process the mining models manually then try to run only predictions in SSIS, I get many of the same errors. I'll keep looking into the problem myself, but I would be very grateful if someone in this forum could shed some light on this issue.
View 4 Replies
View Related
Nov 20, 2006
Hi, all experts here,
Thank you very much for your kind attention.
I got a strange problem with SQL Server 2005 data mining models though. I have selected the input columns for my mining model (which are different from the input columns for its mining structure, since I ignored some of the columns for the selected model). But the mining model still used all input columns from the mining structure rather than those I chose for the mining model.
Would please any one here give me any guidance and advices for that. Really need help for that.
Thanks a lot in advance for any help.
With best regards,
Yours sincerely,
View 7 Replies
View Related
Dec 18, 2007
Hi!
If you make an view you can script it in SSMS and get the DDL.
How to do something similar with mining models/structures getting DMX?
I see you easy can get XMLA, but I would prefer the "dmx-ddl"...
Thanks
Bjorn
View 1 Replies
View Related
Dec 12, 2006
Still new to DM and SSIS...anyand all help is greatly appreciated!
In SSIS they say that you can use the Analysis Services Processing Task to process a mining model/mining structure, however, I do not see where you can give it a relational table to work off of. I know that I can use a data flow to do this but I wanted to go a different route if I could to process my models as I don't really necessarily need the data flow as what I am tring to do is pretty simple.
That brings me to a more general question, what is the best method for training your models using SSIS? I am building a new model everytime the package runs using some variables and the DDL task, running a query on it, and destroying it at the end of the package but I am having logistical problems training it outside of the data flow. I tried using the DM Query task but it requires that you output a result set and I am not sure if I can use it to create and process models.
I would think that they would just give you a DMX task similar to the SQL task but that does not seem to be the case. Also, when I browse the AS objects via the processing task I can only see the mining structures and not the mining models.
Am I just missing something here?
Cheers,
Dan Meyers
View 5 Replies
View Related
Feb 26, 2007
Hello friends,
Can u give some idea about the Algorithm in Data Mining for Clustering..
Please reply...
View 1 Replies
View Related
Jun 28, 2007
Hi !!
I read that it is possible to create a custom algorithm and use it as a plug in to sql server 2005. What programming language are available for this purpose ? C++ only ? Can I use .net ?
Thank you!
View 3 Replies
View Related
Jul 25, 2007
Hello!
I'll try to explain what I'd like to do. On my SQL Server 2005, SSAS contains a mining model (In fact a cluster one).
I'd like to show a detailed diagram build from this model on a web site. This diagram (and this is why I need automation) would depend on the user who's consulting it.
For example, a firm A will see the number of its customers in each cluster, and this information will be different for another firm B
So, I thought about several steps to perform:
1) Feed the model with the data for each firm
2) Build a Visio diagram from the previous data using the DM addins for Visio 2007
3) Generate HTML using the Visio export wizard
4) Publish HTML
And (important) this should be done automatically.
I made some experiments:
Step 1 is easy to perform with SSIS
Step 3 is also easy to do with the Visio SDK (using, among others, the exportAsHtml control) and Step 4 is trivial
I failed to perform stage 2, even with the SDK, since creating a diagram from a DM template requires the user to fill a wizard. By code, I'm able to create a document from a DM template, Drop a DM Stencil but when the wizard appears I'm unable to get a handle on it. And even if I was, I dunno if it would be a "clean" way of doing.
So my question is: first of all: isn't there an easier way to generate HTML from Mining Models automatically? And if my approach to this problem is the best (or the "least bad"), how to generate datamining Visio diagrams from code automatically?
View 5 Replies
View Related
May 18, 2007
I have worked with many data mining softwares and found SQL Data mining part of Microsoft product needs to imrove a lot to whoo the customers.
View 10 Replies
View Related
Nov 29, 2006
I am using BI Dev Studio for SS2005 in a research (as opposed to a production) environment. Often I want to compare the results of multiple models using the same attributes. If I switch to a different model, the Design view completely resets. Is there any way to retain the same field names with different models in the Design view?
My current workaround is to give my models similar names with AR, DT, CL, LOG, NN suffixes and make global changes in the DMX.
I have consulted the following without finding an answer:
http://msdn2.microsoft.com/en-us/library/ms178445.aspx
http://msdn2.microsoft.com/en-us/library/ms175642.aspx
http://msdn2.microsoft.com/en-us/library/ms175678.aspx
http://msdn2.microsoft.com/en-us/library/ms175637.aspx
Thanks for your help,
Sam
View 3 Replies
View Related
May 27, 2007
Lets take the following example:
Movie train table:
ID Class
1 +
2 +
3 -
4 +
5 -
Actor train nested table:
ID MovieID Gender
1 1 F
2 1 M
3 1 F
4 1 F
5 2 M
6 2 M
7 2 F
8 3 F
9 3 F
10 4 M
11 4 M
12 4 F
13 4 F
14 5 F
15 5 M
We want to build a classifier model in order to predict the Class of a Movie based on the Gender of movie's actors. To deal with the nested table Analysis Services maps each record of the nested table to an attribute of the case table. These attributes are named Actor(n).Gender with n = 1..15, and so they are dependent on the nested table record numbers. Both Microsoft Decision Trees and Microsoft Naive Bayes algorihms use these attributes without any modification.
We are implementing a Relational Naive Bayes algorithm and we are planning to aggregate such attributes in order to make them independent of the nested table record numbers.
Next step we tried to predict some unseen cases and here we face with
a very huge problem.
Lets take more two tables of unseen cases:
Movie test table:
ID Class
6 +
7 NULL
8 NULL
Actor test nested table:
ID MovieID Gender
1 6 F
2 6 M
3 6 F
4 6 F
16 7 F
17 7 M
18 7 F
19 7 F
20 7 F
21 8 M
22 8 M
23 8 F
Predicting the movie 6 Class is not a problem since the movie actors were included in the training dataset and when the records are mapped to attributes because they already exist in the model. But when you
try to predict movies (7 an 8) with unseen actors all new attributes are simply ignored in the ALGORITHM:redict call (in_ulCaseValues is zero!) because they do not exist in the model!
What is the solution?
View 3 Replies
View Related
Aug 31, 2006
Hi,
actually i want to implement a c# component that can help third party data mining algorithm
and i want managed plug-in algorithm (without wrapper),.
is it possible to write whole application in c#
as Analysis servisce is in c++ , can we implement it in c#
thanks in advance
~Chinmay
View 11 Replies
View Related
Sep 7, 2006
managed plug-in framework that's available for download here: http://www.microsoft.com/downloads/details.aspx?familyid=DF0BA5AA-B4BD-4705-AA0A-B477BA72A9CB&displaylang=en#DMAPI.
This package includes the source code for a sample plug-in algorithm written in C#.
in this source code all .cs files are modified for clustering algorithm
if my plugin algorithm is of association or classification type then what modifications are requried in source code???
View 9 Replies
View Related
Dec 23, 2006
I tried Data Mining Add-Ins for Office 2007 - CTP December 2006.
Test settings: Windows XP SP2 english with Italian regional settings, Office 2007 english (RTM), SQL Server 2005 Developer (with SP2 CTP Dec06) and Data Mining Add-ins for Office 2007 (CTP Dec06).
If I keep regional settings in Italian, I get error like this:
Old format or invalid type library. (Exception from HRESULT: 0x80028018 (TYPE_E_INVDATAREAD))
If I change regional settings to English, the Add-in works.
I found this description as the possible cause of the problem: http://msdn2.microsoft.com/en-us/library/ms178780(vs.80).aspx - if this is the issue, it would be necessary to change the ExcelLocale1033Attribute on the component.
Is there another workaround other than to install the Office 2007 MUI?
Marco Russo
http://www.sqlbi.eu
http://www.sqljunkies.com/weblog/sqlbi
View 10 Replies
View Related
Feb 19, 2008
Hi all,
I have a very simple time series model which processing works fine without any problem. However when I run the following query
SELECT
[TimeSeries].[PriceChange],
[TimeSeries].[Symbol],
PredictTimeSeries(PriceChange, -3, 2)
From
[TimeSeries]
WHERE
[TimeSeries].[Symbol] = 'x'
I get the following error:
TITLE: Microsoft SQL Server 2005 Analysis Services
------------------------------
Error (Data mining): A time series prediction was requested with a start time further in the past than the internal models of the mining model, TimeSeries, specified in the HISTORIC_MODEL_GAP and HISTORIC_MODEL_COUNT parameters can process.
The following is the excerpt of the minding model script related to the two parameters:
<AlgorithmParameters>
<AlgorithmParameter>
<Name>MISSING_VALUE_SUBSTITUTION</Name>
<Value xsi:type="xsdtring">Previous</Value>
</AlgorithmParameter>
<AlgorithmParameter>
<Name>HISTORIC_MODEL_GAP</Name>
<Value xsi:type="xsd:int">1</Value>
</AlgorithmParameter>
<AlgorithmParameter>
<Name>HISTORIC_MODEL_COUNT</Name>
<Value xsi:type="xsd:int">10</Value>
</AlgorithmParameter>
</AlgorithmParameters>
These HISTORIC_MODEL_GAP (1) and HISTORIC_MODEL_COUNT (10) should accommodate PredictTimeSeries(PriceChange, -3, 2). Could anyone shed some light on this?
View 3 Replies
View Related
Oct 23, 2006
hi guys,
i am encountering a pretty weird problem on our SQL database stored procs, particularly those concerning datetime functions. we have two setups, one is on london, and another one is here in singapore. both servers have the same regional settings, the same database tables, stored procs and functions, almost the same in every aspect. the only difference we know is that one is a direct database cut (sql 7.0), while the other used database migration in win2k.
here goes: all stored proc functions work just fine on the singapore setup. the london setup however, is not working at all! it does all the functions like change flags and status, but it doesn't update dates nor search for data by date. i have also checked on the collision names, and they are the same. where else could i start searching for discrepancies between the two?
hope to hear something from you guys, i'm getting desperate. thanks in advance!
View 5 Replies
View Related
Nov 24, 2006
Hi, all here,
Thank you very much for your kind attention.
I am wondering if it is possible to use SSIS to sample data set to training set and test set directly to my data mining models without saving them somewhere as occupying too much space? Really need guidance for that.
Thank you very much in advance for any help.
With best regards,
Yours sincerely,
View 5 Replies
View Related
Feb 5, 2008
Hello,
I've just created a Sales Forecasting Mining Model. How do i export the predicted time series values into a table?? Is there any tutorial explaining this.
Regards,
RG
View 6 Replies
View Related
May 9, 2006
Do data mining results store at some files or in the database? I need the mining results at later time for reporting purpose.
View 1 Replies
View Related
Feb 11, 2008
Is there a way to display the actual predicted value for an output attribute for a particular model. For example, say I am trying to predict if a particular customer is going to take advantage of a promotion (0=no, 1=yes) and I use neural networks. I know that I can use "Predict" to give me the prediction "yes" or "no" for each customer. However, the neural network actually spits out a number as a result. For example, a 0.997 would be interpreted as a "yes" for life insurance promotion. I do not want the probability that the prediction is correct. I want the actual output for the network.
The reason being is that I want to compute an error rate between the predicted value and the acutal value (root mean squared error or some other measure). Is there a way to compute this using the mining model prediction tab design view? I do not want to write the actual query as I teach a course in data mining using SQL Server and my students do not know DMX queries.
Thanks for any help you can provide.
View 5 Replies
View Related
Nov 25, 2007
Hi All
Microsoft released an introductory tutorial for data mining in September 2007, and I was attempting to match the results on:
http://msdn2.microsoft.com/en-us/library/ms169911.aspx
I was not able to match the results, so then I went "under the hood" for the tables, and discovered that my copy of AdventureWorksDW has truncated values in the Education field of the ProspectiveBuyer table. I had wanted to connect "EnglishEducation" from the Mining Model to "Education" in the "Input Table" to see if the additional linkage would affect the prediction results (the wizard does not automatically link these fields because they technically do not have the same name).
1) Is everybody's "Education" field in the "Prospective Buyers" table populated with truncated values? (which may be intentional since it is a sample dataset) -- compare to "EnglishEducation" field in the vTargetMail table
2) Am I the only one getting 0.50580 for all rows when I use "Decision Tree" (per the tutorial instructions)? By contrast, I was able to see a variance in expression values for clustering and naive Bayes.
My guess: everybody has a truncated "Education" field, and there is something different in how the data mining model was run under the Decision Tree option to yield the tutorial's numerical results for "expression".
View 8 Replies
View Related
Jul 28, 2006
Hi
I am using association alogorythm for market bsket and I want to publish this results to the CRM system or to any Front end application is it possible and if yes pl let us now how to go head with the same
Thanks In advance
Rajesh Ladda
View 1 Replies
View Related
Sep 21, 2007
I have a table with two columns ( Column A and Column B)
I want to write a Stored Proc or View. Basically which checks if column A is null then select value from Column B.
If Column A is null select from B.
Kindly Help
View 5 Replies
View Related
Jun 14, 2005
Hi,
I am trying to do the conversion function (change from kb - mb-gb) in sql
and this is the function i wrote.
when i give dbo.conversion (1234567) it works
but when i give dbo.conversion (123456789432) it says
"Arithmetic overflow error converting expression to data type int."
what could be the problem. any help ??
CREATE FUNCTION conversion(@Number1 Decimal(30,2))
RETURNS Decimal(30,2)
BEGIN
DECLARE @Result Decimal(30,2)
IF @Number1 < 1024
SET @Result = @Number1
IF @Number1 >= 1024 and @Number1 < (1024*1024)
SET @Result = cast((@Number1/1024.0) as decimal(30,2))
IF @Number1 >= (1024*1024) and @Number1 < (1024*1024*1024)
SET @Result = cast((@Number1/(1024.0*1024)) as decimal(30,2))
IF @Number1 >= (1024*1024*1024) and @Number1 < (1024*1024*1024*1024)
SET @Result = cast((@Number1/(1024.0*1024*1024)) as decimal(30,2))
RETURN @Result
END
select dbo.Addition5('1234567896') [/SIZE]
View 2 Replies
View Related
Apr 11, 2007
Hi,
I am trying to model data in analysis services with the Advance Create Mining Model function in the excel addin. I am having trouble creating an association model that works like the Associate button above the Advanced button.
The format of my data is like this
OrderID Product
100 Bike
100 Helmet
100 Shoes
200 Helmet
200 basketball
200 Bat
300 Shoes
300 Socks
The associate button works perfectly since it asks me which column is the transaction id (orderid) and which column I am trying to predict (product). The advanced create mining model asks me to determine what the columns are...
OrderID=key Product=Input+Predict?
When I run the advance create mining model associate, I get a browser that gives me no rules and the support for only one item itemset (each product but no combination of products).
Does anyone know what I have to do to get it to work like the associate button?
View 8 Replies
View Related
