PredictCaseLikelihood Returns Low Probability For Sequences That Are Very Frequent
Jun 28, 2007
I am working on a text mining application wherein I need to detect unusual/anomalous sentences in text. Certain sentences, that I know occur very frequently, are given a likelihood of 0.2 by PredictCaseLikelihood. Other sentences that are just as frequent get a much higher likelihood (>0.9). I am using the NORMALIZED option. The only significant difference between these sentences is their length. The one with the lower likelihood has only 2 words in it, whereas the one with the higher likelihood has more than 10 words. The problem is that the shorter sentences end up being interpreted as anomalous, when in fact they are'nt. Any suggestions?
View 2 Replies
ADVERTISEMENT
Nov 14, 2006
I'm working with the cluster analysis algorithm (EM) in SQL 2005. I have tried to find documentation on the function PredictCaseLikelihood without luck. Is there any reference on how this function is defined?
View 5 Replies
View Related
Aug 25, 2006
Jamie writes "When Inserting a row on a table where the Primary key is a sequence number. Once inserted is there a key word to find out the Sequence number of the newley Created Row."
View 2 Replies
View Related
Jan 3, 2008
hi
i'm creating a sql hoilday database and i need to write a sequence but i've been told its wrong but i dont know why, can anyone help?
CREATE SEQUENCE SEQ_HOILDAY_SITES
INSERT INTO Details_of_sites_visited(Code_of_the_sites, Sites_name)
VALUES(SEQ_HOILDAY_SITES.NEXTVAL,'124','Yosemite National Park');
thanks
jessica
View 8 Replies
View Related
Apr 12, 2007
hi,
i have a exercise using association datamining
my database have 350 records,
i use 90 records for datamining and it release some rules which i choose on top of mSOLAP_NODE_SCORE,
but when i use select statement to check my result i have 1 records, the same as my result, and 5 records not true;
for example:
rules A=a,B=b-> C=c
select * from <my_table> where A='a' and B='b' and C='c'; ==>1 record return
select * from <my_table> where A='a' and B='b' and C<>'c'; ==>5 records return
C with 3 values c1,c2,c
with the second statement C includes 2 c1 and 3 c2
i don't understand how they work.
i want to choose some best rules can present my database.
how can i choose importance and probability to get best rules.
with database have 90 records and a database have 350 records which values i should use for minimum_probability, Minimum_Support, Minimum_importance...
when i choose rules i should choose on importance or probability.
thanks for your help
View 4 Replies
View Related
May 31, 2005
Im doing a quickie Access Project for a Fileroom orginization DB. Generally, Records are addressed by a 7 character unique ID, with all sorts of rules, assigned by another system. However we need to track Items that for various reasons do not exist in that system. For this, we enter a code that consists of 2 characters of invalid data, followed by a simple incrementing number. In Oracle, i would do this with a Sequence. How do I go about this in Sql Server?basicly, I want to write a trigger that says something like: pseudo code wrote:before insert: if :new.ItemType is Type3 Then-- File local to our file room :new.FileID = "IV" + getNextLocalNumber().ToString() end ifNote that the fileID is not a Key in the normal DB Sense. It is a neumonicly unique identifier w/ some other properties so that people can easily look things up.
View 5 Replies
View Related
Apr 4, 2008
Hi SqlGurus,
I have created table T1 as
created table t1 (no int not null primary key,name varchar(30))
i have created a sequence for this table
when i give the following insert statement
insert into t1 (name) values ('xyz')
the next sequence val should be inserted in column no
this is done generally in oracle with before trigger
how about here .Can we do this ?
View 3 Replies
View Related
Jan 3, 2008
hello !
there is this little dumb thing i don't manage to understand about sequences
when i create a table i put the id as the primary key and write something like this
Code Block[Id] [int] IDENTITY(1,1) NOT NULL,
so each time the sequence is incremented by,
but if i delete a line from a table and then insert a new one
instead of inserting the id 3 for example it inserts id = 4
how can i change this definition sothat if i do
Code Block
delete from table
where id=3
and then i want to to
Code Block
insert into table values ...
it gives a this new line
Code Block
id columnA
3 A
and not
Code Block
id columnA
4 A
thanks in advance for the help
View 6 Replies
View Related
Dec 13, 2006
Hello,
I installed the bike buyer example and i am learning the DMX language. Now i wrote the following query (using MS decision trees):
SELECT
T.[Last Name],
[Bike Buyer],
PredictProbability(Predict([Bike Buyer])) AS [Probability]
From
[v Target Mail]
PREDICTION JOIN
OPENQUERY
(....... And so on..)
Now the result is surprising to me. In the resulttabel all the probabilities are equal.
Bike Buyer Probability
1 0.99994590500919611
0 0.99994590500919611
0 0.99994590500919611
0 0.99994590500919611
0 0.99994590500919611
1 0.99994590500919611
and so on.
Now i am wondering what predictProbability means. I thought that PredictProbability meant the probability that the prediction is correct. Now all the probabilities are the same and the input is different. Can somebody tell me what PredictProbability means or am I using it wrong?
Thanx in advance,
Joris Valkonet
View 6 Replies
View Related
Mar 14, 2008
I have a DMX query like this:
Code Snippet
select * from (
select flattened(*) from (
select att1, topcount(predict([Trans Predictor Unified], INCLUDE_STATISTICS), $Adjustedprobability, 7) as predictedstuff
from [Trans Predictor Model]
prediction join
SHAPE {openquery(DMSCS, 'select distinct CAST(att2 as nvarchar(100)) att1 from DMSCS.dbo.CartProducts order by att1 ')}
append
({openquery(DMSCS, 'select CAST(att2 as nvarchar(100)) att1 , att4, att5 as att3
from DMSCS.dbo.CartProducts order by att1 ')
}
relate [att1] to [att1]) as [Trans Predictor Unified]
as SHAPEQ
on [Trans Predictor Model].[Trans Predictor Unified].att3 = SHAPEQ.[Trans Predictor Unified].att3
) as s
) as t where [predictedstuff.$AdjustedProbability] > 0.5
It's working well. I would like to modify one thing. I would like to chang ethe constant in the where condition, so that it is configurable. That is, I would like to store the constant somewhere (SSAS or relational SQL). I was reading the DMX reference, but it doesn't provide much details about the where's "condition expression". And I looked at a document called "OLE DB for Data Mining Specification version 1.0" of July 2000, which does have in Appendix B the SELECT grammar. There it has
<expression> -> <value>
[...]
| ( SELECT <expression_list> FROM <expression> <where_clause>
[...]
<where_clause> -> WHERE <expression>
If I change the end to
where [predictedstuff.$AdjustedProbability] > (select 0.5 from [Trans Predictor Model] )
, however, just to force some form of query there I get a message saying "The specified column was not found in the context".
I'm running SQL Server 2005.
thanks,
Gustavo
View 1 Replies
View Related
Sep 26, 2001
I want to get the top 10 most frequent cpt's for each dssid.
nuclear medicine 12345
54321
64536
87648
98356
13254
76534
87638
24364
98354
urology 63547
98745
ect...
So,
select dssid,cpt,count(*) from enc_vis_cpt group by dssid,cpt
will give me the cpt's and their frequency for each dssid.
dssid cpt count
SPINAL CORD INJURY 9934120
AMB SURGERY EVAL BY NON-MD 622703
PSYCHOSOCIAL REHAB - GROUP 993414
SPINAL CORD INJURY 983419
AMB SURGERY EVAL BY NON-MD 6327031
PSYCHOSOCIAL REHAB - GROUP 9734114
SPINAL CORD INJURY 9934280
AMB SURGERY EVAL BY NON-MD 6227353
PSYCHOSOCIAL REHAB - GROUP 9934524
How do I limit the output to just display the 10 most frequent cpt's for each dssid. Thank you...
View 2 Replies
View Related
Jul 16, 2007
Hi, all,
I am confused about the value of Probability of Value 1 or 2 (on a particular attribute value) in Neural Network viewer. E.g. the value of Probability of value 1 is actually very low (the same to the value of Probability of value 2), but why the bar which shows the strength of the probability of these two values are still so strong even stronger than other values of probability of value 1 or 2 based on other attribute values which have a much higher probability of value 1 or 2?
And how does the algorithm calculate the Probability of attribute value in nerual network by the way?
Hope my question is clear.
I am looking forward to hearing from you shortly and thanks a lot in advance.
With best regards,
Yours sincerely,
View 3 Replies
View Related
Jul 26, 2007
How is the value of Prediction Probability calculated in the context of decision trees?
View 7 Replies
View Related
Aug 19, 2007
Dear All,
In a data mining model with decision tree algorithm. For example I have the following train case table:
StudentID, IQ,EQ, IsPass.
I put all data in the table into the microsoft decision tree datamining model
StudentID is the key for datamining model
IsPass is prediction only data
IQ, EQ is the input.
1. How can I make a DMX selection to find out all NODE_UNIQUE_NAME with probability of IsPass >0.7.
2. How can I make a DMX selection to find out all the StudentID which belongs to the criteria defined by the Node?
Thanks and regards
Tony Chun Tung Siu
View 1 Replies
View Related
Nov 24, 2006
Hi, all experts here,
Thank you very much for your kind attention.
I am having a question about the node_distribution.PRABABILITY. Some of the attribute values though have a small number of support for the specific node, but why it has a big node_distribution.probability even greater than 1? How can the node_distribution.PROBABILITY be greater than 1? How dose SQL Server 2005 data mining engine calculate the node_distribution.PRPBABILITY for its Clustering algorithm? Really confused and need guidance for that.
Thank you very much for your help.
With best regards,
Yours sincerely,
View 7 Replies
View Related
Jan 16, 2001
Hello all
I'm not sure if this is anything to be concerned about, but I'd appreciate some input. I've created an alert that invokes a log backup scheduled job to start if one of the production database log becomes over 80% full.
I've noticed that the log gets backed up every couple of minutes- sometimes even more frequently- while a series of scheduled jobs are in the process of execution. Otherwise, it runs and backs up the log every hour as scheduled. I've also noticed that some of the jobs are taking longer to complete (they've been running for about 6 months now). Each job truncates the table that it populates with data, so I'm not sure what the cause of the delay is. It doesn't look like there is any fragmentation. Am I missing anything? Thanks
View 1 Replies
View Related
Oct 9, 2006
I have used Base SAS for analysis for a while and it was really great.. everything is easy just with a simple command.. I am sure it's not the same in SQL Server but I need some help on how to start with the following:
I have a field called call_country and another field called call_minute. Each call will be saved with the destination country and the total number of minutes..
and I want to run a query to see what are the TOP frequent destinations in this format:
United States - Count: 420 - Total Minues: 12,345
View 2 Replies
View Related
Jul 14, 2006
There must be a way to do this simply. We're running SQL Server 2000. I'm looking for some generic SQL statement that I can apply.
If I have a table with a person column and a location column and multiple records for the same person / locatioin combination, how do I select the person with the location they most frequently visited? Say George visits Mexico 5 times, and the Bahamas twice and costa rica once. I would have 8 records in my table for George. The data looks something like this:
George/Mexico
George/Mexico
George/Mexico
George/Mexico
George/Mexico
George/Bahamas
George/Bahamas
George/Costa Rica
Ben/Brazil
Ben/Brazil
Ben/Peru
The results would be:
George/Mexico
Ben/Brazil
Thanks!
Myles
View 4 Replies
View Related
Nov 8, 2007
All,
What is the escape sequence in a stored procedure?
Here is what I'm trying to achieve:
ALTER PROCEDURE Test
(
@Func VarChar(1000)
)
AS
DECLARE @SQL VarChar(8000)
SELECT @SQL = 'SELECT DISTINCT TNAME FROM TABLE WHERE FUNC LIKE ' + @Func
Now, my goal is to add single quote (') before @Func and another one after that. For eg, if @Func is "Test", I want my query to be
SELECT DISTINCT TNAME FROM TABLE WHERE FUNC LIKE 'Test'
and NOT
SELECT DISTINCT TNAME FROM TABLE WHERE FUNC LIKE Test
Any help?
View 8 Replies
View Related
May 22, 2008
I have to identify missing records from the example below.
Category
BatchNo
TransactionNo
CAT1
1
1
CAT1
1
2
CAT1
2
3
CAT1
2
4
CAT1
2
5
CAT1
3
6
CAT1
3
7
CAT1
3
8
CAT1
5
12
CAT1
5
13
CAT1
5
14
CAT1
5
15
CAT1
7
18
CAT2
1
1
CAT2
1
2
CAT2
3
6
CAT2
3
7
CAT2
3
8
CAT2
3
9
CAT2
4
10
CAT2
4
11
CAT2
4
12
CAT2
6
14
I need a script that will identify missing records as below
Category
BatchNo
CAT1
4
CAT1
6
CAT2
2
CAT2
5
I do not need to know that CAT1 8 and CAT2 7 are not there as they potentially have not been inserted yet.
I idealy want a nice clean SQL statement and do not particually want to insert new table's or triggers although views i Can deal with to an extent.
Considerations
up to 50,000 records added per day!!!
Only need script to run once a day and I have insert dates to help me.
Only 12 Categorys
Batch numbers always start at 1 for different categorys
View 13 Replies
View Related
Oct 16, 2013
I would like to use sequences and triggers to update table identity field with int value from sequence via before insert trigger. I'm searching on google for a few days and there are no same or identical article about this subject.
Is there any sample how to create table with column Id, Name, Comment and sequence (for generate int numbers for Int field in table) and trigger which will fired before insert and check is inserted Id is NULL and update this field from sequence or nothing if id is set up.
View 15 Replies
View Related
Jul 26, 2004
I have a reviews table where all reviews are submitted. On the main page I want to display the 10 most reviewed products. I have a Product_ID column in this table which identifys the product. How can i write a query which will select the product_ID of records which have the most frequent product_ID's?
I came up with something like this:
"Select Top 10 Product_ID, COUNT(*) AS Occurances FROM reviews GROUP BY Product_ID ORDER BY occurances DESC"
But it does not work.?? It says "Declaration expected" as error
View 5 Replies
View Related
Apr 28, 2008
Hi MSDN ppl,
I seek your expertise yet one other time.
Scenario:
We have 7 databases mirrored on two servers which are mirroring partners. 3 of the 7 databases are live on server1 and mirrored on server2; and the remaining 4 databases are live on server2 and mirrored on server1. The data is exposed through .NET Widows Application.
The configurations of the servers are as follows.
System: Microsoft Windows Server 2003 R2
Standard x64 Edition
Service Pack 2
Computer: Intel(R) Xeon (R) CPU
5130 @ 2.00 GHz
2.00 GHz, 32.0 GB of RAM
SQL Version: Microsoft SQL Server 2005 - 9.00.3175.00 (X64) Jun 14 2007 11:45:39
Copyright (c) 1988-2005 Microsoft Corporation Enterprise Evaluation Edition (64-bit)
on Windows NT 5.2 (Build 3790: Service Pack 2)
Problem:
The databases for no apparent reason keep randomly failing over to one server quite frequently. At least twice a day. There is no pattern associated for me to make out as to why this is happening.
My Questions:
1. Is it a good practice to divide the databases on each server, the way it is now? Or should all the databases be kept on one server and mirrored on other all the time?
2. From the above mentioned scenario, do you find the reason for database to 'failing over' so frequently? Could the Win Application which is used to expose the data be responsible for the failovers?
3. What steps can be taken to check for the reason which is causing the databases to failover? Alternatively and most importantly, how can I this problem of 'Databases Failing over randomly' be solved?
Thank you,
Little_Birdie
View 13 Replies
View Related
May 30, 2008
Hi guys , may I know is there any way for getting the information about the tables that most use frequently in the db?
Best Regards,
Hans
View 2 Replies
View Related
Aug 31, 2014
can i create Custom Auto-Generated Sequences with SQL Server like DD-0001,DD-0002,DD-0003...... to DD-000........
View 1 Replies
View Related
Nov 24, 2006
Hi, all here,
Thank you very much for your kind attention.
I dont understand another problem within my mining model. When I query the mining model content ,finding that the same attribute_value have different support and probability for the same node within my clustering model. Why is that? Really confused. And really need help for that.
Thank you very much in advance for your help.
With best regards,
Yours sincerely,
View 9 Replies
View Related
Jun 5, 2007
What function(s) can be used to find the mode of data? I have a column that is populated with codes and I'd like to summarize the data by the code that occurs the most frequently. Any help is appreciated!!
View 5 Replies
View Related
Mar 11, 2008
How can I make a statement that will return the 10 most frequently occuring values in a column?
I have no idea if that is even possible, if you have an idea on how I could do that I would really appreciate it.
Im trying to make a page that would show some statistics on a table I have.
Im also trying to make something that would show the count of the number of records inserted in the last 24 hours, week, month, year etc. The table has a column called "DateInserted" as SmallDate, right now i can use a Where DateInserted > '20080310' to get the count, but its not dynamic, is there anyone to merge all these results into one row with each column being a diffrent time period?
I know this a lot of questions, but I would really appreciate any pointers.
View 4 Replies
View Related
Jun 30, 2014
we have a handful of developers and each of us are responsible for laying out and creating our own database backends. This often leads to inconsistencies in table and column structures.One obvious situation that comes up often is whether or not the other developers are building in history into their primary tables, using history/archive tables or (usually in cases of helper tables) no historical data at all.
My thought on how to alleviate this a little was to suggest that we all build a IS_DELETED computed column into our tables so that someone else trying to work with their data doesn't have to play the guessing game. In most cases, this column would just be running date comparisons on an Expiration Date and either checking to see if it's in the future (usually 12/31/9999) or NULL.
I have read that computed columns can be a performance hit if used/returned unnecessarily but is that also the case on fields where their main use would be filtering? It just seems that the calculation that the computed column is doing would be necessary for the WHERE anywhere so it seems like a wash ... and worth the benefit of not having to decipher someone else's work.
View 0 Replies
View Related
Jul 20, 2007
Howdy all,
I've seen several posts about reading and writing files that have different record types with varying column metadata. My particular file has 11 record types plus several header types and looks something like:
<Header1>
<Header2>
<Detail01-#1>
<Subdetail02>
<Subdetail03>
...
<Detail01-#2>
<Subdetail02>
<Subdetail03>
...
...
Since i need to get different detail and subdetail records, i can't really use the technique of 3 dest file connection managers found in http://forums.microsoft.com/MSDN/ShowPost.aspx?PostID=87269&SiteID=1
I've tried using an exec sql to get the main detail records and then a forech ADO en umerator that would get the subdetails, but it all seems so kludgy. I'm starting to think that I should just write the bulk of the file creation code in a c# app instead of trying to smush this into SSIS. Opinions? Am I missing some trick in SSIS?
TIA,
-Peter
View 7 Replies
View Related
Oct 5, 2006
Hi
I am constantly getting this error message in the Application log after installing SQL 2005 last night followed by SP1 (say 5 times a minutes). See below:
EventType sql90exception, P1 reportingservicesservice.exe, P2 9.0.2047.0, P3 443f5953, P4 sqldumper_unknown_module.dll, P5 0.0.0.0, P6 00000000, P7 0, P8 00e8ed9d, P9 00000000, P10 NIL.
I have reservice packed SQL 2005 but made no difference - it is running on a Windows 2003 server with all the latest MS patches.
Does anyone know the solution or possible solution to this issue?
Thanks
Matt
View 4 Replies
View Related
Jul 23, 2005
Grrr!I'm trying to run a script:print 'Declaring cursor'declare cInv cursor forward_only static forselectdistinctinv.company,inv.contact,inv.address1,inv.city,inv.state,inv.postalcode,inv.cmcompanyidfromdedupe.dbo.ln_invoice as invleft joindedupe.dbo.customerid as cidondbo.fnCleanString(inv.company) = cid.searchcowhere((inv.customerid is nulland cid.searchco is null)and (inv.date >= '01/01/2003' or (inv.date < '01/01/2003' andinv.outstanding > 0.01))and not inv.company is null)print 'Cursor declared'declare@contact varchar(75),@company varchar(50),@address1 varchar(75),@city varchar(30),@state varchar(20),@zip varchar(10),@cmcompanyid varchar(32),@iCount int,@FetchString varchar(512)open cInvprint 'cursor opened'fetch cInv into@company,@contact,@address1,@city,@state,@zip,@cmc ompanyidprint 'Cursor fetched @@Cursor_rows = ' + cast(@@cursor_rows asvarchar(5))All the prints are there to help me figure out what's going on!When I get to the Print 'Cursor fetched @@cursor_rows....the value is 0 and the script skips down to the close and deallocate.BUT, if I just highlight the Select...When section, I get over 2,000rows. What am I missing?Thanks.
View 6 Replies
View Related
Nov 17, 2007
Hi all,
I am using ODBC to connect SQL Server 2000 + SP4 server runnig on Windows 2003 standard edition Server +SP1. But, sporadically my application server connectivity to DB fails and i receive the following error messages
[Microsoft][ODBC SQL Server Driver][DBNETLIB]ConnectionWrite (WrapperWrite()).
[Microsoft][ODBC SQL Server Driver][DBNETLIB]General network error. Check your network documentation.) In D:de.cpp 702 Apr 12 2006 20:34:47
[Microsoft][ODBC SQL Server Driver]Communication link failure) In D:de.cpp 702 Apr 12 2006 20:34:47
20071017 08:23:10 TID (00000ff0) Sev (3) Err (0) Msg (Read failure.
General SQL error.
Strange thing is that i receive this error only sporadically, Please Please advice !
Thanks ,
Jitender
View 1 Replies
View Related
