Sensitivity To The Distribution Of The Binary Dependent Variable
May 23, 2007
Hi all,
I have a data mining model that has a binary dependent variable (let's say 'Yes' and 'No'). I noticed that the model is very sensitive to the distribution of the binary dependent variable. I could not come up with an explaination. I know the model is not over/under fits and multicollinearity is not an issue. And I have enough data size to train the model.
I would expect some sensitivity but not to this extend:
Yes % in training data
Yes % in the prediction results
with DT
with LR
with NN
30.0%
6.0%
11.0%
14.0%
35.0%
12.0%
15.0%
24.0%
37.5%
20.0%
23.0%
27.0%
40.0%
28.0%
33.0%
36.0%
50.0%
43.0%
52.0%
61.0%
DT - Decision Trees
LR - Logistic Regression
NN - Neural Networks
And to make things more interesting, although the precision (% of correct 'Yes's ), true positive ratio and false positive ratio is changing; the overall accuracy (and therefore the score in the mining accuracy chart) remains almost same.
If anyone has an idea it would be highly appreciated.
I am working on a C#/asp.net web application. The application has a text box that allows a user to enter a name. The name is then saved to the database. Before the name is saved to the database, I need to be able to check if the name already exists in the database. The problem here is that what if the name is in the database as "JoE ScMedLap" and somoene enters the name as "Joe Schmedlap" which already exists in the database,but just differs in case. In other words how do deal with case sensitiviy issues.
On SQL Server 2005 SP2 for Publisher and Distributor on the same instance, my old snapshots are not being cleaned up.
The following error is in the agent history:
Executed as user: DomainMyUser. Could not remove directory '\vmsql01ReplDatauncPublication_TRANSACTIONAL20070702104416'. Check the security context of xp_cmdshell and close other processes that may be accessing the directory. [SQLSTATE 42000] (Error 20015). The step failed.
xp_cmdshell is enabled and I can run commands like :
exec master.dbo.xp_cmdshell ' md c:TestFolder'
The permissions to the snapshot share and file system are that DomainMyUser has full control.
I have logged into the machine as this user and can remove snapshots so it does not seem to be a permission issue.
On other machines I do not get any errors but the snapshot folder still is not cleaned up.
I have a table with 10 rows with a varbinary column
I wish to concatenate all the binary column into a single binary column and then write that to another table within the database. This application splits a binary file (Word or PDF document) into multiple segments (this is Column2 as below)
I'm using a bit-wise comparison to effectively store multiple values in one column. However once the number of values increases it starts to become too big for a int data type.you also cannot perform a bitwise & on two binary datatypes. Is there a better way to store the binary data rather than int or binary?
I am wondering if T SQL in SQL Server 2005 is case sensitive.
I running am running a query in a stored procedure whchi compares a passed in value with that in a field in the database, as such, is their a need to do thisSELECT * FROM table WHERE UPPER(column_name) = UPPER(@var) or will this return the same resultsSELECT * FROM table WHERE column_name = @var
Hi I have two questions about case sensitivity; 1. Do most people write their queries with UPPERCASE keywords or is that only for the textbooks. Personally I find it a hassle, My little finger aches because it is always on and off the SHIFT key 2. I noticed that when I did the following query ----------------------------- use AdventureWorks go Select * employee ----------------------------- I get an 'invalid object' message But if I use; -------------------------- use AdventureWorks go Select * Employee -------------------- It works. SO obviously the query is case sensitive. Did this use to be the case with SQL 2000. I don't seem to remember having to be careful about case.
Is SQL server defaults to case insensitive? I am sure there are ways toinstall case sensitive SQL server instance but coming from Sybase(which is always case sensitive) case insensitivity is something new tome (it requires coding change etc).Besides, is there any option I can set to turn on case sensitivity or Iam stuck with it? (I can not reinstall SQL server).Thanks.
I scripted a function to a sql file, transferred the file to a different machine and when I run it it fails. It turns out that it thinks @result does not exist because I delcared @Result. Changing it to uppercase R resolves the problem. I usually keep my case sensitivity but missed this one. This is the first time I've ever seen sql care about the casing though! Is it a setting?
Hello all, I've come across a bit of an issue, I'm testing software delivered by another company that includes an SQL2005 database. I've rolled back their live database on my developer database, but my server has a binary collation set as default server collation. Because of this (or so I think) all variable/table/etc names in any SQL code are case sensitive, even when called in the context of a database with a case insensitive collation. As an example, when I run the following code on my developer computer I get an error, on their live server I don't.DECLARE @number INT SET @Number = 1
Because of this I am getting alot of errors from their code, since their entire application has differing capitalization usage from page to page. Is there any way around this or do I need to reinstall the SQL Server on my computer to set a different collation? Any help would be appreciated, thanks in advance.
Hi,I believe my SQL server was configured as Case sensitivity. I have anumber of stored procedures which were moved from a non-Casesensitivity SQL server. Because of the Case sensitivity, I have to doa lot of editing in those stored procedures. Is there a quick way toavoid the editing?Something like ignoring the case in one statement?Thanks in advance, your advice will be greatly appreciated.
This is kind of a big generic question, if anyone could even just point me to an article or white paper on this it would be helpful, but I can't find an answer. I do work for a international company that currently does all in english, but is looking to support more languages.
What exactly do the Accent and Kanatype Sensitivity tags in RDL refer to, and how would/when should they be used?
Hi All, How to remove case sensitity from database like table names,column names etc. If we type either select * from AUTHORS" or "select * from authors" should result the same value. Abdul
I require to change the case sensitivity of a server. If I run rebuildm (changing the sensitivity to the desired value) will that not mean that I won't be able to reattach the databases?
Is there a chance to change the case sensitivity option for a running instance of sql server?
I know, during the install process, i can activate the case sensitivity. But can I change this option on an already installed server (to case sensitiv or back)?
I have a SQL Server database hosted with a web hosting company. TheSQL Server was clearly set up to be case sensitive, however, I wantthis particular database to be case-insensitive.I have searched high and low, the best suggestion I can find is toreinstall SQL Server and select case-insensitive. But since this isthe web host's SQL, that isn't an option here.With default language I can use the sp_defaultlanguage to change toBritish settings (for example). Is there something similar I can useto make just this database case insensitive?--Popular uprising?http://www.blairfacedlies.org/statue.htmcaptain(underscore)flack(squirlything)hotmail(you know what)com
I have a query which filters records containing uppercase andLowercase i.e.Smith and SMITH, Henderson and HENDERSON etc.Is there a way that I can filter only those records that contain thefirst uppercase letter and the remaining lowercase letters for myquery i.e. Smith , HENDERSON etc.Thanks
Hi All - I am using OpenXML to insert data into SQL Server 2005. All works okay except for the fact that in my XML I have an attribute tag and elements as follows.<Location> <Place City = "LA" State = "CA"> <Place city = "NY" State = "NY"> <Place City = "St. Louis" state = "MO"></Location> If you notice in the above code, the 2nd city NY has element tag "city" in lower case and others are "City" with a Upper Case "C" same for state in case of MO which is lower case and others are Upper Cases..Now I understand that XML is case sensitive. What should I do if I have XML files in the above format with inconsistent element tags? Because if I run the below code.. using OpenXML...Insert Into TableXSelect *FROM OPENXML (@DocHandle, '/Location/Place', 1) WITH (City varchar(255) '@City', State varchar(255) '@State') ox Then in the above scenario it won't insert values for city "NY" and state "MO" since both have lower case elements tags and in my OpenXML I have specified as @City and @State with Upper Case "C" and "S" respectively. Now in such a scenario what can one do? Any thoughts is to how I can also grab value for lower case "city" and "state" in the above XML. Any help would me much appreciated. Thanks a billion.Regards- web
So I have been unfortunate enough to inherit a whole estate of sql servers running a collation of Latin1_General_BIN. I have a batch of maintenance and monitoring scripts that I want to implement on these servers. However I seem to have problems compiling the stored procs as the case of the column names varies throughout the script and table definitions.
Ok I know I could recode them, but is there way to overide the case sensitivity in insert and update statements so they dont have to be recoded?
I have a ABC service on a server1 that is dependent on MSSQLSERVER service running on server2. Does anyone know how to setup the service ABC on server1 to have a dependancy on MSSQLSERVER running on server2?
I am trying to query only the Max date dependant on quantity
Create Table dbo.TestParts (Part char(30), Desc1 char(50), Desc2 char(50)); Create Table dbo.TestStructure (Model char(30), Part char(30), EDATE smalldatetime, QtyPer float); GO Insert INTO dbo.TestParts Values('101111','Widget A', 'Batteries Not Included'),
[Code] ....
Looking for a return of:
PART Description EDATE QtyPer 101112 Widget B ..... 2012-12-03 3 101113 Widget C ..... 2012-12-03 5 101114 Widget D ..... 2012-12-01 1
Widget A should not show because the last date the qty was changed to Zero
Closest that I have come..........(which is pulling the part 101111 which should be incorrect)
Select ts.Part, RTRIM(tp.Desc1) + ' ' + RTRIM(tp.Desc2) as Description, ts.EDATE, ts.QtyPer FROM testing.dbo.TestStructure ts Inner Join (Select Part,MAX(EDATE) as Date FROM testing.dbo.TestStructure WHERE QTYPER <> '0'
SELECT INVENTORY_ITEM_TAB.ITEM_NO, INVENTORY_ITEM_TAB.DESCR, INVENTORY_ITEM_TAB.STATUS_FLG, INVENTORY_ITEM_TAB.PRICINGUOM, INVENTORY_ITEM_TAB.PURCHUOM, INVENTORY_ITEM_TAB.ITEM_CATEGORY, INVENTORY_ITEM_TAB.ICINTERNALNOTES, INVENTORY_WHS.QTY_ON_HAND, INVENTORY_WHS.QTY_ON_ORDER, INVENTORY_WHS.QTY_ALLOCATED FROM INVENTORY_ITEM_TAB INNER JOIN INVENTORY_WHS ON INVENTORY_ITEM_TAB.ITEM_NO = INVENTORY_WHS.ICWHSPCODE WHERE (INVENTORY_ITEM_TAB.ITEM_CATEGORY = 'SS') AND QTY_ALLOCATED, QTY_ON_HAND, Qty_On_ORDER <> 0 ORDER BY INVENTORY_ITEM_TAB.DESCR
But I don't want to select records if all of these fields have a 0 in them - QTY_ALLOCATED and QTY_ON_HAND and QTY_ON_ORDER.
how to do this type of query. If any of those fields doesn't have a 0 then I would want to return it. I just don't want them if all three of those fields have 0.
Suppose I have the following table:col1 col2hammet jonesjlo afflectafflect armandwills snoptarmand hammetjones smithIf someone choses armand, then I'd like to returnamand hammet jones smithThe first selection goes over to the second column, gets that valueand locates it back in column one and returns column 2 and so on.One way of doing it is to set up a separate query for each one andthen construct a new query to get them all.I'm thinking there's a more elegant way to do this. Any suggestionswould be appreciated.-David
I was just analysing the security which can be given to different users to access respective databases. So i tried, on my local server, to deny permission to myself to access Model database. After this i am not able to connect to my local server at all. Error: Permission denied. I am using windows authentication mode.
I have also deleted the local server registration, and re-registered it, but still the condition is same. Do i need to uninstall SQL Server completly to get rid of this prob.? I also registered a new data server, and there every thing is going fine. So now what do i do to get connected to my local server.
I have a report which uses a parameter called "Interval" with possible string values of "Daily", "Weekly", and "Monthly" and I want these values to set the default values for my StartDate and EndDate parameters. However, the default value remains static. And I have tried all of the solutions found in this forum. Any suggestions?
I am using SSRS 2005. And I have tried using an iif statement for StartDate, also tried Property->Custom Code, and also tried using stored procedure to populate Start and End Dates, but none of these provide dynamic parameter setting like I was hoping for.
And I think this should work because I know that if it is not a date that it works correctly. For example if I have one parameter called country and a subsequent one named state, every time I change the country it will let me re-pick the state. But this is not working correctly here.
Any ideas on what I am doing wrong or how to fix this?
I am not able to delete a row due to the presence of its parent table. There is no circular relationship and the child has no dependencies when I verify the Delete trigger. Any advice? See error msg below:
Msg 30010, Level 16, State 1, Procedure tD_My_child_table_name, Line 43
Cannot DELETE last My_child_table_name CI because My_parent_table_name exists.
Msg 3609, Level 16, State 1, Line 1
The transaction ended in the trigger. The batch has been aborted.
With Sql Server Management Studio, while creating/modifying a table I want to specify one of its columns to store values in lower case only. Can it be done through the designer or by some other means?
