Sensitivity To The Distribution Of The Binary Dependent Variable

Hi all,



I have a data mining model that has a binary dependent variable (let's say 'Yes' and 'No'). I noticed that the model is very sensitive to the distribution of the binary dependent variable. I could not come up with an explaination. I know the model is not over/under fits and multicollinearity is not an issue. And I have enough data size to train the model.

I would expect some sensitivity but not to this extend:











Yes % in training data
Yes % in the prediction results


with DT
with LR
with NN

30.0%
6.0%
11.0%
14.0%

35.0%
12.0%
15.0%
24.0%

37.5%
20.0%
23.0%
27.0%

40.0%
28.0%
33.0%
36.0%

50.0%
43.0%
52.0%
61.0%



DT - Decision Trees

LR - Logistic Regression

NN - Neural Networks



And to make things more interesting, although the precision (% of correct 'Yes's ), true positive ratio and false positive ratio is changing; the overall accuracy (and therefore the score in the mining accuracy chart) remains almost same.



If anyone has an idea it would be highly appreciated.



Thanks in advance, Afsin