GBM prediction confidence #83

Activity

1. mdabros commented on Jul 30, 2018

   

   mdabros

   on Jul 30, 2018

   Owner

   More actions

   Hi @gaop123 ,

   If you are creating a classification model, you can get the confidence of the GBM by using the PredictProbability method on the model:

   /// Single new data point 
   var observation = new double[] { 1, 2, 200, 1, 3.4 };
   var probabilityPrediction = model.PredictProbability(observation);
   
   // This is a dictionary from target value to probability/confidence
   var probabilities = probabilityPrediction.Probabilities;
   
   // targetValue of the predicted class
   var prediction = probabilityPrediction.Prediction;
   
   // This will give the probability/confidence of the predicted class.
   var predictionProbability = probabilities[prediction].

   If you need the probability/confidence for the other classes, you can get the probability by parsing their target values to the probabilities dictionary.

   Best regards
   Mads

   👍React with 👍1Reacted by pjsgsy
2. gaop123 commented on Jul 30, 2018

   

   gaop123

   on Jul 30, 2018 via email

   Author

   More actions

   Hello, Thanks for the quick response, By confidence i meant for a way to know the probability that the input is actually belongs to one of the labels, Because the probability that the gbm model returns allways sums to one there is no way to know if the label belongs to none of the labels, Thanks! בתאריך יום ב׳, 30 ביולי 2018, 8:53, מאת Mads Dabros ‏< notifications@github.com>:

   …

   Hi @gaop123 <https://github.com/gaop123> , If you are creating a classification model, you can get the confidence of the GBM by using the PredictProbability method on the model: /// Single new data point var observation = new double[] { 1, 2, 200, 1, 3.4 };var probabilityPrediction = model.PredictProbability(observation); // This is a dictionary from target value to probability/confidencevar probabilities = probabilityPrediction.Probabilities; // targetValue of the predicted classvar prediction = probabilityPrediction.Prediction; // This will give the probability/confidence of the predicted class.var predictionProbability = probabilities[prediction]. If you need the probability/confidence for the other classes, you can get the probability by parsing their target values to the probabilities dictionary. Best regards Mads — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#83 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ANFWWZ9mAmTMQxHyepCh56RvpFSE9LuKks5uLp9dgaJpZM4VlMkA> .
3. mdabros commented on Aug 2, 2018

   

   mdabros

   on Aug 2, 2018

   Last edited by mdabros

   Owner

   More actions

   @gaop123 Ahh yes, that is something different indeed. Currently, there are no build in methods for dealing with classes unknown at training time.

   There are a few ways you can handle it, for instance add an anomaly detector in front of your classifier. The anomaly detectors purpose would be to flag all observations which are different from what the classifier was originally trained on, separate these out, and only parse on the observations that seem to belong to the original data distribution.
   This is by no means a perfect method though, and it will have some draw backs, since the anomaly detector won't be 100% correct. Sometimes you will still get unknown classes sent to the classifier, and sometime the anomaly detector will filter out known classes.

   More consideration can be found here: what-image-classifiers-can-do-about-unknown-objects