Posts

Evaluation Metrics 101

Image
Building an End to End Machine Learning Solution is hard. It would cost you tons of resources and time. More often ML component of your project becomes relatively small compared to the other aspects of it. Nevertheless, the actual value generated by the solution is still reliant on this component. Therefore the decision on "whether to productionalize a model?" is heavily swayed by the performance of the model. But how do you measure the performance of the model even before people start using it? Trusty KPIs are not yet available, so how do you decide if a model is worthy of becoming an end to end solution? Model Performance Metrics The answer to the question above is quite simple. You test the model with questions, to which you already know the answers. Then you arrive at a number that is meaningful for the task that your model is performing. Depending on the context you may want this number to be higher or lower. Where do you get these queries and corresponding ground truth...
Post a Comment
Read more

Categorical Cross Entropy loss vs Sparse Categorical Cross Entropy loss vs Kullback-Leibler Divergence loss

Image
  Introduction How would we say if a model is good or bad? We need a measure of performance to assess the model. This measure based on the context is either minimized  or maximized to optimize the model. A measure which is minimized to optimize the model is called “Loss”. Several factors like type of the data, learning algorithm used etc. come into play when choosing a Loss function. In this post let’s explore a few Loss functions which are based on Entropy. Cross entropy Entropy is the measure of “uncertainty” in the possible outcome of random variables. In the context of classification If all the classes are equally probable then the system is of high entropy and vice versa. It is mathematically given as  What does this entropy mean in terms of assessment of a model performance. Let’s understand this with the help of the infamous CIFAR 10 dataset. This Data set contains tons of images belonging to 10 mutually exclusive classes. Here’s an example- Unique classes in Dat...
Post a Comment
Read more