Archives
All the articles I've archived.
2024 3
May 1
Likelihood Ratio Trick a.k.a REINFORCE
The trick that you need when you need to differentiate through a sampling process -or in other words- through a stochastic node.
April 1
Softmaxing is actually Utility Maximization
In this post I show that the softmax function which is used to convert logits to probabilities, is actually utility maximization(with specific type of error on the utility).
January 1
Log-Derivative Trick
Just a simple chain-rule we need to encapsulate in a name.
2022 3
September 3
Gumbel-Max Trick
Introducing the gumbel max trick and connecting it with the literature.
Softmax is actually a softer version of argmax.
ow many different ways can we think of to interpret softmax function?
Softmax to Gibbs
What kind of probability distribution you get when you softmax your logits?
2021 1
October 1
Upcoming
These posts are next!