Writing has always been a way for me to clarify my thoughts, and this blog serves as a repository of ideas that I may want to revisit. Some computational and mathematical tricks, from scratch implementations and key intuitions of papers that I like are and going to be the main content, I hope this collection helps others make sense of some of these just as it does for me.

Featured

• Likelihood Ratio Trick a.k.a REINFORCE

  The trick that you need when you need to differentiate through a sampling process -or in other words- through a stochastic node.

• Softmaxing is actually Utility Maximization

  In this post I show that the softmax function which is used to convert logits to probabilities, is actually utility maximization(with specific type of error on the utility).

• Gumbel-Max Trick

  Introducing the gumbel max trick and connecting it with the literature.

Recent Posts

• Log-Derivative Trick

  Just a simple chain-rule we need to encapsulate in a name.

• Softmax is actually a softer version of argmax.

  ow many different ways can we think of to interpret softmax function?

• Softmax to Gibbs

  What kind of probability distribution you get when you softmax your logits?

• Upcoming

  These posts are next!