The Neural Von Mises Mixture Model
Or: how to randomly draw angles from a neural network-generated angular distribution
Hello fellow datanistas!
You’ve probably heard of how there are waves of neural network-based protein design tools coming out over the past few years: AlphaFold, RosettaFold, ProteinMPNN, OpenFold, FoldingDiff, and more. On this theme, I just did a deep dive into a fascinating paper titled "The Continuous Language of Protein Structure" by Billera et al., which I'm eager to share with you.
This work explores autoregressive generation of protein sequence backbones. It’s like Transformers to generate text except with a twist: The essence of this paper, and what I've unpacked in my latest blog post, revolves around the innovative use of autoregressive models for generating not just discrete outputs, but continuous values that model the very angles making up the backbone of proteins. This approach leverages the von Mises mixture model in a way that's both ingenious and looks promising for the future of bioinformatics and protein design. I was deeply curious about both this idea and the von Mises distribution, so I decided to take a deep dive into one part of the paper, the “neural von Mises mixture model”, just to see how it works.
Wait, what? “Neural”, I get, and “mixture model too”, but what is this “von Mises” thing? Curious to see more? You can read the full post here. :) It turns out to be simpler than one might think, and I can’t wait to hear from you what you think!
Cheers,
Eric