We will explain a bijection between admissible representations of affine Kac-Moody algebras and fixed points in affine Springer fibers. We will also explain how to match the modular group action on the characters with the one defined by Cherednik in terms of double affine Hecke algebras, and extensions of these relations to representations of $W$-algebras. This is based on joint work with Dan Xie and Wenbin Yan.