Anderson mixing (AM) is a useful method that can accelerate fixed-point iterations by exploring the information from historical iterations. Despite its numerical success in various applications, the memory requirement in AM remains a bottleneck when solving large-scale optimization problems in a resource-limited machine. In this talk, I will discuss our work on a short-term recurrent AM method that significantly reduces the computational burden. Various experiments on network training will validate the effectiveness of the proposed method. Finally, I will introduce the extension of AM to minimization problems on Riemannian manifolds.