Multi-Agent Reinforcement Learning (MARL) has been gaining popularity in stochastic applications. As a result, Markov Perfect Equilibrium (MPE), a fundamental solution concept in Stochastic Games, has received substantial attention and effort in its computational forefront. We present an approximate algorithmic approach for solving MPE and proves its PPAD-completeness. This approach has the potential to enhance the computational efficiency of MPE and enable its application in a wider range of stochastic games.