Weakly coupled Markov decision processes (WDPs) arise in dynamic decision-making and reinforcement learning. These models are often high dimensional but decompose into smaller component MDPs when coupling constraints are relaxed. Lagrangian relaxations of WDPs that dualize linking constraints...