The Goldilocks Principle of Cooperation : Understanding Federated Learning in Healthcare via Iterated Prisoner's Dilemma

Limited access to large-scale data is one of the key obstacles to building machine learning and artificial intelligence applications in healthcare, partly due to a reluctance of information exchange among healthcare institutions out of privacy and data security concerns. To address this issue, federated machine learning techniques have been proposed that enable decentralized model training via an orchestrating platform. Despite its superior privacy protection property, a lack of systematic understanding of the economic incentives and strategic trade-offs in federated learning deters its wide adoption in practice. In this paper, we use an iterated prisoner's dilemma (IPD) framework to characterize the decisions - to share information or not - faced by participants (e.g., healthcare institutions) in a federated learning partnership, and to derive boundary conditions under which stable information exchange can arise. We find that cooperation outcome depends on the participants' cost sensitivities, information acquisition capacities, and their temporal preferences. Stable cooperation requires that participants are not too cost averse and collect not too much information per round during the federated learning process. Interestingly, it also requires participants to be neither too myopic nor too forward-looking, which is different from the conventional IPD results (where cooperation only requires non-myopic players). We further examine the cooperation dynamics (1) when participants differ in several key characteristics and (2) when non-cooperative behaviors take other forms (e.g., by sharing partial or poisoned information). Our work informs the design of contracts and incentive mechanisms for federated learning that promote private and secure information exchange in healthcare