This paper builds a dynamic model of borrowing and default to study the term structure of sovereign bonds in emerging markets. The borrower in the model can buy short and long bonds at contingent prices that reflect the timing of default events. The model generates a yield curve that is upward sloping on tranquil times as in the data. The reason is that if default events are likely in the future but not in the near term, only the long yield will be adjusted for this. However if default is a likely event in the near future the yield curve is inverted. In this case, long bonds are safer than short bonds for lenders in present value terms, because if the economy avoids the stressed period, it may repay its debt obligations in all future states. This matches the data in emerging markets bonds where in times of crises yields of shorter bonds are higher. The model also delivers that long bonds are issued primarily on tranquil times and short debt is used more heavily during crisis as in emerging markets. In the model long debt provides a good hedge against future bad shocks because the effective cost for such borrowing is lower exactly in times of high interest rates. Thus the borrower prefers in tranquil times long bonds because of the additional benefits. In addition, this effect increases borrowing incentives which in equilibrium translates into higher default probabilities. We calibrate the model to Brazil and find that the model can match various features of the data including the volatility of long and short bonds yields.