Many countries run substantial inflations. They do this despite the overwhelming theoretical arguments (and empirical evidence?) showing that price stability (or even mild deflation) is "better." This paper provides a simple model in which a government sometimes uses the inflation tax because it is the only way to balance its budget. Thus, inflation occurs, at least under some policy regimes, not as an outcome that is chosen in preference to other options, but as the only feasible alternative. This possibility of 'fiscal desperation' is created by limiting the government's options. Government spending is exogenous and stochastic. In particular, it is a first-order Markov process with two states, high and low. The government has two sources of revenue, a flat-rate tax on labor income and the inflation tax. The key assumption is that the tax on labor income cannot vary over time. The government can issue debt, however. Thus, when spending is high, it can use the inflation tax immediately or it can run deficits. If the (cumulative) debt gets too large, however, it may be forced to use the inflation tax. This feature is reminiscent of Sargent and Wallace's "Unpleasant Monetarist Arithmetic." The paper has three goals: (a) to characterize feasible tax/debt/inflation policies for fixed spending levels; (b) to characterize the optimal policy (or policies) for fixed spending levels; (c) to show that optimal policies involve, at least occasionally, high rates of inflation. Regarding point (c), the idea is not to show that inflation is a good policy. Rather, the goal of the paper is to explore constraints on government policy (here it is an inability to raise the wage tax) that make inflation the 'fiscal policy of last resort.