We show that, for discrete exponential families, the sample mean of n observations does not stochastically dominate a single observation when estimating the population mean. This is in stark contrast to the case of a normal distribution.