Stochastic orderings with respect to a capacity and an application to a financial optimization problem

By analogy with the classical case of a probability measure, we extend the notion of increasing convex (concave) stochastic dominance relation to the case of a normalized monotone (but not necessarily additive) set function also called a capacity. We give different characterizations of this relation establishing a link to the notions of distribution function and quantile function with respect to the given capacity. The Choquet integral is extensively used as a tool. In the second part of the paper, we give an application to a financial optimization problem whose constraints are expressed by means of the increasing convex stochastic dominance relation with respect to a capacity. The problem is solved by using, among other tools, a result established in our previous work, namely a new version of the classical upper (resp. lower) Hardy–Littlewood's inequality generalized to the case of a continuous from below concave (resp. convex) capacity. The value function of the optimization problem is interpreted in terms of risk measures (or premium principles).