We propose a simple approach for approximating small sample distributions based on the method of minimum relative entropy density. Unlike previous studies that use this method as a curve-fitting tool, we focus on the 'small sample asymptotics' of the approximation to distributions of general...