The aim of this paper is to analyze the sensitivity of Value at Risk (VaR) with respect to portfolio allocation. We derive analytical expressions for the first and second derivatives of the Value at Risk, and explain how they can be used to simplify statistical inference and to perform a local analysis of the Value at Risk. An empirical illustration of such an analysis is given for a portfolio of French stocks.