Analytic solution to variance optimization with no short selling

File pdf 628KB

A portfolio of independent, but not identically distributed, returns is optimized under the variance risk measure, in the high-dimensional limit where the number N of the different assets in the portfolio and the sample size T are assumed large with their ratio r=N/T kept finite, with a ban on short positions. To the best of our knowledge, this is the first time such a constrained optimization is carried out analytically, which is made possible by the application of methods borrowed from the theory of disordered systems. The no-short selling constraint acts as an asymmetric l1 regularizer, setting some of the portfolio weights to zero and keeping the out of sample estimator for the variance bounded, avoiding the divergence present in the non-regularized case. However, the ban on short positions does not prevent the phase transition in the optimization problem, it merely shifts the critical point from its non-regularized value of r=1 to 2. At r=2 the out of sample estimator for the portfolio variance stays finite and the estimated in-sample variance vanishes, and a new critical parameter related to the portfoliöo weights diverges.
We have performed numerical simulations to support the analytic results and found perfect agreement for N/T<2. Numerical experiments on finite size samples of symmetrically distributed returns show that above this critical point the probability of finding solutions with zero in-sample variance increases rapidly with increasing N, becoming one in the large N limit. However, these are not legitimate solutions of the optimization problem, as they are infinitely sensitive to any change in the input parameters, in particular they will wildly fluctuate from sample to sample. With some narrative license we may say that the regularizer takes care of the longitudinal fluctuations of the optimal weight vector, but does not eliminate the divergent transverse fluctuations.
We also calculate the distribution of the optimal weights over the random samples and show that the regularizer preferentially removes the assets with large variances, in accord with one's natural expectation.

Author(s):

Imre Kondor    
London Mathematical Laboratory
United Kingdom

Gabor Papp    
Eötvös University, Budapest, Institute for Physics
Hungary

Fabio Caccioli    
University College London, Department of Computer Science
United Kingdom