UBS and statistics

Last night I read the risk management section of UBS's 2008 annual report. What is it about some risk officers that makes them have a blind spot concerning basic statistical principles?

Take a look at this quote from p130 As UBS's VaR model uses a look-back period of five years it does not respond quickly to periods of heightened volatility as experienced in 2008. ... UBS experienced 50 backtesting exceptions in 2008 compared with 29 backtesting exceptions in 2007. Here backtesting is concerned whether a 99% 1-day VaR is exceeded. No backtesting failures had been seen from 1998 to 2006. So, in a period of about 9*250 = 2250 days, where UBS should have seen about 22 exceptions, it saw none, and in a period where it should have seen 2 or 3 exceptions it saw 50 ! What do they say about this? These results highlight the limitations of VaR ... Now come on guys! Why don't you admit that your VaR model is rubbish. You had 9 years of experience showing you the model didn't work and still you kept at it. Now it's failed the other way, and you still keep the model in your armoury?

How do UBS describe VaR? VaR is a statistically based estimate of the potential loss on the current portfolio from adverse movements ... VaR is derived from a distribution of potential losses.

The comment that VaR is statistical is a common theme in UBS's discussion. OK, I'll admit it's statistical, but UBS's use of it is lousy statistics. The way UBS describe this makes it sound like it's a reasonably OK statistical tool, but with some limitations, which they describe. This is bordering on the completely misleading, and here's why.

Using a simple 5 year historical simulation to calculate VaR means you are calculating an unconditional VaR i.e. you are not conditioning your estimate of VaR on the current market conditions. VaR is supposed to be an estimate of what you could lose tomorrow with some probability. It makes no sense to calculate the distribution of possible losses tomorrow without taking into account that we may be in a period of high market volatility. Basel II capital requirements are that we need to hold capital that is suitable for the current conditions. If we have high volatility then we need to increase our VaR immediately, not wait a few months until it starts to be a significant part of our 5 year data set.

The continued production by UBS risk management group of this VaR statistic shows that they do not understand some basic ideas of risk measurement. It is a number that has almost no useful interpretation for day-to-day risk management or for capital management. That the Swiss regulator allows this measure to be featured so prominantly in the annual report shows that they have not adopted a sufficiently rigorous supervisory role over their banking system. That similar measures are used in other banks worldwide show that there is still something seriously wrong in banking risk management.