UBS and statistics

Last night I read the risk management section of UBS's 2008 annual report. What is it about some risk officers that makes them have a blind spot concerning basic statistical principles?

Take a look at this quote from p130 As UBS's VaR model uses a look-back period of five years it does not respond quickly to periods of heightened volatility as experienced in 2008. ... UBS experienced 50 backtesting exceptions in 2008 compared with 29 backtesting exceptions in 2007. Here backtesting is concerned whether a 99% 1-day VaR is exceeded. No backtesting failures had been seen from 1998 to 2006. So, in a period of about 9*250 = 2250 days, where UBS should have seen about 22 exceptions, it saw none, and in a period where it should have seen 2 or 3 exceptions it saw 50 ! What do they say about this? These results highlight the limitations of VaR ... Now come on guys! Why don't you admit that your VaR model is rubbish. You had 9 years of experience showing you the model didn't work and still you kept at it. Now it's failed the other way, and you still keep the model in your armoury?

How do UBS describe VaR? VaR is a statistically based estimate of the potential loss on the current portfolio from adverse movements ... VaR is derived from a distribution of potential losses.

The comment that VaR is statistical is a common theme in UBS's discussion. OK, I'll admit it's statistical, but UBS's use of it is lousy statistics. The way UBS describe this makes it sound like it's a reasonably OK statistical tool, but with some limitations, which they describe. This is bordering on the completely misleading, and here's why.

Using a simple 5 year historical simulation to calculate VaR means you are calculating an unconditional VaR i.e. you are not conditioning your estimate of VaR on the current market conditions. VaR is supposed to be an estimate of what you could lose tomorrow with some probability. It makes no sense to calculate the distribution of possible losses tomorrow without taking into account that we may be in a period of high market volatility. Basel II capital requirements are that we need to hold capital that is suitable for the current conditions. If we have high volatility then we need to increase our VaR immediately, not wait a few months until it starts to be a significant part of our 5 year data set.

The continued production by UBS risk management group of this VaR statistic shows that they do not understand some basic ideas of risk measurement. It is a number that has almost no useful interpretation for day-to-day risk management or for capital management. That the Swiss regulator allows this measure to be featured so prominantly in the annual report shows that they have not adopted a sufficiently rigorous supervisory role over their banking system. That similar measures are used in other banks worldwide show that there is still something seriously wrong in banking risk management.

Normal distributions

Let's get this straight! If X is Normal and Y is Normal then X+Y is Normal only if X and Y come from a bivariate Normal distribution.

Simple counterexample - X and Y are both standard Normal; X=-Y if abs(X)<1; X=Y otherwise.

One area in which I'd like to give further thought is looking at the formal (logical) language in which we can discuss risk. First order predicate calculus seems too powerful. My gut feel is that first order multiplicative intuitionistic linear logic would be suitable. In this logic, when you have riskA and riskB, then there are things that can happen that are not just the standard interactions of riskA and riskB. For a physical analogy, consider entangled photons, which can arise when we physically have a state of two photons, but there is no way in classical physics that we can model what entangled photons do. In fact, in classical physics they can't exist. Quantum physics has a similar structure to intuitionistic linear logic. The possible states with two photons "photonA and photonB" are more than you can get by just looking at the individual states of photonA and photon B - "riskA and riskB" is different from "riskA and riskB" (See the difference in the logical operator? The "and" is not the same.)

Lots more thinking to do.

What is risk?

For some reason I've been involved in discussions, over the last couple of days, as to what risk is. Some say its an event, some describe it as an emergent process from a complex organisation (Milliman), ISO31000:2009 say it's the effect of uncertainty on objectives. I think we need to make sure that we don't have a category error creeping into our thinking. A suitably generic notion of risk is that it's exposure to something that may be disadvantageous in some way. We can then start from here.

Just-in-time and manufacturing declines

What happens if lots of companies have tightened up their supply chains to cut down on inventory costs, and the economy goes into a severe recession? The shock of lower sales should be transmitted through the supply chain much more quickly than it has been in earlier bad recessions. Maybe the big, sudden drop in manufacturing output is partly driven by this factor. In which case, what we're doing is getting to the bottom of the cycle much quicker than before.

With any luck, the bounce out will be equally quick. I'm not holding my breath.

Some Boards are getting up my nose!

It is very difficult for the general body of shareholders to affect the Board in any meaningful way. While nominally being elected by the shareholders, a Board is self-propagating unless something seriously bad happens.

Despite all the best will in the world, such an environment is dynamically unstable. A good Board would ensure it has regular external and independent reviews of its structure, capabilities, and mechanisms. It would embrace robust discussions at AGMs and would welcome feedback from shareholders and a constructive dialogue (with the shareholders' mostly determining what is constructive). Because of the dangers of ossification, there would be a regular turnover of Board membership.

This process is unstable because in the absence of shareholders being able to affect the Board, any sufficiently large departure of the Board from its exacting standards causes its self-correction mechanisms to fail. Self-serving rationalisations can be made for why the independent Board review got it wrong, or the Board reviewer may not be changed regularly enough and so loses their external and independent nature (no matter how good you are, a close working relationship over a number of years destroys the independence - we have tons of psychological evidence for this in general, and I can't see why Board reviewing (and company auditing for that matter) would be any different). A small departure from good behaviour allows slightly larger departures, and so on in ever widening spiral of departure from best practice. And all the time the Board is arguing it's doing a good job, and steadily entrenching themselves - for the good of the company! And they believe it.

OK, so how to fix it. We know how - a mechanism whereby the beneficiaries of the company (the owners) can exercise control over the executive function, which is essentially by electing the Board. While I object to some of the activities of proxy advisors, they are doing a good job by getting more coordinated action by the end shareholders.

A nice piece from the FT on the level of losses from the "AAA" rated tranches of the sub-prime linked CDOs. There is still going to be a lot more losses to be revealed presumably.

http://www.ft.com/cms/s/0/2970532c-0421-11de-845b-000077b07658.html