Sunday, December 10, 2006

N=?

In my last job prior to doing my current second career stint as a Consulting CFO, I had the job of understanding stuff that made my head hurt: biostatistics. There are lots of reasons why being a CFO is the second hardest job in the world--being a CEO is the hardest--but the number one main reason is that you have to understand every %^@$%&%* aspect of your business, even if it is outside your realm of expertise--experience and/or education. But a CFO (a good one anyway) has to know everything that can possible go wrong and advise the president and other executive team members of such looming dangers. It's the executive team's job to mitigate these dangers. I've always believed that the CFO is the key supporting team member who spearheads this effort.

My last "business" was disease management. I hated statistics in college. I had a combination of brilliant-but-unintelligible Asian professors (I'm not being mean, but when you are a doofus student trying to learn statistics, getting concepts from professors where English is their second language is tough--and I admire them that they have a second language and the balls to teach in it) to southern-backwater "there is nothing worse than bringing an unwanted child into this world" teaching me mean, median, mean, confidence intervals and n='s. Trust me, I'm not levying any judgment, but as a 18 year old college student, it was tough to garner any sense of "this is going to help me in business life". So young. So stupid.

Dial forward 23 years. What can I say but, "Wrong. Wrong. Wrong. " Public accounting....statistical sampling--that was just 3 years out, and they gave you a sheet that any 12 year old could fill out. After public accounting, I didn't use it, didn't need it. However, had I worked for some Fortune 500 Company that used Deming models for quality and the like, I'd have to slog through it again. I was in middle market, so no statistical models for me until.....2001. Selection bias and regression to the mean were dancing in my head at night as I (with my team) were negotiating multi-year, multi-million dollar disease management contracts. One of my favorite sayings to staff members and colleagues (just to make sure they understood the magnitude of my question), "If you had to bet the farm, would you state that (fill in the blank)?" These were not just ordinary contracts; these were 100%-fees-at-risk-if- you-were-wrong contracts (and you are a sub of a Fortune 500). Horrific, wake-up-in the middle-of-the-night-heart-thumping stress. When you are making "bet-the-farm" decisions you have to ask the questions of team members in "bet-the-farm" vernacular, just so that there is no misunderstanding regarding the import of the question, OR the responsibility that you are levying on them. Of course, you give those valuable but squirming team members the option of saying "I don't know."

My point is that when we look at the "no-recession-in-history-but....(fill in the blank) had a (fill in the blank)"....you have to understand that the "n"---the number of occurrences-- is so f'in (I was in advertising and the vernacular at the time was such that a sentence without the "f word" was incomprehensible to staffers younger than 30) small that there is no way that these relationships have any statistical validity.

An "n" of 10, or whatever the number (and apologies that I have not done my homework to give you "n") has no statistical significance. The "n" is not large enough to give you a high degree of confidence. It's anecdotal. So while yield curves or my spouting off Zweig observations or any of the stuff that "talking heads" might espouse MIGHT sound weighty and convincing, these things are really not. The "n" is too small to have a high degree of confidence. That doesn't mean that there is not a high correlation, it just means that the "n" is not high enough for you to have certitude.


P.S. I'd happily be taken to task for any utterance above, though please provide a rational explanation that fits within the confines of accepted statistical theory.

1 comment:

russell1200 said...

A friend of mine is a number crunching financial academia guy. I asked him about Shiller's statement that there is not enough data points in the US market to even know if "reversion to mean" is applicable. He said that that is absolutely correct.

Most of the better studies use "all western economies" rather then US data alone. But even they tend to say things like: "in the past this is what happened", rather then state: "when x happens, y has z probability of occurrence". When you see things like "there is a 45% chance of a recession in 2007" its called f'n "marketing.

If you shift off the market and look at the economy as whole, non-linear relationships in cause and effect drive the models crazy.

An interesting approach is the merger of physics and economics referred to as econophysics. Some interesting papers can be found here (tabs at bottom of page):

http://www.unifr.ch/econophysics/

A lot of heavy math, but often the summaries are quite comprehensible.