by Charles Schwartz, Professor Emeritus, University of California,
Berkeley
schwartz@physics.berkeley.edu
October 2, 2000
>> This series is available on the Internet at http://socrates.berkeley.edu/~schwrtz
Introduction
At the July meeting of the Board of Regents there were a few speakers, during the Public Comment Period, who asked pointed questions and voiced sharp complaints about the regents' handling of the UCRP (Retirement Plan) investments. They cited a front page newspaper story which suggested political payoffs in the awarding of a UC consulting contract and a backstage move to oust Patricia Small, the University's Treasurer who had an outstanding record of success in managing the investments. It is rare that regents respond to these public critics; but on this occasion, Regents S. Sue Johnson, Chairman of the Board, and Judith L. Hopkinson, Chair of the Committee on Investments, issued a formal Joint Statement (July 19, 2000):
My first reaction, upon reading this Statement, was that the regents made no mention of the Treasurer and that looked like an ominous sign. Sure enough, within a month UC announced that Patricia Small was resigning. A generous severance package was given to this career public servant  28 years of service to the University, 19 years as the Regents' Treasurer and Associate Treasurer  and she has been silent since then.
"We feel compelled to respond to this morning's public comments, stemming from recent misleading news reports on UC investment policies."First, we want to assure employees that their pension funds are safe and that they will continue to be safe. Safety of the pension funds is the uppermost concern of the regents, and it will continue to be. Safety was a major impetus in deciding to undertake a comprehensive review of investment activities.
"Here is some background on the issue.
... [The full text may be found at http://www.ucop.edu /bencom/news/regentnewitem.html ] ...
"The Wilshire firm [Wilshire Associates Inc.] was ultimately selected as the most qualified. The consultant has functioned solely as reviewers, advisers and monitors, and not as investment managers. Wilshire is a firm of international repute in the investment field. Its clients include a number of large international businesses, as well as other public funds. ...
"Throughout the process, the consultants' review and recommendations, and asset allocation plan recommendation, were reviewed on a continuous basis by the Commission, the Investment Committee, the Treasurer's Office, and the full Board of Regents. The plan was unanimously approved by the Regents in a public meeting in March 2000."
My second reaction, reading the last sentence
in the regents' Statement, was this bit of cynicism: After more than a
year of meeting in secret to work it all out, the regents held one brief
open session, in March, to formally ratify the plan, and they act as if
that should satisfy the public interest. When I checked into this
later, however, I discovered that no such "public meeting in March
2000" ever took place.
On July 27, I wrote the following letter to UC's president, Richard Atkinson.
I am trying to make sense of the recent public outcry concerning the University's investment policies and performance, following the article published in the San Francisco Examiner on July 16. I have a copy of the July 19 Joint Statement issued by Regents S. Sue Johnson and Judith L. Hopkinson, but this seems to raise more questions than it answers.
Please send me copies of all reports, letters of opinion and advice, etc., fromI will return later to tell more about this quest for official documents and the continuing cloak of secrecy over this whole matter. Now, let's get to some substance.
1) the Commission to Review the Office of the Treasurer;
2) Wilshire Associates;
3) Treasurer Patricia Small;
and any other documents that may help me obtain a fully informed understanding of these matters.
Digging Into the Wilshire Report
By mid August, a friend had pointed my browser to www.ucop.edu/bencom /news/revisedasset.html, where one can download a 30page document: "Investment Strategy Study", dated March 16, 2000, addressed to The Regents of the University of California. This is the authoritative and (as far as we in the public are allowed to know) complete analysis and recommendations prepared by Wilshire Associates. I will limit myself to Section 2 of this report, pages 311, that concerns the UC Retirement Plan (UCRP), with $38 Billion in assets as of 6/30/99.
This is all about numbers. There are six tables of data (Exhibits 4 to 9) in this section of the report and plenty more numbers throughout the text. Here is Wilshire's description of their work (on page 6):
"Forecasting the longrange performance  periods exceeding 20 years  of different types of investments is a critical function of investment consulting firms like Wilshire Associates. Shortterm market predictions can vary significantly and are almost always wrong. But, fortunately, there is a much better chance of forecasting the longterm return on financial markets. A number of academic studies examining the 200 year history of the U.S. financial markets have shown that equity and bond returns are more predictable over 20 year periods than any individual year. For example, while the oneyear return on the U.S. stock market has ranged from 43% to +54%, the 20year range in return is a much tighter +2% to +17%."
I have never studied Econometrics, which
is, I think, the name of the academic specialty they are talking about
in this paragraph. I do know a fair bit of mathematics,
however,
and I have the research scientist's habit of inquiry  that is, the nerve
to be skeptical, to ask questions and to poke around for possible answers.
Let me first give a couple of
definitions/reminders
for any readers who may need it.
Return on Investment: If I invest $100. in some stock
(and leave any dividends to be reinvested) and one year later this
investment
has a market value of $110., then I have realized a gain of $10., which
is 10% of my original investment. We say that the annual return on
this investment was 10%.
Compound Asset Growth: If I leave this investment
untouched
for 5 years and the rate of return continues to be 10% per year, then my
total assets at the end of this time will be
$100. (1+10%)(1+10%)(1+10%)(1+10%)(1+10%) = $100. (1.10)^{5}
= $161.05.
Here is the nub of the Wilshire report, their
Exhibit
5  UCRP Asset Allocation Policies.
A  B  C  D  
U.S.Equity  0%  48%  53%  63% 
NonU.S. Equity  0  7  7  0 
Fixed Income  100  40  35  35 
Private Equity  0  5  5  2 
100%  100%  100%  100%  
Expected Return  6.3%  8.4%  8.5%  8.3% 
Expected Risk  7.0%  11.7%  12.3%  12.4% 
The top part of Exhibit 5 shows how the total
investment is to be divided among four different asset classes  U.S.
Equity
(domestic stocks), NonU.S. Equity (foreign stocks), Fixed Income (bonds)
and Private Equity (venture capital partnerships, etc.)  for each of the
different policies considered, A, B, C and D (the
current
former Regents' policy.)
The bottom part of Exhibit 5 shows the results of Wilshire's calculations: the Expected Return and the Expected Risk for each of the four policies considered. Their conclusion: "Overall, Policy C is superior to the current policy because it both increases future return and reduces risk." How were those numbers for Expected Return and Expected Risk calculated? The report does not say; but in Part 2 of this series I will talk about this important question. What I want to do here is this: Accept the numbers as given by Wilshire in Exhibit 5 and ask, What does this mean?
A first (and healthy) reaction is to say, That doesn't look like much of a difference between the Expected Returns, comparing policies C and D: 8.5%  8.3% = 0.2%. How can we judge if that 0.2% difference really matters? Here is where the calculation of Compound Asset Growth comes in, especially when we look at a manyyear projection. Take the $38 Billion current assets of UCRP and multiply it by the factor (1.085) 30 times over and see what you get 30 years from now; do the same using the factor (1.083) and compare results. Wilshire does a part of this calculation and shows the result in their Exhibit 9, the relevant parts of which I reproduce below.
Exhibit 9  Projected Assets, Liabilities, etc., under Recommended
Policy C
($ billions)
Accrued
Liability 
Assets at
Market Value 

1999  $22.2  $38.1 
2000  23.9  40.7 
2005  36.1  56.6 
2010  55.5  78.8 
2015  84.8  108.4 
2020  126.6  150.4 
2025  185.5  219.7 
2029  250.0  296.1 
Actually, they do a more realistic calculation
for the growth of assets in UCRP, replacing the 8.5% return on investments
by a 7.1% annual rate to allow for the fact that the fund is continuously
making benefit payments (to me, among the thousands of UC retirees)
that amount to 1.4% per year. Here, then, is my comparison
calculation:
38.1 (1.071)^{30} = 298 for Policy C
38.1 (1.069)^{30} = 282 for Policy D.
The difference is 16 Billion dollars  not exactly chicken feed.
(Wilshire's Exhibit 9 gives the result 296 for Policy C after 30 years,
which is close enough to my number 298; strangely, they give no
comparable
numbers for Policy D.) If I said that this $16B difference amounted
to only 5% of the total assets and maybe not worth the fuss of changing
policies, I'd get laughed at as a silly absent minded professor.
But you don't need to be a professor to ask the next question: How reliable is the forecast that gave us those numbers in the first place? Nobody knows the future (I am talking about rational, not religious, things); any predictions must always be shrouded in uncertainty. How, indeed, did Wilshire arrive at those numbers (8.5% and 8.3% Expected Returns) and what uncertainty should we assign to them?
This Wilshire report does not describe what mathematics they use and no references are given. However, there are some clues one can pick up in the text; the most useful one is a single sentence (in parentheses) on page 7: "Risk is measured as the annualized standard deviation of return, a measure of return volatility."
Aha! Any scientist or engineer or anyone else who is at all educated in the subject of Statistics will recognize that phrase "Standard Deviation". This is how we describe uncertainty in using the theory of probability. So look back at the bottom row of data in Exhibit 5; take those "Expected Risk" numbers and interpret them as the Standard Deviation (uncertainty) to be assigned to each value of Expected Return. With this new information, I have a kneejerk reaction: If there is a 12% standard deviation for each of the results for policies C and D, and the difference between them (8.5%  8.3% = 0.2%) is so much smaller than this standard deviation, then it seems absurd to claim that there is any significant difference between these two policies. But this, too, is a mistake, which I will leave to the interested reader to figure out.
In order to get a clear view of what is going on here, we need a bit more mathematics. I'll redo the calculation of compound investment growth introducing some random variables to allow for the unpredictable market fluctuations that Wilshire talked about in the paragraph I quoted earlier.
UCRP Assets after 30 years = 38.1 (1+r+x_{1}) (1+r+x_{2}) (1+r+x_{3}) ... (1+r+x_{30})
where r is the net rate of increase which we used before (0.071 or 0.069 for policies C and D respectively) and the x's are a set of independent random variables which have the specified standard deviation (0.123 or 0.124 for C and D respectively). For this calculation I have to use my computer, which has a builtin random number generator; and I do the calculation many times over. Each time the computer gives me a different result for the total assets after 30 years; and this simulates the many possible futures for how the financial markets might behave over the next 30 years. I average this large amount of data and present it in a scattergraph (probability distribution) shown below.
Projection of Wilshire Associates'
Recommended
Policy "C" for UCRP  30 Years in the
Future
*
* * *
* * *
Each Point * is a Likely Future Outcome
* * *
* * * *
* * * *
* * * *
*
This is a Probability Distribution with:
* * * *
*
Average = 298
* *
* * * *
*
Standard Deviation = 207
* *
* * * * *
* *
* * * * * *
* *
* * * * *
*
*
* *
* * * * *
*
* *
* *
* * * * *
*
* * * *
* * * *
* * * * *
*
* * * * *
*
*
0 100 200 
300 400 500
600
700 800
900
Assets in $ Billions
The line

marks the Accrued Liability of $250 Billion in 2029
The average of this distribution (298) is the same as the result from our earlier calculation. (This distribution is quite asymmetrical; there is a long tail which isn't shown.) The basic thing to understand about probability theory is this: Every single point (*) shown in this graph is as likely to happen in our real (but unknown) future as any other point. There are more points clustered about the central region; and so we see that an outcome in the range between 100 and 400 is more likely to occur than a result either below or above this range. The average value, 298, is no more likely to be actually realized than is any other value in the range from 100 to 300.
To my mind, the right question to ask is this: What is the probability that UCRP, if it follows Wilshire's recommended policy C, will be able to meet its Accrued Liabilities (that means pay all future benefits it will have promised) 30 years in the future? The answer, from my calculations, is: There is a 49% probability that UCRP's Assets will fall above the required $250 Billion mark; or put another way, there is a 51% probability that it will be necessary to make further payments into the UCRP fund in order to meet its obligations in 30 years. I find it inexcusable that Wilshire Associates would not have done this calculation or that they would not include this essential piece of information in their Study for The Regents. (For an example of how this is usually done, see www.ucop.edu/regents/regmeet/sep00/507.pdf page 5.) As it stands, Wilshire's Exhibit 9 presents completely misleading "forecasts" for the unwary reader.
Naturally, I have done the calculation for the current policy D in the same way: the scattergraph looks almost identical to the one above; the numbers are: Average = 282; Standard Deviation = 198; probability that the Assets will be above 250 = 45%.
The chief conclusion is that there is a HUGE range of uncertainty in trying to predict the outcome of Wilshire's recommended policy C (or of policy D) 30 years into the future. The standard deviation seen in this probability distribution (207) is 69% of the predicted average value of the Assets (298). Compare this with the oneyear prediction, where the standard deviation was only about 11% of the expected assets (.123/1.071); and we see that the accumulated uncertainty gets larger as we try to make predictions farther into the future. This sounds commonsensical; and it is mathematically correct.
But this brings us to a serious problem. The conclusion I just stated seems to be in direct contradiction to the authoritative statement made by Wilshire Associates in their opening paragraph about Investment Assumptions, quoted earlier:
"A number of academic studies examining the 200 year history of the U.S. financial markets have shown that equity and bond returns are more predictable over 20 year periods than any individual year." (emphasis added)I can guess how a certain mathematical trick, which may be used by experts in this field, could resolve this apparent contradiction (if you know what the meaning of "are" is). But, here, again, I find that the Wilshire report appears to be very misleading to an average reader.
Conclusions to Part 1
The advantage of choosing Policy C (with its 8.5% expected return) over Policy D (with its 8.3% expected return) is placed into considerable doubt. The most relevant way to present the comparison  accepting for now everything else about Wilshire's model and input data  is to ask this question:
What is the probability that UCRP will be able to meet its Accrued Liabilities in future years without the necessity of additional payments into the retirement fund?
Here are the results I calculate, using Wilshire's numbers.
Year  Policy C  Policy D 
2010  78.4%  76.5% 
2020  57.5%  54.4% 
2029  48.7%  45.0% 
Did the regents ever get shown data like this? If the "Investment Strategy Study" by Wilshire Associates is the whole story (it is the only available one), then the answer is, No. When you, reader, look at this data now, do you feel that this makes a significant argument in favor of changing to Policy C from Policy D?
When I showed you a $16,000,000,000. difference between the 30year projections using only the "Expected Returns"  without acknowledging the uncertainty inherent in those numbers  you were quickly drawn to see the advantage of C over D. Now, with a better appreciation of probabilistics in the making of forecasts, this appears to be a much more dubious choice.
What about comparing the Expected Risk numbers (12.3% for Policy C and 12.4% for Policy D)? Wilshire cites this difference of 0.1% as part of the reason for choosing C over D. Is this really a significant difference? I have not been able to find any reason to say so. In the calculations that give the numbers shown in Table 1 above, almost all of the difference between results for C and D comes from the Returns and not from the Risk inputs.

The next installment  Part 2, which I hope
to release in about a week  will continue with critiques of more
bad math and missing logic in the Wilshire report.

I want to thank Professors D. Brillinger and S.L. Messinger for some very helpful discussions.
Readers: Your feedback is invited.