Wednesday, 27 August 2008

Where there should be outrage, there is silence

This chart should produce howls of outrage. It compares the retail price index (RPI) to the consumer price index (CPI) since 1997, when the Bank of England gained its independence.

The chart illustrates how the RPI measure of inflation has outstripped the CPI counterpart. Today, the RPI is 13 percent higher than the CPI compared to 1997. Why should this be? After all, both measures are supposed to track the same phenomenon - inflation.

There are differences between the two measures which, in a statistical sense, explain this 13 percent divergence. However, a closer examination of these differences reveal that the RPI is a far superior measure of price changes. So what are these differences?

The basket used to calculate the CPI is a much more restricted measure of consumer expenditure. The CPI excludes council tax and mortgage interest payments. Both of these exclusions are highly dubious. The council tax covers essential services such as waste disposal, libraries, and education. If we have to pay more for them because council tax payments have gone up, then it is obvious that we are worse off.

The exclusion of mortgage interest payments is also nonsense. These interest payments represent the cost of home ownership, and if interest payments go up, it is obvious that households are becoming worse off.

The CPI also includes certain financial service charges. In this area, the CPI does capture something that is missing in the RPI, but the loss of information isn't that great since these charges are typically only a small component of overall household expenditure.

The RPI also use a more reliable data source when constructing the actual consumption basket. The RPI weights comes from the expenditure and food survey. In contrast, the CPI weights come from the household expenditure survey, which is primarily used for estimating the National accounts.

The two indices also use different mathematical formulas for combining the price is collected for each item in the basket. This is quite a complex issue, but the bottom line is fairly straightforward. In practice, the CPI formula ensures that the average price for each item is always lower than or equal to the average price for the same item within the RPI. In other words, CPI prices are always lower than or equal to RPI prices. If only this were true in real life.

The RPI tries to track the expenditure of the average household. It therefore sensibly excludes households in the top 4% of the income bracket. It also excludes some pensioners who tend to have highly atypical expenditure patterns. The CPI includes these groups.

As an aside, both measures of inflation are subject to outrageous Hedonic price adjustments. Actual inflation rates are reduced on the basis of arbitrary adjustments that supposedly capture the improved quality of goods. These changes are extremely widespread and have greatly undermined the validity of recent price data.

These differences may seem like an arcane issue that should only concern professional statisticians. Unfortunately, this issue really matters to all of us. The government chose CPI as the inflation target for the Bank of England. Since it produces consistently lower inflation rates, this target is a fix. It is therefore all the more shocking and worrying that the central bank have been unable to meet this relaxed and distorted target.

Measures of inflation based on the CPI are totally unreliable. We all know this to be true. The proof is hammered home every time we reach the supermarket tills and open up our purses. Prices are rising far faster than the headline rate suggests. This should be no surprise, the CPI has been designed to understate the true extent of inflation.


Anonymous said...

No outrage?

Maybe you should go to the pub!

electro-kevin said...
This comment has been removed by the author.
electro-kevin said...

Nu Lab's biggest lie.

Low inflation.

House inflation meant our children and young workers getting poorer - the hidden story of Brown's 'boom' times.

Anonymous said...

Maybe you are allowing your protest to be defined by others. Neither the CPI or the RPI are measures of inflation. They are measures of prices.
Inflation is an increase in money supply.
Look at credit expansion/contraction. You seem to state that CPI is totally unreliable yet continue to refer to CPI in your blog. It is -only- price measurement. Nothing to do with the inflation or deflation in Sterling.
Great blog as usual.

Anonymous said...

Well for more fun and interesting graphs, plot the Price Index (not the inflation rate) against M0. Do it as 2 separate graphs the beginning til 1992 and then 1992 onwards. (When you do the graph you'll see why).

Put linear interpolations in both sections - look at the correlation.

Anonymous said...


The latest lie is 'core CPI'...

"The core CPI index excludes goods with high price volatility, such as food and energy..." (Wikipedia)

Every time something gets pricey, they move to a new measure which doesn't include it...

And if you can't exclude it, then try "Hedonic Substitution"

If they can't afford beef, they'll eat chicken; if they can't afford chicken, they'll eat fish; if they can't afford...etc. anything... they will eat grass... paper...

B. in C.

Alice Cook said...

B. in C.

I know the Americans mess around with the weights to capture the substitution effects that you highlighted, I need to see whether the ONS do the same thing. However, the ONS do use hedonic quality adjustments, which are, in my view deeply dishonest.

Core inflation is just a joke. Recently, I have noticed that the BoE have started to de-emphasize it. For example they don't actually publish a series. At least, I have never found a public series.


Alice Cook said...


It sounds like a treasure hunt, I might try it. I am going to guess that I will find no relationship post-1992.


Anonymous said...

The biggest con was no longer linking the pound / dollar to any real commodity - such as gold.

This means they have the gold in the central vaults, and the citizens get the paper promise... which tends to diminish in value, depending on how many promises are in circulation.

Doesn't seem a fair exchange, somehow!

I see the promise is still there: I promise to pay the bearer on demand twenty pounds...

Twenty pounds of what? Twenty pounds of promises, I suppose.

Hold on - that doesn't mean they promise me anything at all!!!!!

B. in C.

Anonymous said...

Using gold as a currency is barbaric. What we need is something like Keynes' bancor.

Anonymous said...

"It sounds like a treasure hunt, I might try it. I am going to guess that I will find no relationship post-1992."

Oh there is. I'll have to dig out my old spreadsheets, but from memory, it's a pretty good one too. The most revealing part is the equation of the interpolated line in the form of y=mx+c - especially when you figure out what m and c actually mean.

I'll just leave the poser: If there is no money, can there be any prices?

Anonymous said...

barbaric how?

Because any time you save you know that your value is stored? Any time you borrow you do so at a low rate and with no inflation premium? Because governments cannot print money to "balance" a budget? Because you can take your money out of a country anytime the government gets punitive? Because goods and services get cheaper as productivity increases? Because you have an incentive to save and thus increase the pool of capital available to business?


Anonymous said...


You suggest that “CPI has been designed to understate the true extent of inflation.” By whom was it designed with this aim in mind?

CPI was designed by Eurostat in consultation with the national statistical offices of the EU member states. Moreover it was designed in the mid-1990s with the intention of being applicable on a European wide basis. Are you really suggesting that 15 years ago Eurostat deliberately designed CPI so that a future Labour Government could “fiddle” the inflation figures?

Housing costs, particularly mortgage costs are excluded for two reasons:

1. CPI is intended primarily as an inflation targeting measure. Such measures usually do exclude interest payments – otherwise the target measure is raised when the central bank raises interest rates. This is why the current Government and its Conservative predecessor both used RPIX rather than RPI as their IT measure.

2. The housing market, particularly the rental market in many European states is far from being a free market. It is distorted by varying degrees of subsidy and regulation. Eurostat excluded housing costs from CPI because they were unable to come up with a methodology which could be applied Europe-wide. Since the introduction of the CPI measure, they have continued working on this problem and had hoped to include some measure of housing costs by 2010. My understanding is that this date has since slipped.

You also seem highly dubious about hedonistic adjustments. Again, I do not see why. My first car cost 14K in 1990 (although I didn’t pay the new price). My current car cost 15.6k last year. It has a full leather interior, electrically adjusted seats, electrically heated seats, parking censors, air conditioning, climate control, ABS, cruise control, 12 air bags, a six CD player with 12 speakers, and a ten year anti-perforation warranty. My first car had none of these and started to rust after 2 years (it was a Rover). If I’d wanted to bring it up to the specification of my current car I would have had to spend at least 28K and it would still have been a pile of Longbridge-built rubbish! So, if you don’t take quality improvements into account it’s impossible to make sensible price comparisons over time, as your not comparing like with like.

Young Mark

Anonymous said...

yeah, yeah, yeah

Alice Cook said...
This comment has been removed by the author.
Alice Cook said...

Young mark,

Thanks for your comments. I thought they were intelligent and insightful.

Yesterday's post focused on the systematic divergence between two measures of inflation; the RPI and the CPI. These indices measure the same thing - inflation. Yet one is 13 percent lower than the other. This is not a trivial difference and leads to an obvious question; which measure more accurately reflects changes in the price level. I think that both are flawed, but that the RPI is. overall, a better measure.

Why? As I said yesterday, the CPI excludes housing costs, whilet the RPI has an imperfect measure, mortgage interest payments. Excluding this huge item of expenditure is unacceptable. The US CPI has an imputed housing measure. So, it really isn't so difficult to do. Eurostat could have produced something but as you say, it hasn't managed it yet.

BTW, it is just a small point, but Eurostat produce the HCPI. The CPI the BoE uses is a domestic creation. Having said that, there are obvious methodological similarities.

As for house prices in Europe being controlled, well a surprising amount of prices could be considered to be "controlled". For example, in the UK, electricity and telecommunications are regulated by the state. However, that doesn't mean that these items should be excluded from inflation. At least half of the petrol price is determined by tax; likewise cigarettes and alcohol. The state effectively determines these prices, but again, it doesn't follow that these items should be removed from the index.

Then we come to hedonistic adjustments. I have at least three objections.

First, these quality adjustments confuse technological growth with inflation. We all know that some goods improve over time. This phenomenon should be confined to our GDP measure. BTW, you can go a long way to addressing your concens about quality improvment simply by excluding old products like crappy British cars and including excellent german cars. Of course, this is what the RPI has been doing for years.

Second, these adjustments lack transparency. Hedonistic prices are synthetic; generated from regression analysis. No one actually buys a hedonistic priced good. What is more, no one beyond a couple of statisticians in the ONS actually knows what this price is, or how it was estimated.

However, my third objection is my most serious one. Hedonistic adjustments only go in one direction. They only every reflect improvements in quality. Take, for example, the price of an airline ticket. A trip across the Atlantic today is far more stressful and miserable than 10 years ago. More time is spend dealing with security; the probability of baggage loss is higher, and planes are delayed more often. This quality deterioration does not find its way into the CPI.

Frankly, I can think of loads of products where quality has deteriorated in the last ten years. Here are just a few; travelling on the tube; driving a car on the M1; eating in Pizza hut; white bread (added fat to make it last longer); films (added violence) and so on.

I have only ever seen two hedonistic adjustments; one for digital cameras, the other for computers. I was really shocks how modest changes in "quality" produced enormous price changes. A slight increase in pixels, which these days can barely be seen in most photos turned a sizable price increase into a double digit price fall. Even if you are right in principle here (which I dispute), you really should be concerned at the magnitudes of the effects that hedonistic price adjustments create.

So, the CPI is biased downwards. What more there is solid statistical proof. It is the divergence relative to the RPI. What is more, it is not hard to explain this difference. Important items like housing costs; and local services are excluded. Hedonistic adjustments are becoming all to prevalent. This adjustments are fundamentally dishonest.

The result is that we are being asked to believe that there is no inflation when every day we see prices rising.

Anonymous said...


I’m afraid you’re wrong about the difference between HCPI and CPI. They are the same thing. It’s simply that the UK version of HCPI has been referred to as CPI since 2003. Obviously the weights will vary across the various member states of the EU, but the methodology doesn’t. In fact, Eurostat monitors members states’ statistical services to ensure they are complying with the HICP regulations.

You say that RPI and CPI are measuring the same thing. This is only partly true, as they are doing so for different purposes. CPI is used primarily for inflation targeting, whilst RPI is used for measuring the cost of living. So when the Government indexes benefits, it does so using RPI, not CPI. Similarly index-linked savings certificates use RPI, not CPI. For wage bargaining purposes, both sides of industry use RPI, not CPI. Your accusation of fiddling might stand up if the Government had replaced RPI with CPI, or was downgrading its importance in some way, or attempting to hide the fact that mortgage interest payments are excluded from CPI. The fact is that it hasn’t done any of these things. Even when the Government started using CPI for IT purposes, it was simply a switch between two measures, both of which excluded mortgage interest payments.

You say “Hedonistic adjustments only go in one direction” and that you’ve only ever seen two “one for digital cameras, the other for computers”. That’s because the ONS uses other techniques for making the quality adjustments for the great majority of goods. For the most part they use producer costs and option costs. Let’s say a car gets air-conditioning as standard and the price rises by £100. The ONS ask the manufacturer how much the new feature costs to make. If the manufacturer can’t supply an accurate figure, the ONS looks at how much the new feature costs as an option. One final word on quality adjustments, before we slide into a quagmire of stodgy white bread and rusting Austin Montego cars: within CPI, there are internationally agreed procedures for adjusting prices to reflect improved quality. RPI also makes quality adjustments. It, on the other hand, is a wholly British affair.

Incidentally, if Eurostat do ever devise a method of including housing costs, my guess is that we will end up with two versions of CPI: one measure excluding housing costs, to be used for inflation targeting purposes, the other with such costs included, to be used as a measure of the rising cost of living.

Young Mark

Alice Cook said...

Young Mark

I understand that they comprise of the same subindices. I was referring to the fact that the weights are different. So the HCPI and the CPI give different index values. In the sense that they have different weights, they are not the same thing.

The key point here is whether housing costs should be in the index. The answer is, of course, yes. In what sense is an increase in housing costs not inflation?

As for the hedonic adjustments being "internationally agreed", so what? I don't agree with them, and I think I have made strong valid arguments why this adjustment is flawed. As for the alternative methods, hedonic adjustments are becoming very popular. My mind was made up on this issue when I went through two of two of the ONS notes on these issues in EMLR. Once I understood what the ONS were doing, I was appalled and horrified.

Besides, you would agree that life doesn't always improve and quality adjustments should go both ways. We can agree that this makes sense. I am sure you can accept that particular weakness in the process.

Mark, you haven't addressed my key point. Why have the two indices systematically diverged. They are both supposed to measure the same thing. A 13 percent divergence is huge. It simply doesn't make sense.

You say that the two indices are used for different purposes and that the CPI is used for inflation targeting purposes. Why would you use an inflation measure that systematically understates the effect on the cost of living? That is bound to bring the whole inflation targetting exercise into disrepute. In short, I don't recognise this distinction between cost of living adjustment purposes and inflation targetting purposes.

Again, thank you for your comments. I enjoyed reading them.


Anonymous said...

"Let’s say a car gets air-conditioning as standard and the price rises by £100. The ONS ask the manufacturer how much the new feature costs to make."

And with a Government looking ever vigilantly for "wind-fall" tax opportunities, the manufacturers all fall over each other to say "The cost is really small and we make shed-loads of money on this..."

And the inquiring mind will wonder if that cost is in itself inflation adjusted.

Mind you, since wheat costs more and more to make every year, it must be so much better than wheat of even five years ago.

vado said...

The more you find out about measuring inflation, the more it looks like a conceptual morass.

I have a better idea. Forget about the CPI, target the money supply.

Anonymous said...

"I have a better idea. Forget about the CPI, target the money supply."

I've been doing a lot of "deflate by M0 rather than RPI" and it does correspond a lot better to what happens on the ground than by using RPI.

Strangely, it seems "more accurate" than even M4, and that I can't explain.

Anonymous said...


You say I haven’t addressed the reason for CPI/RPI divergence. The divergence arises for two reasons (as I’m sure you know): coverage effects and formula effects.

The main coverage effect arises from the fact that various housing-related costs (depreciation, estate agent’s fees, insurance etc) are excluded from CPI. Because house prices have been rising rapidly over the last few years, the coverage effect has been exaggerated. When the switch was made to CPI in 2003, the ONS estimated that the coverage effect was responsible for about 0.5% per annum of the disparity between CPI and RPI (it has since risen). Over the next few years house prices will drop and the disparity will diminish. I don’t deny that the coverage effect exists. I do object however when people suggest that the Government or the ONS is somehow “fiddling” the CPI figure, when no evidence exists to support such a charge.

The formula effect arises from the fact that individual prices are aggregated using the geometric mean in CPI, but the arithmetic mean in RPI. Using the geometric mean implies a substitution effect, whereas the arithmetic mean implies that people continue buying the same quantities of goods when their relative price changes. I would suggest that the former more closely represents consumer behaviour. In other words, RPI actually overstates the true rate of inflation. The ONS once again estimates the effect to give rise to a fairly stable 0.5% divergence between the two indices.

You ask why mortgage interest payments should be excluded from CPI, simply because it is an IT measure. I would suggest it is because the BoE’s only weapon in the fight against inflation is its control of interest rates. It would be perverse in the extreme if inflation rose as a direct response of the Bank’s attempt to control it. This is why the Conservatives used RPIX, not RPI as their inflation target and why the present Government carried on using RPIX until its replacement with CPI in 2003.

So long as people are aware of the differences in both composition and purpose between CPI and RPI, I see no reason why the IT regime should be brought into disrepute. Indeed, compared with previous attempts to control inflation, such as prices and incomes policies, monetary targeting and ERM membership, the current IT regime has a positively glowing reputation and unparalleled longevity.

Young Mark

P.S. Keep up the good work with your blog. It’s vastly superior and more entertaining than the drivel one reads in the mainstream press.

alice cook said...

Young Mark,

My real concern is how the CPI fits into the monetary framework, which is inflation targetting. I think that the CPI, because it expressly excludes housing, and makes those hedonic changes, allows the central bank to generate huge amounts of liquidty. This generates asset bubbles, while the BoE hides behind these really low CPI inflation rates. We have ended up with a huge inflationary bubble, whatever the CPI might tell us. Now, that it is bursting, we will have to deal with these policy errors that the CPI obscured.

I might do another post on hedonic price adjustments. I really do think that these adjustments are far to obscure and arbitrary.

Thanks for the kind words about the blog. I've enjoyed responding to your comments, and I have learnt something.


Anonymous said...


A fascinating debate, which I think we’ve probably exhausted for the moment.

I agree that there has been a bubble in asset prices, most particularly house prices. The interesting question for me is “why did some asset prices inflate so much, whilst others, notably shares, didn’t”. Maybe this could be a topic for the future.

Have a good weekend.

Young Mark.