Measurement noise in ESG ratings

Here’s a note from Florian Berg:

In our new working paper, “ESG Confusion and Stock Returns: Tackling the Problem of Noise”, we take a first step towards solving the problem of measurement noise in ESG ratings.

As we all know, ESG data is pretty noisy. The main reason is often that we do not know how to measure ESG issues perfectly. We then resort to proxies. For example, we would like to know how women are treated in the workforce but only observe the percentage of women in the workforce or the gender pay gap. Sometimes, we do not even want to measure those issues perfectly. In the case of discrimination, if we attempt to perfectly measure how women are treated in a company, we would run into data privacy issues. A perfect measure would signify that we need to measure all interactions and non-interaction between everyone in a company! This would be far from ideal, and hence we are willing to accept a measurement error in our discrimination indicators.

This measurement noise creates a downward bias (attenuation-bias) in correlations between financial performance and ESG indicators. Intuitively speaking, it is impossible to find the true link between two measures where one is mismeasured. In practice, a simple way of reducing the noise would be an average of ratings from different raters. Measurement errors tend to cancel each other out if they are randomly distributed. In simulations, we show that there is a much better approach than averages (or principal component analysis).

We propose an instrumental variables approach to estimate the effect of ESG performance on stock returns. We show that this link has been previously underestimated by 60% in the academic literature! Our dataset includes scores from Truvalue Labs, RepRisk, MSCI IVA, Sustainalytics, Refinitiv, Moody’s ESG Solutions, ISS ESG, and SP Global CSA.