Study analysis of "West et al. (2023): Action needed to make carbon offsets from tropical forest conservation work for climate change mitigation"

February 1, 2023

In the wake of the accusations made by The Guardian and DIE ZEIT against Verra-certified forest conservation projects and the REDD+ mechanism, ClimatePartner has subjected the cited sources to a detailed analysis. ​ Please see below the analysis of "West et al. 2023: Action needed to make carbon offsets from tropical forest conservation work for climate change mitigation".

Other studies reviewed by ClimatePartner are West et al. 2020 and Guizar-Coutiño 2022. ​ ​

A summary assessment of the study comparison as well as links to statements by other market participants are available here.

Study brief

Authors: Thales A. P. West, Sven Wunder, Erin O. Sills, Jan Börner, Sami W. Rifai, Alexandra N. Neidermeier, Andreas Kontoleon​ ​

Publishing details:​ January 2023 (not yet published in a scientific journal)

​Method of West et al. (2023):

  • A synthetic control method is used to define counterfactuals and assess reduction in forest loss. This is applied by systematic comparison of deforestation baselines (ex-ante) with counterfactual constructed deforestation ex-post. ​ ​
     
  • Synthetic controls were constructed based on control areas exposed to similar levels of deforestation pressure and similar characteristics as the chosen REDD+ projects. ​ ​
     
  • To define control areas, the authors created circular polygons of the same size as the project area across the project country. The West et al. (2020) study, in contrast, was able to use Brazil’s registries. As this was not possible for all project countries in this study, this new approach was applied. ​ ​
     
  • The method is applied to 27 REDD+ projects from 6 different countries.
     

Projects evaluated in West et al. (2023)

27 projects were analysed, of which 7 are or have been in ClimatePartner’s portfolio.

Please note: Projects highlighted in bold are or have been part of ClimatePartner’s portfolio.

Main findings of West et al. (2023)

  • 6 of the evaluated project areas showed evidence of higher reductions in deforestation compared to their control areas.​ ​
     
  • Implications for carbon offsets generated by data from this study:​ ​
     
    • Of the 27 projects analysed, 89 million carbon offsets could have been generated by 18 projects (based on their baseline deforestation rates) until 2020. 9 projects were discarded during the course of the study as unfit. ​ ​
    • 71% of the 89 million carbon offsets would have originated from projects that have not significantly reduced deforestation compared to their control area. 29% would have been generated from projects that are associated with some avoided deforestation, albeit not to the extent expected by the developers.​ ​
    • If the ex-ante baselines of the projects are replaced with the deforestation from the control sites, only 6.2% of the 89 million ex-ante offsets would have likely been additional. ​ ​
       
  • When properly selected, synthetic control methods can capture contemporary changes to deforestation drivers. Adopting dynamic approaches would likely reduce additionality problems. Verra may include this in the future.​ ​
     
  • Only a minority of the evaluated projects significantly reduced deforestation compared to the control sites.

Shortcomings of the study, as identified by West et al. (2023)

  • West et al. acknowledge the following challenges of dynamics methods:
    ​
    • Control areas may not always be available and/or could be manipulated​.
    • Dynamic baselines may not capture all relevant determinants of deforestation​.
    • Long-lasting projects may outlive control sites needed for their baseline calculation​s.
       
  • West et al. (2023) also reference the Guizar-Coutino et al. (2022) study as reporting significant reductions in deforestation rates on average, which is in line with their findings for Peruvian and African projects, however these were statistically insignificant. This may stem from a lower sample size used. ​ ​
  • The authors see that defining the control areas is a key challenge with counterfactual analysis.

Verra’s Technical Review of West et al 2020 and West et al 2023

Verra finds that the studies by West et al. 2020 and West et al 2023 are “patently unreliable” and identify a number of deficiencies in their methodologies:

  • Ignoring key factors of deforestation: No consideration of other criteria including forest type, geography, economic activity, agricultural practice, and other socio-economic factors​.
     
  • Comparison of projects with likely unsuitable reference areas: Selection of areas that likely faced zero to a very low threat of deforestation​.
     
  • Insufficient number of comparative areas: The small sample size of reference areas leads to larger uncertainty of the results​.
     
  • Use of unsuitable datasets: The datasets used in both studies were either found to be to crude (2020), or relied on datasets that are known to be unfit for this type of analysis without consideration of additional land-use datasets (2023).

Additional comments by ClimatePartner

  • Jurisdictional baselines might pose a solution, which is also proposed in the study and is suggested by Verra. ​ ​
     
  • The synthetic control method ignores the unique circumstances of each REDD+ project.​ ​
     
  • The synthetic controls (defined control areas) are often located in a different region of the country from the project area. This synthetic data remoteness thus removes the effect of localised deforestation drivers.​ ​
     
  • The authors used biophysical datasets that are either not granular enough, not accurate enough, do not provide good temporal coverage, or are not site-specific to depict actual local circumstances, which is needed for a detailed assessment. Some of the data used in this study (Global Forest Change) additionally is not suitable for assessing precise deforestation rates.