Study analysis of "West et al. (2020): Overstated carbon emission reductions from voluntary REDD+ projects in the Brazilian Amazon"

In the wake of the accusations made by The Guardian and DIE ZEIT against Verra-certified forest conservation projects and the REDD+ mechanism, ClimatePartner has subjected the cited sources to a detailed analysis. ​Please see below the analysis of "West et al. (2020): Overstated carbon emission reductions from voluntary REDD+ projects in the Brazilian Amazon".

Other studies reviewed by ClimatePartner are Guizar-Coutiño 2022 and West et al. 2023. ​ ​

A summary assessment of the study comparison as well as links to statements by other market participants are available here.

Study brief

Authors: Thales A.P. West, Jan Börner, Erin O. Sills, Andreas Kontoleon

​Publishing details: September 2020​,  “Proceedings of the National Academy of Sciences” Vol. 117, No. 39

Method of West et al. (2020):

• A synthetic control method is used to define counterfactuals and assess reduction in forest loss. This is applied by systematic comparison of deforestation baselines (ex-ante) with counterfactual constructed deforestation (ex-post).​ ​
   
• The synthetic control method applied in this study uses biophysical data to identify comparison areas, and then constructs counterfactuals based on similar biophysical characteristics found in project areas. This means that comparison areas can incorporate contemporary drivers of deforestation. ​ ​
   
• Geo-referenced property boundaries (held in Brazil’s Rural Environmental Registry) are used to construct synthetic controls. ​ ​
   
• The method is applied to 12 VCS REDD+ projects in the Brazilian amazon from 2008 to 2017, which are mentioned in the study.​

Projects evaluated in West et al. (2020)

12 projects were analysed, of which 4 are or have been in ClimatePartner’s portfolio.

_{Please note: Projects highlighted in bold are or have been part of ClimatePartner’s portfolio.​}

_{*For these projects definitive conclusions can not be drawn. We do not have access to the raw data used in the studies, but can only see the graphics published in the paper. From the graphs we can only see that these projects performed sometimes better and sometimes worse than their comparison area. As no average is mentioned in the study, we thus marked them yellow and “with unclear results”.}

Main findings of West et al. (2020)

• The study finds no significant evidence that the REDD+ projects have mitigated forest loss and no systematic evidence that the offsets claimed are additional reductions.​ ​
   
• Leakage (shifting deforestation to neighboring areas) could have occurred in three projects (Maísa, Florestal Santa Maria, and Manoa)​.
   
• The study questions offsets calculated using deforestation counterfactuals based on the continuation of historical trends (the study uses biophysical data).​ ​
   
• The authors state the need to reassess the current approach to measuring additionality, as their findings indicate that using historical data has limitations. ​ ​
   
• The study proposes that periodic baseline updates based on recent deforestation trends could help mitigate the current limitations. Verra currently requires this at the latest every 10 years. West et al. argue that this should be more frequent. Baseline updates could be based on control areas that share similar characteristics to the project.​ ​
   
• The study sees the following as potential reasons for limited impact of the projects:​
  • Projects are not adopting the most effective actions to achieve their objectives​
  • Difficulty with on-the-ground implementation and execution of activities planned​
  • Challenges with commercialisation of carbon offsets and corresponding limited revenues for activities.

Shortcomings of the study, as identified by West et al. (2020)

• West et al. acknowledge that synthetic controls do not perfectly match REDD+ project areas in terms of size, accessibility, and biophysical characteristics and may not have included all relevant structural determinants of deforestation​.​
   
• The authors also state that the period of analysis of the projects' running time may not have been long enough to observe significant impacts in some cases​ ​
   
• Shapefiles available on the public Verra registry did not always match the actual project boundary​. ​

Verra’s Technical Review on West et al. (2020) and West et al. (2023)

Verra finds that the studies by West et al. (2020) and West et al. (2023) are “patently unreliable” and identify a number of deficiencies in their methodologies:

• Ignoring key factors of deforestation: No consideration of other criteria including forest type, geography, economic activity, agricultural practices, and other socio-economic factors.
   
• Comparison of projects with likely unsuitable reference areas: Selection of areas that likely faced zero to a very low threat of deforestation.
   
• Insufficient number of comparative areas: The small sample size of reference areas leads to larger uncertainty of the results.
   
• Use of unsuitable datasets: The datasets used in both studies were either found to be to crude (2020), or relied on datasets that are known to be unfit for this type of analysis without consideration of additional land-use datasets (2023).

Additional comments by ClimatePartner

• The synthetic control method ignores the unique circumstances of each REDD+ project.​ ​
   
• The synthetic controls (defined control areas) are often located in a different region of the country from the project area. This synthetic data remoteness thus removes the effect of localised deforestation drivers.​ ​
   
• The authors used biophysical datasets that are either not granular enough, not accurate enough, do not provide good temporal coverage, or are not site-specific to depict actual local circumstances, which is needed for a detailed assessment. Some of the data used in this study (e.g., PRODES, TerraClass, MapBiomass) additionally does not provide data on a yearly basis or with sufficient accuracy.