When experts disagree: the need to rethink indicator selection for assessing sustainability of agriculture

Evelien M. de Olde*, Henrik Moller, Fleur Marchand, Richard W. McDowell, Catriona J. MacLeod, Marion Sautier, Stephan Halloy, Andrew Barber, Jayson Benge, Christian Bockstaller, Eddie A.M. Bokkers, Imke J.M. de Boer, Katharine A. Legun, Isabelle Le Quellec, Charles Merfield, Frank W. Oudshoorn, John Reid, Christian Schader, Erika Szymanski, Claus A.G. SørensenJay Whitehead, Jon Manhire

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review


Sustainability indicators are well recognized for their potential to assess and monitor sustainable development of agricultural systems. A large number of indicators are proposed in various sustainability assessment frameworks, which raises concerns regarding the validity of approaches, usefulness and trust in such frameworks. Selecting indicators requires transparent and well-defined procedures to ensure the relevance and validity of sustainability assessments. The objective of this study, therefore, was to determine whether experts agree on which criteria are most important in the selection of indicators and indicator sets for robust sustainability assessments. Two groups of experts (Temperate Agriculture Research Network and New Zealand Sustainability Dashboard) were asked to rank the relative importance of eleven criteria for selecting individual indicators and of nine criteria for balancing a collective set of indicators. Both ranking surveys reveal a startling lack of consensus amongst experts about how best to measure agricultural sustainability and call for a radical rethink about how complementary approaches to sustainability assessments are used alongside each other to ensure a plurality of views and maximum collaboration and trust amongst stakeholders. To improve the transparency, relevance and robustness of sustainable assessments, the context of the sustainability assessment, including prioritizations of selection criteria for indicator selection, must be accounted for. A collaborative design process will enhance the acceptance of diverse values and prioritizations embedded in sustainability assessments. The process by which indicators and sustainability frameworks are established may be a much more important determinant of their success than the final shape of the assessment tools. Such an emphasis on process would make assessments more transparent, transformative and enduring.
Original languageEnglish
Pages (from-to)1327-1342
Number of pages16
JournalEnvironment, Development and Sustainability
Issue number4
Early online date11 May 2016
Publication statusPublished - 1 Aug 2017


  • Indicator selection
  • Multi-criteria assessment
  • Ranking
  • Sustainability assessment
  • Temperate agriculture


Dive into the research topics of 'When experts disagree: the need to rethink indicator selection for assessing sustainability of agriculture'. Together they form a unique fingerprint.

Cite this