Metric Development

A key component to environmental benefits analysis is development of metrics to evaluate achievement of restoration objectives from both ecological and societal perspectives. Metrics are herein defined as measurable system properties that quantify the degree of objective achievement. Moreover, a metric should measure the level of performance, raise awareness and understanding, measure progress toward programmatic goals and objectives, and support decision making. Generally speaking, metrics can be quantitative (e.g., length), semi-quantitative (e.g., big, bigger, biggest), non-quantitative (e.g., color) or nominal (e.g., yes or no); however, USACE policy requires restoration projects use metrics that are “expressed quantitatively” (USACE 2000, ER 1105-2-100, Planning Guidance Notebook ).

Types of metrics
Ideally, a metric (or metric set) is applied throughout a project life cycle (i.e., from reconnaissance to operation). However, metrics often evolve as a project moves from preliminary site screening to alternative selection to post-project success monitoring in order to meet shifting needs. There are five general types of metrics common to restoration projects:

• Outcome metrics: At the most detailed level of analysis, planning objectives should be specific, measurable targets that highlight desirable physical, ecological, economic, or social outcomes of a project (e.g., mean July temperature less than 300C). Outcome metrics measure these objectives with the highest resolution of information. Outcomes are generally parameters being manipulated by an alternative and forecasted over future with and without project conditions.
• Output metrics: Although outcomes provide significant data and information, they must be combined into an output metric for cost-effectiveness and incremental cost analyses .
• Decision factors: In addition to outputs of a project, other decision factors likely influence the selected alternative, including: thresholds in output with incremental investment, uncertainty in alternatives , risk of failure, capacity to reverse or adaptively manage the decision, acceptability to stakeholders, and myriad other “intangibles” that may or may not be quantifiable.
• Performance measures: Project performance must be assessed relative to the original planning objectives of the study and/or focus on triggering adaptive management actions.
• Process metrics: In addition to project goals and objectives, the team or agency may have objectives (and metrics) associated with how the project is conducted.

Metric Development
Following development of a complete and clear set of objectives, metrics may be identified to evaluate those objectives and inform restoration decision making. The following three-step process can be used to structure metric set development.



Step 1: Metric Selection. Given that metrics are mapped to specific project objectives, no complete list of universal metrics can exist; however, one need not reinvent the wheel for each project. Exhaustive lists of metrics for various ecosystems exist and provide excellent starting points for metric selection. Conceptual models, reference ecosystems, past projects with similar (albeit not identical) objectives, and subject matter experts are also good sources of appropriate metrics, methods, and available data. At this stage, metric sets should be as comprehensive as possible since metrics will be screened and removed in the evaluation conducted below. That having been said, preference should be given to direct over indirect metrics.

Step 2: Metric Evaluation. Once a metric or metric set has been selected, metrics may be evaluated based on whether the metric set adequately addresses project objectives and meets desirable qualities of metrics. The goal of metric evaluation is to create the most parsimonious metric set (i.e., the simplest possible, but no simpler). Six fundamental qualities of a “good” metric set are:

1. Relevant – The metrics address project objectives and priorities and are scientifically valid at a temporal and spatial scale appropriate to project decisions.
2. Unambiguous – The metrics clearly measure the consequences of different alternatives, reveal direction of preference, and have minimal natural and computational uncertainty.
3. Comprehensive – Taken together, the metrics address the full suite of project objectives and potential consequences.
4. Direct - The metrics directly quantify project outcomes. If aggregate scores or indices are used, these are constructed and documented in a way that supports direct and clear interpretation in terms of project effects.
5. Operational – The metrics can be developed, forecast, and potentially monitored within the time, labor and budget limits for the project.
6. Understandable – The metrics can be communicated in plain language to decision-makers and the public.

Step 3: Metric Documentation. The final step in metric development is an obvious, but often overlooked issue, documentation and archival. Metric documentation is critical understand why a metric was used, what objective(s) it addressed, what logic was used to develop the metric, what support exists for use of the metric, what the assumptions and limitations are, how professional judgment is or is not used in the metric, what techniques should be applied to forecast or monitor the metric, and a host of other supporting information. Furthermore, metric documentation and data should be archived for future use in restoration projects to avoid using ineffective techniques or repeating mistakes.