Perspectives on statistical power analysis to evaluate the sensitivity of metrics used in multimetric rapid bioassessment protocols (RBPs)

B.D. Marshall. The Center for Biomonitoring, Inter-Mountain Laboratories, Bozeman, Montana, USA.

Selection of environmentally sensitive metrics is a key step in multimetric index development; variation caused by disturbance should exceed natural variation among reference sites. This type of sensitivity is often evaluated by visual or mathematical comparison of box-plots or interquartiles. Criticisms of these methods have cited high or unknown rates of error (type-I (α) or type-II (β)). For biomonitoring, β is at least as important as α and should be incorporated into multimetric index development. Investigators may let α = β and use power analysis to solve for the amount of detectable change (δ) to describe sensitivity. δ needs to be estimated using the same design used for RBPs (one-sample t-test) because other designs over estimate power. However, with this design, δ is a function of variation among reference streams; not a measure of the responsiveness of metrics to disturbance. Environmental sensitivity can be described by scaling δ relative to the amount of change observed at disturbed sites (Δ). The use of Δ/δ resulted in statistically insensitive metrics (e.g., %shredders) with high environmental sensitivity because the response of metrics exceeded the natural variation dramatically (Δ/δ >>1). Δ/δ estimates can evaluate the benefits of alternative designs in terms of environmental sensitivity.