“In reading [the Consortium document entitled] Comments Regarding “Independent Third-Party Monitor for Chautauqua Lake Macrophyte Management – 2020 Herbicide Treatment Program” I find the following comment very misleading:
Although the Report (Pg 24) describes the reduction of milfoil to be “sizeable”, it states (Pg 21), “The target species, Eurasian watermilfoil, was ‘Sparse’ during Pre-Treatment and was also ‘Sparse’ Post-Treatment.” This suggests there was no significant difference between pre- and post-treatment in Sherman’s Bay. Thus, the effectiveness of the herbicide treatment on the target species, Eurasian watermilfoil, appears to be not significant.
In the referenced Princeton Hydro report, page 24 actually states, in addition to your reference to the reduction in Eurasian watermilfoil as “sizeable”: Reductions in Eurasian watermilfoil biomass in the Treatment zone was sizeable with a mean biomass of 22.9 g/m2 reduced to 1.5 g/m2. This represented a 93.5% reduction in biomass.
I don’t understand how you could characterize a 93.5% reduction as “not significant”.
We thank you for your question. With our commitment to only verifiable statements, if our words require a further explanation, we are happy to provide such.
First, a referral to the Racine-Johnson Aquatic Ecologists’ Rake-toss Abundance Rating graph is needed. The five categories of rake-toss biomass have taken into consideration the amounts of actual collected plants. Within the category, the real values of plant biomass have been verified. The clustering of data points of varying amounts produces the Rake-Toss Abundance Rating numbers and allows various types of statistical analyses to be performed. It must be noted that the Abundance Rating numbers are numerical, yet their corresponding biomasses are logarithmic. That is, the changes in the upper end are greater than the changes at the lower end of the Rake-Toss Abundance Rating. Thus, using the Rake-Toss Abundance numbers keeps the comparisons between pre- and post-plant abundance as real biomass numbers.
The best reporting and interpreting of data are practiced when both relative and real, called absolute, information is provided. Relative numbers are dependent on other numbers while absolute numbers are real numbers. It is recommended by data reviewers that the less sensational number be presented first. Princeton Hydro (PH) followed both of these practices. Likewise, the Consortium review states both types of information and is correct to point out that the more sensational, relative percentage should not overshadow the absolute information.
This review is caught in a common misrepresentation: “Absolute changes on small numbers often look small and relative changes on small numbers often look big.” (dataschool.com) The real numbers for Eurasian watermilfoil show that 22.9 g/m2 reduced to 1.5 g/m2 creating a change of 21.4 g/m2. Because both 22.9 and 1.5 g/m2 are “Sparse” rake‐toss values (both represent small amounts of plant material on the sample rake) then this means there was no real change using the rake‐toss collection method. PH later reports the “sizeable” relative number of “… a 93.5% reduction in biomass.” This percentage is identified as “relative” because the percent change is based on the small size of the pre-treatment biomass. Consider if a “Dense” rake‐toss mean of 250 g/m2 reduced to a “Sparse” rake‐toss mean of 13 g/m2. Here, a large, real number change of 237 g/m2 occurred. It is a noticeable reduction in biomass on the rake and it could be described as “sizeable.” For this sample, the relative reduction would be reported as 94.8%. Thus, the change of 21.4 above has nearly the identical relative percentage as this change of 237.
We hope this satisfactorily explains that while the 93.5% change was correctly calculated, it should not be described as “sizeable”. Further, a reduction did occur but because it is small and it did not move the Rake-Toss Abundance Rating, it is appropriate to describe it as “not significant”.