ASHS 2015 Annual Conference

Understanding and implementing evaluation data from vegetable trials conducted across multiple years and environments by multiple raters presents numerous challenges. In order to select new broccoli hybrids suitable for eastern production, the SCRI East Coast Broccoli Project has conducted over 32 phase I and 80 phase II trials that have included over 120 unique broccoli hybrids grown at different locations along the eastern seaboard (SC, NC, NY, ME, VA). These broccoli hybrids were evaluated for ten traits including bead uniformity, head color, head firmness, head smoothness, bead size, color, and overall quality. Initially, hybrid selections and recommendations were based predominately on the “overall quality” trait. Theoretically, this trait rating takes into account many different attributes; however, it is also likely the most susceptible to human perceptual bias. In Spring 2014, two East Coast Broccoli phase I trials conducted at the U.S. Vegetable Laboratory in Charleston SC, were concomitantly evaluated by three different raters in an effort to elucidate and account for potential rater bias. Four evaluation instruments or indices (e.g., different weighted linear combinations of traits) were proposed including an instrument which accounted for variation in overall quality relative to the more specific traits measured. Intra-class correlation (ICC), a statistic used to quantify both rater agreement and rater consistency, was used to compare methods. The evaluation instrument that accounted for human perceptual bias was shown by these experiments to significantly increase both ICC agreement and consistency between raters (P < 0.001). This improved evaluation instrument allows for both greater selectivity and precision in the analysis of multi-site and multi-rater data sets, and also for use with comparative methods such as stability or principal component analysis. Moreover, this work should be readily applied to trials of other horticultural crops.