Citrus canker assessment data were used to investigate effects of using the Horsfall-Barratt (H-B) scale to estimate disease compared to direct estimation to the nearest percent. Twenty-eight raters assessed each of two-hundred infected leaves (0-38% true diseased area). The data were converted to the H-B scale. Correlation (r) showed that direct estimates had higher inter-rater reliability compared to H-B scaled data (r = 0.75 and 0.71 for direct estimates and H-B scaled data, respectively). Lin's concordance correlation (LCC, ρ c ) analysis showed individual rater estimates by direct estimation had better agreement with true values compared to H-B scaled data. The direct estimates were more precise compared to H-B scaled data (r = 0.80-0.95 and 0.61-0.90, respectively), but measures of generalised bias or accuracy (C b ) were similar for both methods (0.38-1.00). Cumulative mean disease and cumulative variance of the means were calculated for each rater on a leaf-by-leaf basis. Direct estimates were closer to the true severity 59.5% of the time, and to the cumulative true sample mean 53.7% of the time, and to the cumulative true sample mean variance 63.6% of the time. Estimates of mean severity for each leaf based on estimates by 3, 5, 10, 20 and 28 raters were compared to true disease severity. LCC showed that rater-means based on more raters had better agreement with true values compared to individual estimates, but H-B scale data were less precise, although with means based on >= 10 raters, agreement was the same for both assessment methods. Magnitude and dispersion of the variance of the means based on H-B scaled data was greater than that by direct estimates. H-B scaling did not improve reliability, accuracy or precision of the estimate of citrus canker severity compared to direct visual estimation.