Benchmarking Veterans Affairs Medical Centers in the delivery of preventive health services: comparison of methods

OBJECTIVE: To identify consistent provision of clinical preventive services, we sought to benchmark all acute care Veterans Affairs Medical Centers (VAMCs) against each other nationally on the basis of multiple evidence-based, performance measures to identify facilities performing consistently higher and lower than expected.METHODS: The 1998 Veterans Health Survey assessed the self-reported delivery of evidence-based clinical preventive services in a stratified national sample of 450 ambulatory care patients seen at each VAMC. Proportions appropriately receiving each service within the recommended time interval were calculated for 138 VAMCs. Percentile ranks for each outcome were assigned. Two approaches were used for benchmarking performance. First, a scaled score for each facility was calculated across the set of 12 measures. Second, facilities were ranked based on the sum of the percentile ranks over a range of specific high cutoffs (eg, 70-80%) and above a range of lower cutoffs (eg, 40-50%). Ranking was validated by comparing with deciles of ranks on chart audit (External Peer Review Program, EPRP) data using Kendall's tau-b and chi2 quality-of-fit test. Differences between consistently high adherence (CHA) and low adherence (CLA) facilities were compared using the Wilcoxon rank sum test on 14 VHS and 11 EPRP outcomes.RESULTS: Data from 39,939 patients (67% response rate) were examined. In combination, cutoffs of greater than 50th percentile and greater than 75th percentile rank yielded 12 of 14 VHS and 6 of 11 EPRP measures different between CHA and CLA facilities. The scaled-score approach resulted in 20 CHA and 14 CLA facilities. The sum of outcomes ranked above 50th percentile and over 75th percentile for CHA facilities (n = 17) was 15 or more. The sum of outcomes ranked above the same cutoffs for CLA facilities (n = 16) was 3 or less. EPRP and 1998 VHS data demonstrated that the survey measures and benchmarking approaches were both reliable and valid. Both approaches resulted in multiple differences between CHA and CLA facilities; differences were greater using the percentile rank approach.CONCLUSIONS: The VA has successfully encouraged adoption of evidence-based clinical preventive services throughout its health care system. However, facilities show wide variation in their levels of delivery and can be distinguished on the basis of their consistently high or low levels of adherence. Examining service delivery across multiple performance indicators allows identification of opportunities to improve clinical practice guideline implementation and the delivery of preventive services. This approach identifies model institutions where focused investigation of factors associated with consistent performance may be particularly fruitful.