Background: External quality assessment (EQA) uses a standard deviation index (SDI), based on a peer group, to evaluate laboratory performance. However, evaluations using peer group SDIs often have limited applicability, because they are not statistically valid unless the number of institutions in the same peer group is large. The present study proposes a statistical model for simultaneously evaluating the performance of all participating institutions, as well as the performance of instruments on the market.

Methods: By assuming that proficiency test results were affected by the manufacturer, the instrument, and the institution, the effects of those factors were estimated using a linear mixed model. We used these effect estimates to calculate manufacturer, instrument, and institution SDIs. Using simulation, we evaluated the false positive rates and efficiencies of the proposed linear mixed model.

Results: Simulations showed that the linear mixed model empirical type I error rates preserved the nominal significance level. This model was also more statistically efficient than the peer group SDI. Rates of unacceptability were lower when using institution SDI than they were when using peer group SDI. Additional outliers that could not be evaluated using the current system were detected by the institution SDI statistic. The instrument SDI statistic detected outliers among different instrument groups.

Conclusions: Institution and instrument SDIs are robust and efficient tools for EQA, and they can replace the currently used system of peer group SDI.
(J Lab Med Qual Assur 2016;38:43-51)

Keywords: Quality assurance, Laboratory proficiency testing, Linear mixed model

1. Karkalousos P, Evangelopoulos A. Quality control in clinical laboratories. In: Ivanov O, editor. Application and experiences of quality control. Vukovar: In Tech, 2011:331-60.
   
2. Belk WP, Sunderman FW. A survey of the accuracy of chemical analyses in clinical laboratories. Am J Clin Pathol 1947;17:853-61.
   
3. Klee GG, Westgard JO. Quality management. In: Burtis CA, Ashwood ER, Bruns DE, Tietz NW, editors. Tietz textbook of clinical chemistry and molecular diagnostics. 5th ed. St. Louis (MO): Elsevier Saunders, 2012.
   
4. College of American Pathologists. Proficiency testing manual. http://www.cap.org/apps/docs/proficiency_testing/2011_surveys_excel_manual.pdf (Accessed March20, 2016).
5. Ehrmeyer SS, Laessig RH. External proficiency testing (interlaboratory quality control). In: Cembrowski GS, Carey RN, editors. Laboratory quality management:QC and QA. Chicago: American Journal of Clinical Pathology, 1989.
6. Carey RN, Cembrowski GS, Garber CC, Zaki Z. Performance characteristics of several rules for self-interpretation of proficiency testing data. Arch Pathol Lab Med 2005;129:997-1003.
   
7. Cembrowski GS, Hackney JR, Carey N. The detection of problem analytes in a single proficiency test challenge in the absence of the Health Care Financing Administration rule violations. Arch Pathol Lab Med 1993;117:437-43.
   
8. Harville DA. Maximum likelihood approaches to variance component estimation and to related problems. J Am Stat Assoc 1977;72:320-38.
   
9. Jenny RW, Jackson-Tarentino KY. Causes of unsatisfactory performance in proficiency testing. Clin Chem 2000;46:89-99.
   
10. Ehrmeyer SS, Laessig RH. An assessment of the use of fixed limits to characterize intralaboratory performance by proficiency testing. Clin Chem 1987;33:1901-2.
    
11. Ehrmeyer SS, Laessig RH. An evaluation of the ability of proficiency testing programs to determine intralaboratory performance: peer group statistics vs clinical usefulness limits. Arch Pathol Lab Med 1988;112:444-8.