Proficiency Testing

Proficiency Testing is integral to the successful operation of a Shared Resource.  It is how we can be confident that our results are similar to those of multiple other laboratories, both nationally and internationally, when identical cells or analytes are assayed.

DartLab technologists have successfully participated in Proficiency Testing and Assay Harmonization since 2005 as part of the Cancer Immunotherapy Consortium (Table I). Elements supporting DartLab's superior assay performance include quality-enabling laboratory infrastructure, specifically well-trained and qualified technologists, the use of defined standard operating procedures (SOPs), and the use of calibrated and maintained instrumentation.

IMMUNOASSAY

PARTICIPATION

IFNgamma ELISPOT

2005, 2006, 2007, 2008, 2009, 2010, 2011, 2012

HLA-multimer staining of human CD8 T cells

2007, 2009, 2010

Intracellular cytokine staining of human T cells

2007, 2009, 2010

Human T cell lymphoproliferation

2007 (not offered since)

Luminex multiplex cytokine analysis

2010, 2011, 2012

Flow cytometry gating strategies

2010

Assay Harmonization

The inherent heterogeneity and variability associated with essentially every aspect of individual laboratory-specific protocols (e.g., reagents, instrumentation, operators, and training) precludes an objective comparison of data across sites.  An alternative approach that facilitates the comparability and integration of data across multiple laboratories is “assay harmonization”.

The harmonization process involves the participation of multiple laboratories in a consortium-based iterative testing process to identify the variables crucial for assay performance.  Essential components are assay proficiency panels in which individual laboratories perform parallel quality control experiments on replicate samples.  Each participating laboratory uses its own reagents, instrumentation, and protocols.  A central laboratory manages logistics for the proficiency panel, receives raw and analyzed data sets from each participating laboratory, and provides independent central data analysis.  During initial proficiency panels, variables that impact laboratory performance are identifed.  Subsequent panels harmonize use of performance-stabilizing variables across laboratories.  The outcome of this iterative process is a set of harmonization guidelines, which can be implemented both by the panel participants and by the larger scientific community. Harmonization can be applied to all assay variables from the protocol and raw data acquisition, processing, interpretation, and reporting, so that comparison of data between laboratories becomes objective and supportive of immune biomarker research (van der Burg et al., 2011).

These harmonization guidelines allowed us to identify relevant variables that impacted assay performance and to improve these variables within our SOPs without the need for assay protocol standardization. These independent and harmonized data sets from multiple laboratories facilitated the establishment of assay-specific reference values for background, lower limits of detection, and replicate variation, which serve as community-wide benchmarks for assay performance (Moodie et al., 2012). The primary outcome of harmonization is that data sets generated across multiple harmonized laboratories are directly interpretable with regard to test performance. The ultimate objective for assay harmonization is to accelerate therapeutic development of novel immunotherapy agents and to support the identification of relevant assays to generate biomarker data that correlate with clinical outcome. In the field of cancer immunotherapy, assay harmonization is an essential component of a new methodological framework that has the potential to enable accelerated clinical development and higher probability of success for new agents (van der Berg et al., 2011).

The results of proficiency testing have been published in important assay harmonization papers (see below), and have culminated in proposed Minimal Information About T cell Assays (MIATA) (Janetzki et al., 2009) and www.miataproject.org. These are minimum criteria for reporting immune-monitoring experiments and results in order to facilitate more objective data interpretation and meta-analysis.

 

Britten CM, Janetzki S, Ben-Porat L, Clay TM, Kalos M, Maecker H, Odunsi K,
Pride M, Old L, Hoos A, Romero P; HLA-peptide Multimer Proficiency Panel of the
CVC-CRI Immune Assay Working Group. Harmonization guidelines for HLA-peptide
multimer assays derived from results of a large scale international proficiency
panel of the Cancer Vaccine Consortium. Cancer Immunol Immunother. 2009
Oct;58(10):1701-13. Epub 2009 Mar 4. PubMed PMID: 19259668; PubMed Central PMCID:
PMC2714899.

Janetzki S, Panageas KS, Ben-Porat L, Boyer J, Britten CM, Clay TM, Kalos M,
Maecker HT, Romero P, Yuan J, Kast WM, Hoos A; Elispot Proficiency Panel of the
CVC Immune Assay Working Group. Results and harmonization guidelines from two
large-scale international Elispot proficiency panels conducted by the Cancer
Vaccine Consortium (CVC/SVI). Cancer Immunol Immunother. 2008 Mar;57(3):303-15.
Epub 2007 Aug 25. PubMed PMID: 17721781; PubMed Central PMCID: PMC2150634.

Janetzki S, Britten CM, Kalos M, Levitsky HI, Maecker HT, Melief CJ, Old LJ,
Romero P, Hoos A, Davis MM. "MIATA"-minimal information about T cell assays.
Immunity. 2009 Oct 16;31(4):527-8. PubMed PMID: 19833080.

Attig S, Price L, Janetzki S, Kalos M, Pride M, McNeil L, Clay T, Yuan J,
Odunsi K, Hoos A, Romero P, Britten CM; CRI-CIC Assay Working Group. A critical
assessment for the value of markers to gate-out undesired events in HLA-peptide
multimer staining protocols. J Transl Med. 2011 Jul 11;9:108. PubMed PMID:
21745365; PubMed Central PMCID: PMC3148571.

Janetzki S, Britten CM. The impact of harmonization on ELISPOT assay
performance. Methods Mol Biol. 2012;792:25-36. PubMed PMID: 21956498.

Moodie Z, Price L, Janetzki S, Britten CM. Response determination criteria for
ELISPOT: toward a standard that can be applied across laboratories. Methods Mol
Biol. 2012;792:185-96. PubMed PMID: 21956511.

van der Burg SH, Kalos M, Gouttefangeas C, Janetzki S, Ottensmeier C, Welters 
MJ, Romero P, Britten CM, Hoos A. Harmonization of immune biomarker assays for
clinical studies. Sci Transl Med. 2011 Nov 9;3(108):108ps44. PubMed PMID:22072636.