Validation of a HLA-A2 tetramer flow cytometric method, IFNgamma real time RT-PCR, and IFNgamma ELISPOT for detection of immunologic response to gp100 and MelanA/MART-1 in melanoma patients

Background HLA-A2 tetramer flow cytometry, IFNγ real time RT-PCR and IFNγ ELISPOT assays are commonly used as surrogate immunological endpoints for cancer immunotherapy. While these are often used as research assays to assess patient's immunologic response, assay validation is necessary to ensure reliable and reproducible results and enable more accurate data interpretation. Here we describe a rigorous validation approach for each of these assays prior to their use for clinical sample analysis. Methods Standard operating procedures for each assay were established. HLA-A2 (A*0201) tetramer assay specific for gp100209(210M) and MART-126–35(27L), IFNγ real time RT-PCR and ELISPOT methods were validated using tumor infiltrating lymphocyte cell lines (TIL) isolated from HLA-A2 melanoma patients. TIL cells, specific for gp100 (TIL 1520) or MART-1 (TIL 1143 and TIL1235), were used alone or spiked into cryopreserved HLA-A2 PBMC from healthy subjects. TIL/PBMC were stimulated with peptides (gp100209, gp100pool, MART-127–35, or influenza-M1 and negative control peptide HIV) to further assess assay performance characteristics for real time RT-PCR and ELISPOT methods. Validation parameters included specificity, accuracy, precision, linearity of dilution, limit of detection (LOD) and limit of quantification (LOQ). In addition, distribution was established in normal HLA-A2 PBMC samples. Reference ranges for assay controls were established. Results The validation process demonstrated that the HLA-A2 tetramer, IFNγ real time RT-PCR, and IFNγ ELISPOT were highly specific for each antigen, with minimal cross-reactivity between gp100 and MelanA/MART-1. The assays were sensitive; detection could be achieved at as few as 1/4545–1/6667 cells by tetramer analysis, 1/50,000 cells by real time RT-PCR, and 1/10,000–1/20,000 by ELISPOT. The assays met criteria for precision with %CV < 20% (except ELISPOT using high PBMC numbers with %CV < 25%) although flow cytometric assays and cell based functional assays are known to have high assay variability. Most importantly, assays were demonstrated to be effective for their intended use. A positive IFNγ response (by RT-PCR and ELISPOT) to gp100 was demonstrated in PBMC from 3 melanoma patients. Another patient showed a positive MART-1 response measured by all 3 validated methods. Conclusion Our results demonstrated the tetramer flow cytometry assay, IFNγ real-time RT-PCR, and INFγ ELISPOT met validation criteria. Validation approaches provide a guide for others in the field to validate these and other similar assays for assessment of patient T cell response. These methods can be applied not only to cancer vaccines but to other therapeutic proteins as part of immunogenicity and safety analyses.

effective for their intended use. A positive IFNγ response (by RT-PCR and ELISPOT) to gp100 was demonstrated in PBMC from 3 melanoma patients. Another patient showed a positive MART-1 response measured by all 3 validated methods.

Conclusion:
Our results demonstrated the tetramer flow cytometry assay, IFNγ real-time RT-PCR, and INFγ ELISPOT met validation criteria. Validation approaches provide a guide for others in the field to validate these and other similar assays for assessment of patient T cell response. These methods can be applied not only to cancer vaccines but to other therapeutic proteins as part of immunogenicity and safety analyses.

Background
Cancer immunotherapy clinical trials often use immunological assessment as secondary endpoints to evaluate vaccine potency. A number of techniques have been established to monitor antigen specific immunologic responses in patients. Many of these assays monitor T cell responses and were comprehensively reviewed by Keilholz et al. [1]. Most commonly used methods include: (1) direct measurement of serological cytokines, (2) T cell functional analysis for cell proliferative response, CTL, and cell associated cytokine production by Flow Cytometry and ELISPOT, and cytokine gene expression by real time RT-PCR, (3) cell phenotypic analysis (multi-color Flow Cytometry) including antigen specific T cell detection using HLA tetramers and additional cell phenotypic analysis for activated T cells, regulatory T cells (Treg), and naïve/memory T cells. Assay development studies (IFNγ Real Time RT-PCR and ELISPOT, HLA-A2 Tetramer analysis) and monitoring specific vaccine response in cancer patients are described by a number of investigators [2][3][4][5][6][7][8][9][10]. Although many different assays are used to monitor immune response in cancer patients, few of these assays are validated when used for clinical applications [1, 3,11,12]. Furthermore, the validation of immunoassays was identified as one of the critical areas for improvement when using these assays to evaluate immune responses in the clinic [1].
Unlike assays used for research studies, clinical assays need to be simple and robust, with reasonable turn around time, and high throughput. Minimal sample manipulation during sample collection, processing, shipment, storage, and testing are added advantages. Assays requiring small sample volume are also preferable. Methods that meet these criteria are optimized for each component and step during assay development/pre-validation studies. Standard Operating Procedures (SOP) and assay validation plans with acceptance criteria are followed in validation studies to further assess assay performance characteristics. Regulatory agencies and published white papers provide guidance on validation of analytical methods and immunogenicity methods to monitor anti-protein drug antibody response. Less information is available for validation of flow cytometry and T cell functional assays, which are generally more challenging.
We developed and validated HLA-A2 flow cytometry, IFNγ real time RT-PCR, and IFNγ ELISPOT assays to monitor specific CD8 + T cell responses in HLA-A2 melanoma patients immunized with genetic vaccines encoding glycoprotein 100 (gp100) or MART-1, two melanoma-associated antigens. We report our study on validation of the three methods using TIL cells alone or spiked into normal PBMC samples. The performances of the assays were further confirmed using PBMC from immunized patients. Assay performance met validation criteria and all three assays were shown to be effective for their intended use, monitoring patient's antigen specific T cell response. serve as antigen presenting cells (APC) in real time RT-PCR and ELISPOT analysis. Proof of principle studies were performed using frozen PBMC from three melanoma patients (kindly provided by Dr. Francesco Marincola, NCI, NIH, Bethesda, Maryland).

Patient PBMC samples
Frozen PBMC from the fourth melanoma patient which demonstrated immunologic response is also included as an example; samples from this patient are part of the clinical testing to monitor cancer vaccine potency of a Phase I/II clinical trial conducted by Genzyme Corporation.

Tetramers
The following HLA-A2 (A*0201) tetramers (Beckman Coulter Immunomics, San Diego, CA) were used including Negative Control (T01044, containing a proprietary irrelevant peptide not being recognized by human TCR), gp100 209-217(210M) (T01012, IMDQVPFSV), MART-1 26-35(27L) (T01008, ELAGIGILTV), and Influenza-Flu (T01011, GILGFVFTL) tetramer. Modified gp100 and MART-1 tetramers with prolonged stability and high affinity were used. To minimize assay variability, tetramers used here for assay validation were from the same lot as the ones for clinical sample testing. All three tetramers (gp100, MART-1, and Negative) were assembled from the same Biotinylated HLA-A2 monomer lot and the same Streptavidin-PE lot. Stability of the tetramers was monitored using TIL cells. All tetramers contain HLA-A2 restricted peptides, therefore only CD8 + T cells are expected to be detected.
2-8°C) was also evaluated and RT was chosen. Co-staining with anti-CD3 showed decrease tetramer binding probably due to proximity of CD3 and TCR, therefore anti-CD3 staining was not used. Fixed cells were shown to have decreased binding as compared to fresh. Therefore, freshly thawed, unfixed PBMC were used for validation study and clinical sample testing.
Since there is a very low percentage of gp100 and MART-1 tetramer positive cells in healthy subjects, TIL cells were used for method validation studies. TIL1520 (gp100 specific) or TIL1143 (MART-1 specific) at 1-5 × 10 4 cells/100 μL/tube were stained in FACS buffer (PBS without Ca 2+ and Mg 2+ , 1% BSA, 0.1% Sodium Azide) with 10 μL of tetramer-PE (0.1 μg/μL) and 10 μL of anti-CD8-FITC at room temperature (RT) for 1 hour in a 23-25°C incubator. Cells were washed with 3 mL of FACS buffer and harvested by centrifugation at 290 g (1500 rpm) for 7 minutes. Cells were re-suspended in 0.5 mL of FACS buffer. Ten μL of Propidium Iodide (PI) was added before acquisition for viable cell gating. Total of 10,000 to 20,000 TIL cells (un-gated events) were acquired. For frozen PBMC analysis, same staining procedure was used except that a total of 10 6 freshly thawed cells were stained and 500,000 cells were acquired. Data was analyzed using Cell Quest Pro Software. Percent tetramer positive cells among viable CD8 + cells were shown in quadrant statistics from CD8-FITC vs. Tetramer-PE dot blot. Viable CD8 + cells were defined by simultaneous gating on the triple regions, region 1 (lymphocytes from FSC vs. SSC), region 2 (viable cells-PI negative cells from FSC vs. PI), and region 3 (CD8+ cells from FSC vs. CD8). Assay validation was performed under GLP and following the method SOP.
As an example, Flu tetramer binding to frozen PBMC from a HLA-A2 healthy subject is shown in Figure 1, including gating sequence (A) lymphocyte-FSC vs. SSC, (B) viable cells (PI negative)-FSC vs. PI, and (C) CD8 + T cells-FSC vs. CD8 FITC. Tetramer positive cells are illustrated in (D) on gated viable lymphocytes-CD8 FITC vs. Flu Tetramer PE gated on viable lymphocytes, CD8 negative cells that lack tetramer binding are also shown.

IFNγ real time PCR assay
Freshly thawed HLA-A2 PBMCs at 10 6 cells/mL/well, duplicate wells in 24-well plate, were cultured for 2 hours at 37°C with 5% CO 2 and 95% humidity in serum free medium (AIM-V, GIBCO/BRL) stimulated with gp100 209 , gp100 pool , MART-1, Flu, PHA (positive control), or HIV (negative control). Peptides were used at 10 μg/mL/well for gp100 209 , gp100 pool , MART-1, Flu, or HIV. TIL1520 (gp100 specific) and TIL1235 (MART-1 specific) spiked into PBMC at various cell numbers were used as positive controls. After stimulation, cells were harvested and RNA prepared following Qiagen RNA extraction protocol. RNA was stored at <-60°C until use. RNA was thawed and concentration and purity were determined by spectrophotometer at wavelength A 260/280 (OD 260 /OD 280 ratio). Synthesis of cDNA was done following manufacturer's protocol (Promega) using AMV Reverse Transcriptase with 25 μM of RT primer for IFNγ or CD8. Samples were stored at -15°C until further analysis.
Real Time RT-PCR analysis was performed using forward and reverses primer (each at 25 μM) for IFNγ or CD8. The probes were used at 0.2 and 0.3 μL for IFNγ and CD8, respectively.
Positive control cDNA (IFNγ and CD8 plasmid, Invitrogen) were run in duplicate at various concentrations to generate standard curves for IFNγ and CD8. Copy numbers for IFNγ and CD8 was determined.
Following culture, the cells were discarded and plates were washed with PBS. Biotinylated anti-human IFNγ was added at 100 μL/well (1.5 μg/mL, Pharmingen) and plates were incubated for 2 hours at room temperature (in a 22-26°C incubator). Plates were washed and 100 μl of Strepavidin-Alkaline Phosphatase (Pharmingen)at

Statistical analysis
Tetramer flow cytometric analysis was performed using Cell Quest Pro software (BD Biosciences) and % tetramer positive cells were obtained from quadrant statistics among gated viable CD8 + T cells.  Additional statistical analysis was performed to examine assay accuracy and precision using Microsoft Excel. Accuracy was assessed by % Recovery, (detected value/expected reference value) × 100. Precision was examined using % CV (coefficient of variation), (SD/Mean) × 100. Linearity of Dilution (linear regression analysis) was performed using GraphPad Prism 4 (Version 4.02). Regression analysis of post-vaccine immunologic response in the representative melanoma patient was performed using JMP 7 software.

Part 1: Tetramer assay validation
Specificity Specificity (Selectivity) is the ability of an analytical method to differentiate and quantify the analyte in the presence of other components in the sample.
Tetramer assay specificity is defined as TIL cells which lack binding to negative tetramer and irrelevant tetramer and show specific binding to the relevant tetramer (TIL1520 binding to gp100 and TIL1143 binding to MART-1). Low background binding was observed from cells with no tetramer (0.00% for TIL1520 and 0.02% for TIL1143, data not shown) or stained with the negative tetramer (0.09% for TIL1520 and 0.02% for TIL1143), Figure 2
Unlike the high percentage of binding of gp100 tetramer to TIL1520, MART-1 tetramer binding to TIL1143 was at a much lower percentage probably due to activation associated TCR down modulation on TIL1143 (data not shown). To confirm that MART-1 tetramer can maximally detect all of the MART-1 specific T cells under the assay conditions used, Jurkat cells that were genetically modified to express TCR that recognizes MART-1/HLA-A2 (generously provided by Judi Baker and Ray Zane, Beckman Coulter Immunomics, San Diego, CA) were used and 97% of MART-1 tetramer positive cells were detected; irrelevant gp100 tetramer binding to the MART-1 Jurkat cells was minimal (0.04%), Figure 2(B). Control Jurkat cells did not show binding to MART-1 tetramer while there was some background binding to the gp100, Figure 2(B). Due to the following acquisition sequence (MART-1 Jurkat/ gp100, MART-1 Jurkat/MART-1, Control Jurkat/gp100, and Control Jurkat/MART-1), we believe that carry over of the MART-1 Jurkat/MART-1 tetramer sample caused back-ground staining in Control Jurkat/gp100 tetramer. This experiment could not be repeated due to an insufficient number of cells.

Accuracy
The accuracy of an analytical method describes the closeness of mean test results (detected) obtained by the method to the true value (expected) of the analyte. Accuracy was assessed by percent recovery [(detected value/ expected value) × 100] and 80-120% is considered acceptable.
Due to the lack of true value from a standard reference material for the tetramer assay and lymphocyte phenotype analysis using flow cytometric methods in general, our attempt at assessing accuracy was unsuccessful. We used detected data values from undiluted TIL cells to establish reference true value for the diluted samples (by multiplying the dilution factor); % tetramer positive cells detected especially at the low level, were found to be outside of 80-120% of the reference value, data not shown. TIL cells showed tetramer binding variability due to culture conditions and cell passages; this variability makes establishing a true value using detected values from undiluted samples challenging.
To monitor long term assay performance, we generated TIL1520 and TIL1143 working cell banks stored in liquid N 2 in a single using aliquot and used freshly thawed cells (no additional cell culture) as assay quality control material. (data is shown under precision-long term inter-assay performance assessment).

Precision
The precision of an analytical method describes the closeness of agreement (degree of scatter) between a series of measurements obtained from multiple sampling of the same homogenous sample under the prescribed conditions.
Intra assay precision (repeatability) expresses the precision under the same operating conditions over a short interval of time (in a single assay). Intra assay precision is determined by % CV (coefficient of variation) as (SD/ Mean) × 100 tested multiple times by one analyst in a single assay. Inter assay precision (Intermediate Precision) is defined as the variability of a sample (% CV) tested in multiple assays on more than one day. For example, factors that contribute to inter assay variability for the tetramer assay include cell preparation, staining methods, machine setting, gating during acquisition and data analysis. Percent CV <20% is considered acceptable for analytical assays in general. For flow cytometry assays to detect cells at a very low level, a higher %CV is expected. Since a low frequency of tetramer positive cells is expected among Tetramer assay specificity MART-1 J ur kat (lower left) MART-1 J ur kat (lower r ight) -CD8 FITC vs. gp100 PE -CD8 FITC vs. MART-1 PE patient PBMC, using a high percentage of gp100 tetramer positive cells among TIL1520 is not suitable for assessment of assay precision at the low level. TIL1520 was also spiked into the negative population (TIL1520 stained with the negative tetramer) to generate two samples containing a low percentage of gp100 tetramer positive cells (Low 1 and Low 2) for assessment of assay precision. Undiluted TIL cells were included as a high control (High).
Intra assay precision (% CV) for both gp100 and MART-1 tetramer are acceptable (<20% CV). Representative data is shown in Table 1. Precision for gp100 tetramer showed precision of 2%CV using undiluted TIL1520 (High). Percent CV was 16 and 10% when TIL1520 were further diluted to generate samples with a lower percentage of tetramer positive cells. For MART-1, % CV is 6%.
Inter assay precision (% CV) for gp100 was 18% and MART-1 was 15%, and therefore both met the validation criteria (<20%), Table 1. Analyst variability (%CV) between 2 operators is 12% (gp100) and 20% (MART-1); equipment shut down/re-start variability (% CV = 2% for MART-1) was minimal (data not shown). Due to high assay variability inherent in flow cytometric methods and the low level of tetramer positive cells (expected in patients), we a designed clinical testing regimen to minimize assay variability. In this testing regimen, frozen longitudinal PBMC samples from each patient were tested in a single assay by a single operator.
TIL cells maintained in culture at different passages experience variation in TCR expression level which could contribute to variability in the tetramer assay. To monitor long term assay performance, a working cell bank was prepared for each line (TIL1520 and TIL1143) and cells were frozen in single use aliquots. Freshly thawed cells (without additional culturing) were analyzed in each assay for clinical sample testing, serving as quality controls. This practice allows us to analyze long term (2 year) inter-assay precision (February 2003 to May 2005) which was not feasible during assay validation. Precision (%CV) from 48 assays performed by three different operators showed that gp100 tetramer analysis had acceptable %CV (7%), Table  1. MART-1 tetramer analysis variability was high with % CV of 45%, probably due to the low level of tetramer positive cells in combination with the high inter-assay variability that is expected in flow cytometric methods. This finding supported our clinical testing regimen; all longitudinal frozen PBMC samples from each patient were tested in a single assay by a single operator, allowing assessment of vaccine potency compared to pre-treatment baseline values in each patient. Data is shown as % Tetramer positive cells (Range, Mean, n, SD, and %CV). Cells are used as undiluted (High) or diluted in the negative population (Low 1 and Low 2 for TIL1520). Long term assay precision (inter assay) is shown using data collected in 48 tests.

Spike and recovery
Assessment of spike and recovery of an analyte in biological matrix (matrix effect) is defined as the direct or indirect alteration or interference in response due to the presence of unintended analytes or other interfering substances in the sample.
Due to the lack of a standard reference material to establish a true value, recovery (% tetramer positive cells detected) could not be assessed. In addition, the TIL cells showed unexpected FSC vs. SSC properties. Compared to resting T cells among PBMC, TIL cells resembled activated lymphocytes. (lymphocyte blasts). The use of a single gate to analyze the mixed cell population (TIL spiked in PBMC) was also found to be challenging (data not shown). Although TIL cells have the same HLA-A2 allele as the PBMC used here, the non-A2 alleles are expected to be different for other HLA loci (DR and DQ, for example), which could result in cell-cell interaction (aggregation).

Limit of detection (LOD) and limit of quantification (LOQ)
LOD is defined as the lowest concentration of an analyte that the bioanalytical procedure can reliably differentiate from background noise.
LOQ is defined as the lowest amount of an analyte in a sample that can be quantitatively determined with suitable precision and accuracy.
Due to the lack of a standard reference material to establish a true value, LOQ was not examined for the tetramer assay. Assay LOD and sensitivity was examined.
MART-1 (27L) tetramer is known to be recognized by CD8 + T cells in healthy subjects, therefore, % MART-1 tetramer positive cells in normal PBMC samples (endogenous level), shown in distribution study (Table 2), could not be used to assess background signal. Low % positive cells were detected among 20 PBMC samples using the negative control tetramer and gp100 tetramer, 0.11% and 0.07%, respectively (Mean value from 20 samples, described in Normal Distribution studies). At such low level, assay variability is expected to be higher and SD was found to be 0.11% (negative tetramer) and 0.09% (gp100). It is not a common practice in the field to use the negative control tetramer binding to establish assay background noise level; most laboratories use values from unstained cells. Our data showed that unstained cells had 0% tetramer positive cells in most cases. However, on occasion, positive cells were found with values less than 0.06% (data not shown).
Assay sensitivity can be improved by collecting a larger number of events on the cytometer. Due to the limited supply of TIL cells and clinical PBMC samples from patients and the need for reasonable assay throughput/ turn around time to maintain cell viability during acquisition, we evaluated total acquisition events vs. cell quality (viability by PI and % tetramer positive cells). Our data supported collection of 10,000-20,000 TIL cells and 200,000-500,000 PBMC. To further assess assay sensitivity under our assay condition, we spiked Flu positive donor PBMC at various percentages (100, 50, 25, 12.5, 6.3, 3.1, and 0) into the negative PBMC (unstained cells from the same donor) and % Flu tetramer positive cells were analyzed from total of 200,000 events collected. At the lowest level assessed (3.1% Flu positive PBMC among negative PBMC), Flu tetramer positive cells were detected in 2 tests at 0.022 % (1/4545) and 0.015 (1/6667). We expect that with increased total acquisition events, our assay sensitivity could reach the level found by other laboratories (0.01-0.0125%, equivalent to 1/8000-1/ 10,000). Studies were also performed using TIL1520 spiked into TIL1143 stained for gp100 and TIL1143 spiked into TIL1520 stained for MART-1. Assay sensitivity was 1/1000 to 1/2000 due to the lower number of events (10,000) collected. We believe our assay sensitivity is equivalent to the level found by other laboratories. Due to limited volume of samples collected in melanoma patients, we were limited to acquiring the number of events as described in this manuscript.

Calibration standard curve and linearity of dilution
Due to the lack of a standard reference material and knowing that TIL cells have different binding characteristics (affinity, specificity, etc) compared to patient PBMC, a calibration standard curve was not used to quantify tetramer positive cells.
The highest % tetramer positive cells were detected using undiluted TIL cells. TIL cells were further diluted into the negative cell population to assess assay linearity.

Sample stability
Sample stability was assessed and a summary is described here (data not shown). Short-term stability (room temperature and 2-8°C) was poor for both fresh blood (<48 hour) and PBMC (<24 hour); such storage is not recommended. Clinical blood samples were processed at the site upon collection using the Ficoll gradient method for PBMC isolation. The PBMC were then cryopreserved and stored in liquid nitrogen (LN 2 ) until shipment to Genzyme (on dry ice). Upon thawing, long term stability (LN 2 , -120°C) was evaluated using trypan blue exclusion and by additional T cell functional analysis (proliferative response to mitogen PHA using 3 H-TdR incorporation). Frozen PBMC were found to be stable for at least 5 years and we continue to evaluate the stored PBMC samples over time. Freeze/thaw stability is limited to 1 cycle, which is well-documented. Freshly thawed samples were analyzed immediately in Tetramer, Real time RT-PCR, and ELISPOT assays.
PBMC stability for real time RT-PCR and ELISPOT will not be discussed separately.

Normal distribution
HLA-A2 PBMCs from 20 healthy subjects were tested in the tetramer assay to define normal distribution ( Table 2). Among 20 normal individuals, binding to negative tetramer (0.11%) and gp100 (0.07%) was low. Higher MART-1 (27L) binding (0.55%) was observed. MART-1 tetramer is known to be cross-reactive in healthy PBMC samples, described previously by Pittet et al. [13]. MART-1 positive cells detected in normal PBMC samples were found to have low MFI (median fluorescent intensity), in contrast to MART-1 positive cells detected in TIL1143. It is difficult to distinguish MART-1 positive cells with low MFI from the negative cells and the percent is largely dependent on quadrant position. Therefore, defining the tetramer positive cell population in patients cannot rely solely on the percentage of positive cells especially those with low MFI. Identification of a distinct population, well separated from the negative population, and with high MFI is also important.
Determining reference ranges for assay controls Assay controls consisted of single use aliquots of TIL1520 (gp100 control) and TIL1143 (MART-1 control) working cell banks stored frozen in LN 2 . Freshly thawed longitudinal PBMC samples from each patient were analyzed for gp100 and MART-1 tetramer binding in a single assay using these positive controls. Data from TIL controls was compared to historical data. Negative control tetramer binding to TIL cells and PBMC was also used as negative controls.
PBMC viability (>80% viable by trypan blue exclusion after thaw) and PI exclusion during flow cytometry data analysis were additional cell quality controls. Tetramer assay linearity of dilution

Part 2: IFNγ real time RT-PCR validation Specificity
IFNγ real time RT-PCR specificity is defined as lack of response to irrelevant peptides and HIV negative control peptide and positive response to relevant peptide stimulation (TIL1520 with gp100 peptides and TIL1235 with MART-1 peptide).
The real-time RT-PCR assay showed a high level of specificity through the validation process. HLA A2 PBMC alone from healthy subjects did not show response to melanoma peptides; a dose dependent IFNγ response, fold increase (IFNγ relevant peptide/CD8)/(IFNγ HIV / CD8), was only seen in PBMC with spiked TIL cells stimulated with relevant peptide, TIL1520 stimulated with gp100 209 and gp100 pool and TIL1235 stimulated with the MART-1 peptide ( Figure 4). As expected, these TIL cells did not respond to the irrelevant peptide (data not shown) or the negative control (HIV) peptide. The positive control PHA response produced consistently high IFNγ expression levels indicating cell viability and expected cell function (described later in Spike and recovery, LOD and LOQ, and Normal distribution studies). Variability was observed among individual donors, which was probably due to differences in % CD8 + T cells and antigen presenting cells as well as cell functionality. A complete data set will be shown and discussed in normal distribution studies.

Accuracy and precision
The real time RT-PCR assay was examined for assay accuracy and precision by spiking 1000 copies of IFNγ plasmid per sample in 80 repeats (n = 80) for intra-assay and 18 repeats (n = 18) for inter-assay performance characteristics. Two analysts performed the analysis. Assay was found to be both accurate and precise with % recovery between 80-120% (analyst 2 had a 123%) and % CV < 20%, respectively (Table 3).

Calibration standard curve and linearity of dilution
A standard curve was run using plasmid (10 to 10 8 copies, 1:10 serial dilution) and no-template controls ( Figure 5). Linearity was determined by using a standard curve (starting quantity vs. threshold cycle-Ct) generated using plasmid IFNγ at 10-10 8 copies. Linear amplification of log serial dilutions was observed with Slope (-3.368), Y-intercept (40.155), and Correlation Coefficient (1.000). Standard curve was determined on 6 TaqMan plates and no significant differences were found.
Spike and recovery TIL 1520 and TIL1235 spiked in HLA A2 PBMC (from 10 healthy subjects) and stimulated with peptides were used to further assess real time RT-PCR assay performance characteristics. Dose (number of TIL cells) dependent IFNγ response was observed (Table 4). IFNγ response, [(IFNγ/ CD8) peptideorPHA /(IFNγ/CD8) HIV ], correlated with increased number of TIL cells spiked. TIL1520 responded to gp100 peptides, Table 4(A) and TIL1235 responded to MART-1 peptide, Table 4(B). Response to HIV, Flu, and PHA was also observed as expected. HIV response was low in all donors. Flu and PHA response vary among different individuals, which may due to difference in number of CD8 + T cells and antigen presenting cells, as well as cell function.

LOD and LOQ
LOQ and LOD were determined by spiking IFNγ plasmid and internal control CD8 plasmid at various copy numbers (1 to 10 5 ). Each sample was measured in 12 repeats and assay results were summarized in Table 5. LOQ for both IFNγ and CD8 is determined as 1000 copies where quantification was achieved with acceptable accuracy (% Recovery within 80-120%) and precision (% CV < 20%).
LOD for IFNγ and CD8 is 100 copies where all 12 repeats tested positive above the background.
LOD for gp100 and MART-1 specific IFNγ response was further assessed using TIL1520 and TIL1235 spiked in PBMC, also described in normal distribution studies (Table 4).
LOD was determined as 1/50,000 cells where IFNγ response was detected above the HIV control (fold increase of 1.0) and PBMC only (no TIL spiked). Table 4. Average IFNγ response (fold increase) to gp100 (209 and pool) and MART-1 from healthy subjects (n = 10) is <1.1.

Part 3: IFNγ ELISPOT validation
This assay was first validated using 80 TIL cells spiked into 4 × 10 5 PBMC per well (96 well plate), designated as High PBMC Assay. Due to the limited volume of blood collected from clinical melanoma patients, we also validated the assay using a lower number of PBMC (80 TIL cells spiked into 10 5 PBMC/well), designated as Low PBMC Assay. Peptide concentrations remained the same. Compared to the Low PBMC Assay, IFNγ secreting cells among the same number of TIL cells were found to be slightly higher in the High PBMC Assay, probably due to a higher number of antigen presenting cells in the PBMC population.
Data presented here are from the low PBMC assay except in LOD and LOQ; data from both high and low PBMC assays are shown. Specificity ELISPOT specificity is defined as the lack of response to irrelevant peptides and HIV peptide together with a positive response to relevant peptide stimulation (TIL1520 with gp100 peptides and TIL1235 with MART-1 peptide).
To evaluate assay specificity, a total of 80 TIL cells were spiked into PBMC (10 5 cells/well) and the number of IFNγ secreting cells following peptide stimulation was examined. Two analysts, each using two PBMC lots, performed five assays each. Data from two PBMC lots were comparable and variability between the two analysts was low. Data from PBMC lot 1 by analyst one is shown in Table 6. Among TIL1520, IFNγ secreting cells/well (average from triplicate wells), were detected upon gp100 209 stimulation at an average of 41 secreting cells/well. Stimulation with gp100 pool containing gp100 209 did not result in an increased frequency of IFNγ secreting cells (39 cells/ well) compared to gp100 209 alone, confirming that the TIL1520 is gp100 209 specific. This is consistent with the real time RT-PCR findings (described in Part 2). Similarly, IFNγ secreting cells were detected among TIL1235 following MART-1 peptide stimulation at an average of 9 secreting cells/well. Lower numbers of secreting cells (<2) were detected upon irrelevant peptide stimulation, further demonstrating assay specificity. PBMC stimulated with Flu peptide showed IFNγ response at 212 IFNγ secreting cells/well while HIV response was low with 0.9 IFNγ secreting cells/well (data not shown).

Precision
Cell based functional assays such as ELISPOT are expected to have high assay variability. We consider intra assay precision acceptable with % CV < 20% and inter assay precision acceptable with % CV < 25%.
Data from 80 TIL cells spiked into two PBMC lots at 10 5 cells (low PBMC assay) were analyzed by two analysts each performed eight intra-day assays and 10 inter-day assays. Cells were stimulated with relevant peptide (gp100 209 and gp100 pool to TIL1520 and MART-1 to TIL1235) and IFNγ secreting cells were examined. Data (Table 7) from PBMC lot 1 and Analyst one is shown as an example. Both intra assay (% CV < 20%) and inter assay (% CV < 20%) precision was found to be acceptable, except TIL1520 stimulated with gp100 pool showed %CV of 20.8% Cells stimulated with irrelevant peptide and HIV had very low background signal and % CV was high, as expected.
Accuracy, spike and recovery, and LOQ Due to the lack of a reference standard material to establish a true value, assay accuracy, spike and recovery, and LOQ were not examined. Data is shown as IFNγ gcopy numbers determined by 2 analysts to assess accuracy and precision for intra assay (a single assay with 80 repeats) and inter assay (in 18 inter day runs). Expected value is the copy number used as PCR template. Accuracy is examined using % Recovery (Detected/Expected) and precision is examined using %CV (SD/Mean).

Plate homogeneity
Samples loaded at different locations across a 96-well microtiter plate showed comparable results (data not shown).
LOD and assay sensitivity LOD (assay sensitivity) was assessed by spiking diminishing numbers of TIL1520 and TIL1235 cells into 4 × 10 5 PBMC (High) or 10 5 PBMC (Low) per well. TIL1520/ PBMC were stimulated with gp100 209 and gp100 pool and TIL1235/PBMC were stimulated with MART-1. The LOD was determined to be the least number of secreting cells that could be distinguished from the background (>10 cells/well) upon stimulation with relevant peptide. The acceptable level of background secreting cells was obtained from irrelevant peptide stimulation, HIV peptide stimulation and from the results of the normal distribution study (Table 8). Data from the normal distribution study showed the number of background IFNγ secreting cells (Mean + 2 SD) to be as follows: gp100 209 (8.9), gp100 pool (5.2), MART-1 (6.5), and HIV (6.7). Therefore, we consider background to be 10 IFNγ secreting cells/well.
For the High PBMC assay, the LOD for gp100 was defined as the ability to detect IFNγ secreting cells at frequency of 1/20,000 (15 secreting cells/well) among TIL1520. The LOD for MART-1 is at 1/10,000 (14 secreting cells/well). The data shown in Table 9 demonstrates that the assay sensitivity from the high PBMC assay is similar to the results published by other laboratories.
At first glance, assay sensitivity does not appear to be as good when the lower number of PBMC was used (10 5 cells/well), Table 8. Although we could still detect 10-13 spots, the detection frequency was found to be 1/8000 (10 secreting cells/well) for gp100 and 1/2000 (11 secreting cells/well) for MART-1. This finding is due to the fact that the PBMC cell count is used as the denominator when calculating the detection frequency. The lower cell number in the denominator creates a mathematical artifact of diminishing assay sensitivity. The number of secreting cells (spots) detected per well is also related to the TIL cells used. With high TIL cell numbers, we could generate 100-200 spots per well, however, resolution for counting the spots was decreased. In summary, 10-50 spots/well give us a reliable assessment of the counts, either by manual counting or computer assisted counting (data not shown). Sensitivity of our assay is similar to what described in the field when High PBMC was evaluated.

Calibration standard curve and linearity of dilution
Due to the lack of a standard reference material, calibration standard curves were not evaluated for quantification of cellular IFNγ response.

Normal distribution
Eight normal PBMC samples (10 5 /well) were evaluated in normal distribution studies. Response to gp100 209 , gp100 pool , MART-1 and HIV in all samples are below 10 IFNγ secreting cells/well. The mean (n = 8) and SD are shown in Table 8. Two samples showed low and high level of Flu response, with secreting cells at 15 and 241, respectively.
Determining reference ranges for assay controls A control HLA-A2 PBMC working cell bank was established for use as an assay control. To assure plate to plate consistency, TIL1520 and TIL1235 (80 cells/well) were spiked into 10 5 HLA-A2 PBMC/well and were evaluated for the number of IFNγ secreting cells upon stimulation with gp100 209 , gp100 pool , and MART-1 peptide. HIV peptide was used as a negative controls and PHA (mitogen stimulation) as positive control. Control reference ranges (Mean +/-2 SD) were established to monitor assay performance. TIL cells at different numbers were spiked into PBMC from individual healthy donors and IFNγ response examined. Response (average from 10 different donors) is shown as fold increase. All donors are HLA-A2 positive screened and confirmed to be HIV negative. Peptide stimulation is HLA-A2 restricted and specific for CD8 + T cells. The SD is high due to variability in individual response among 10 healthy subjects. This finding is expected. Fold increase is calculated as follows using CD8 as internal controls: (IFNγ from peptide and mitogen stimulation/CD8)/(IFNγ from HIV stimulation/CD8).

Part 4: Three validated assays demonstrated their intended use: detection of CD8+ T cell response in melanoma patients
Post-treatment PBMC obtained from three melanoma patients treated in an IRB approved melanoma vaccine protocol of the National Cancer Institute, Bethesda, MD (generously provided by Francesco Marincola) were analyzed for IFNγ response by real time RT-PCR and ELIS-POT,  Figure 7).
A regression analysis showed that in the tetramer assay, there is a significant linear trend between time (days) and % MART-1 positive cells with p-value of 0.0071, and the relation could be expressed as: % MART-1 Tetramer Positive Cells = 0.0013 × days + 0.68 NA, not applicable (spiked copy number at 10 and 1 did not show detection in all 12 tests, therefore SD, % Recovery, and %CV is not analyzed). IFNγ plasmid at different copy numbers were used for assessment of LOQ and LOD. Expected and detected values are shown. % Recovery (Detected/Expected) and % CV (SD/Mean) are calculated to assess assay accuracy and precision, respectively.
In the other two assays (IFNγ Real Time RT-PCR and ELIS-POT), samples that were collected at the last patient's visit demonstrated a IFNγ response much higher than both the baseline response (by ELISPOT only, no RT-PCR baseline data) and earlier post vaccine time points. However, there is no statistically significant linear trend between time (days) and the IFNγ response with a p-value > 0.05 (0.3506 for Real Time RT-PCR and 0.1441 for ELISPOT).
In summary, assay performance of each assay met the validation criteria and the three validated assays demonstrated that they served their intended use.

Discussion
The use of a wide variety of different immunoassays to assess immunological endpoints in cancer immunotherapy clinical trials has provoked recommendations that standardization and rigorous validation of these immunoassays is needed [1,11]. In response to these recommendations, we put three immunoassays, the tetramer, ELISPOT, and real time RT-PCR assays through a rigorous validation process in preparation for our cancer vaccine clinical trials. These assays met key validation criteria necessary for generating reliable clinical data. The assays were determined to be specific for each antigen, gp100 or MART-1. Assay precision for cell based functional assays met the criteria with % CV < 20% (intra day) and < 25% (inter day).
Assays were found to be sensitive with the real time RT-PCR being the most sensitive at 1 in 50,000 PBMCs. The tetramer flow cytometric method sensitivity was determined to be 1/4545-6667 (Tetramer Assay collecting 1 million events) and the ELISPOT sensitivity was at 1/ 10,000-20,000 (using high PBMC assay), similar to data reported by others [1]. For ELISPOT, assessment of assay sensitivity depends on number of TIL cells spiked into the number of PBMCs as the negative cell population. Due to the limited number of PBMC that could be obtained from melanoma patients, we also validated the ELISPOT assay using a low PBMC number and assay sensitivity was poor (1/2000); this is due to a mathematical calculation where responder TIL cells were spiked into a smaller PBMC population and this smaller number served as the denominator. Higher TIL cell numbers resulted in a larger number of secreting cells (100-200 cells/well), which were difficult to count due to poor resolution. We performed a TIL cell titration study and demonstrated that 10-50 cells/ well provided significant resolution to achieve a reliable assessment of cell numbers.
Similarly, a larger number of total events collected for the tetramer assay will improve assay sensitivity. With limited patient PBMC samples and the need for assay throughput and cell quality (viability) during sample acquisition, we validated the tetramer assay with ~500,000 total events collected. When one million PBMC was collected, assay sensitivity was improved but samples acquired at a later time showed poor cell viability. We also evaluated the use of fixed cells after staining, and found the MFI to be much lower suggesting tetramer binding to fixed TCR was poor.
Similar differences in sensitivity between different immunoassays have been previously observed [10]. Assay sensitivity is also influenced by the T cell line (TIL cells) used to validate an immunoassay, and few groups use the same T cell lines. For example, only 33% of the cells in the TIL1520 cell line were responsive to peptide stimulation [14]. Comparisons between laboratories will likely be in closer agreement when the same cell lines are used to validate an immunoassay and same TIL cells number/PBMC number is used. As an example, the sensitivity of our ELIS-POT assay was in close agreement with a previously published report where the TIL1520 were used to determine ELISPOT sensitivity [14]. A set of standard cell lines would enable a comparison of assay performance between laboratories.
While effector T cell responses can reliably be measured by each of these immunoassays, an important challenge is in determining the value that constitutes a positive response. A strong positive immunologic response measured by the MART-1 tetramer assay, such as the example shown in Figure 7, is often indisputable. Such a response profile showed a clear defined MART-1 tetramer positive CD8 + T cell population that was well separated from the tetramer negative CD8 + T cell population. This clearly suggests that immunization successfully enhanced the immune response. Low percentages of tetramer positive cells were seen in pre-treatment baseline sample. The binding resembles the tetramer positive cells specific for foreign antigens (Flu) in Figure 1, demonstrating breaking of tolerance to self antigen (MART-1).
On the other hand, positive responses are more likely to be detected at low percentages in the blood making it much more difficult to define a positive immunological response to a cancer vaccine. Therefore, guidelines need to be implemented on data analysis and interpretation based on assay performance characteristics such as precision and LOD. Use of proper negative controls such as the negative control tetramer, will help distinguish a positive response by setting the correct quadrant for data analysis to reduce subjectivity, especially when tetramer positive cells are not well separated from the negative population. Fold increase (>2 fold) of post-treatment response over the baseline value has been used, however, baseline values near zero value could result in an artificially high fold A total of 80 TIL cells were spiked into 10 5 PBMC per well (Low PBMC) and tested in 8 repeats (intra assay) and 10 tests (inter assay) by two analysts. Average IFNγ gsecreting cells per well (triplicate wells) upon peptide stimulation are shown. Mean, SD, and %CV is shown for PBMC 1 by Analyst 1 as an example.   TIL cells were spiked into 4 × 10 5 (High) or 10 5 PBMC (Low) per well and limit of detection was determined (IFNγ Secreting Cell Frequency). The lowest IFNγ secreting cells detected above the background is considered LOD, values shown in bold. The corresponding TIL/PBMC ratio indicates assay sensitivity.
ELISPOT linearity of dilution Interestingly, MART-1 tetramer positive CD8+ T cells were detected among both healthy volunteers and in melanoma patients who received cancer vaccines. Among healthy volunteers, MART-1 positive cells showed low MFI (median) probably reflecting low affinity/avidity (MFI is not shown). In patients, however, MART-1 positive cells had high MFI (Figure 7). Function of these MART-1 positive CD8 + T cells were reported that the cells in healthy volunteers may be of naïve phenotype, which lacks effector function (presence of CTL precursors); however, in cancer patients, these cells have the memory phenotype [15][16][17]. Their effector function was demonstrated in vitro upon MART-1 peptide stimulation (in the presence of APC such as dendritic cells) using methods such as cytokine production (IL-2, GM-CSF, IFNγ) and CTL activity.
Correlation of MART-1 specific CD8 + T cells in peripheral blood with the presence of CTL cells at the tumor site and clinical response in vivo is still not fully established [15,[17][18][19][20]. A majority of the peptide reactive CD8 + cells may not be tumor reactive due to various mechanisms such as down modulation of HLA class I on tumor cell surface and presence of regulatory T cells and TGFβ, etc. Sorting of the tetramer positive cells for generation of CTL in vitro has been used as adoptive transfer (cell based therapy) in melanoma patients [21,22]. Correlation of immunologic response to clinical response (tumor regression) still needs to be established [15,17,20]. MART-1 specific CD8 + T cell response in one patient, as an example, was detected by all three validated clinical assays, HLA-A2 MART-1 tetramer assay, IFNγ real time RT-PCR and ELIS-POT ( Figure 7 and Table 10), demonstrating the assay utility in monitoring patient T cell response. Correlation to clinical response, however, was not demonstrated. A better understanding of the immune response seen in peripheral blood vs. the response at the tumor site will help us more fully understand the mechanisms of cancer vaccine and its potency.
The use of validated methods for clinical patient monitoring is important. When following patient longitudinal responses over time, our understanding of assay performance will assist us in implementing procedures that reduce assay variability and the use of QC samples will allow us  to monitor assay long term performance; making data generated from these validated methods more meaningful. While the validation of these three T cell assays was challenging, the experience we obtained during validation studies and conducting patient screening will assist us and others in the field to validate similar assays for assessment of patient T cell responses to not only to cancer vaccines but to other therapeutic proteins as part of immunogenicity and safety analysis.

Conclusion
In this manuscript, we reported data from validation studies to characterize three T cell assays, HLA-A2 tetramer Flow cytometric method, IFNγ real time RT-PCR, and IFNγ ELISPOT for detection of gp100 or MART-1 specific CD8 + T cell response.
Although challenging, our results showed that T cell functional assays can be validated to support clinical longitudinal sample testing to monitor patient T cell response to cancer vaccines. All three assays demonstrated their intended use for detection of cancer vaccine specific T cell response ( Figure 7 and Table 10). Use of validated assays in clinical patient monitoring minimized assay reproducibility problems and allowed better interpretation of clinical data.