Abstract
Background/Aim: The Epi proLung® BL Reflex Assay [short stature homeobox gene two methylation assay (SHOX2 assay)] (Epigenomics AG, Berlin, Germany) utilizes quantitative methylation-sensitive real-time polymerase chain reaction (QMSP) for the quantification of methylated short stature homeobox gene two (SHOX2) DNA. In the present study, the diagnostic utility of the SHOX2 assay was tested with regard to cytology for different cytological diagnostic categories to assess whether it can complement the cytological examination and the DNA methylation marker panel targeting the gene promoters of adenomatous polyposis coli 1A (APC), cyclin-dependent kinase inhibitor-2A (p16INK4A) and Ras association domain family protein 1 (RASSF1A) regarding lung cancer detection in bronchial aspirates. Materials and Methods: Prospectively collected DNA from 169 patients (cytological diagnosis: 47 tumor-positive, 56 equivocal and 66 tumor-negative) was analyzed for SHOX2 DNA methylation utilizing QMSP. Patients were followed-up for a period of 11 months maximum. Results: When equivocal diagnoses were categorized as tumor-positive, cytology and SHOX2 DNA methylation achieved 72% and 64% sensitivity and 63% and 98% specificity, respectively. SHOX2 DNA methylation identified 66% of the patients with cancer subsequent to a cytological equivocal diagnosis. SHOX2 complements the cytological diagnosis and the methylation marker panel. Conclusion: The assay could be of use for the improvement of diagnostic accuracy if applied subsequent to equivocal or negative cytology (sensitivity=69%, specificity=98%). Furthermore, the SHOX2 assay can complement a methylation-based marker panel.
- Cytology
- bronchial aspirates
- DNA methylation
- lung cancer
- biomarker
- quantitative methylation-specific PCR
- APC
- p16INK4A
- RASSF1A
- SHOX2
Worldwide, lung cancer has been the most common type of cancer for several decades. It is the most common cancer in men, and the fourth most common in women. Lung cancer was also the most common cause of death from cancer in 2008, with 1.38 million deaths (18.2% of the total number of fatalities caused by cancer) (1). In Germany in 2010, lung cancer was the fourth most commonly diagnosed type of cancer (52,070 cases), behind breast, prostate, and colorectaI cancer (70,950; 65,830; and 62,430 cases, respectively). Nevertheless, lung cancer was the most frequent cause of death (42,008 cases) by cancer (2). In contrast to breast, prostate and colorectal cancer, there is no established screening for lung cancer, which is probably one reason for the high mortality rate. The cytological examination of bronchial aspirates, brush biopsies and fine-needle aspiration biopsies, is a widely used method in the diagnostic work-up of suspected lung cancer, recommended by medical guidelines (3). Bronchial aspiration cytology is known for moderate sensitivity (48% and 43% for central and peripheral tumours, respectively) (4). Adjuvant methods are used in order to enhance the diagnostic accuracy of the morphological cytological examination. Both at our Institute and other specialized centres, for example, fluorescence in situ hybridization (FISH) and DNA image cytometry are used in daily diagnostic routine for that purpose. A promising method for the enhancement of diagnostic accuracy is the analysis of DNA methylation (5). Since cytological specimens provide high quality of extracted DNA, as shown on bronchial aspirates (6), the analysis of DNA methylation should be possible with all kinds of the aforementioned bronchial specimens (7).
The Epi proLung® BL Reflex Assay [short stature homeobox gene two methylation assay (SHOX2 assay)] (Epigenomics AG, Berlin, Germany) was introduced as an aid for the diagnosis of lung cancer via bronchial aspirates. The test uses quantitative methylation-sensitive real-time polymerase chain reaction (QMSP) to determine levels of methylation of the short stature homeobox gene two (SHOX2) in specimens. In its validation study on 204 valid measured bronchial lavage specimens, the test achieved a sensitivity of 78% and a specificity of 96% (8). In malignant pleural effusions (MPE), the SHOX2 assay displayed a sensitivity of 40% and a specificity of 96%, and was not specific for lung cancer (9).
After gaining experience in handling the SHOX2 assay on pleural effusions, the aim of our present study was to clarify if the SHOX2 test could complement the morphological cytological examination and the methylation assay used in a previous study on bronchial aspirates (7). Furthermore, the performance of the SHOX2 assay was separately tested for different tumor types and localizations, as well as for different cytological diagnostic groups (tumor-positive, equivocal and tumor-negative).
Materials and Methods
Study design. The collection of 210 specimens, the extraction of DNA from bronchial aspirates and the collection of the follow-up has previously been published by Schramm et al. (7). In short, samples had been collected prospectively in 2008 and early 2009 until 70 cytological samples which tested as cancer-positive, 70 which tested as suspicious and 70 which tested as cancer-negative were gathered. Cytological results were compared to a reference standard, defined in advance, by review of the patients’ charts. The patients were followed-up for 11 months. The SHOX2 assay and the morphological cytological diagnosis of the specimens were both performed blinded to the reference standard and vice versa. The study was approved by the local Ethics Committee (#3059).
Study inclusion criteria. All specimens included in the previously mentioned study (7) were accepted on the condition that there was more than 800 μg of DNA for performing the Epi proLung® BL Reflex Test (Epigenomics AG, Berlin, Germany).
Cytological diagnosis. The morphological cytological diagnoses on bronchial aspirates were divided into three groups: cancer-positive, equivocal and cancer-negative. These initial cytological diagnoses rely on microscopy of smears which are stained according to Papanicolaou. In contrast to routine cytology, several additional specimens (bronchial aspirates from different locations, bronchial brushings or fine-needle aspiration biopsies from regional lymph nodes) and adjuvant methods (DNA image cytometry or FISH) were not included for the cytological diagnosis in this study.
Sample preparation. The DNA extraction from the bronchial aspirates was performed in a previous study (7) utilizing the Puregene DNA Isolation Kit (Gentra Systems, Minneapolis, MN, USA). The DNA was solvated in water and stored at -20°C for up to six years. After thawing the samples at room temperature, the DNA concentration was determined using a NanoDrop 100 Spectrophotometer (Thermo Fischer Scientific Inc., Waltham, MA, USA). An 80 μl aliquot was created for each sample containing 800 μg to 1000 μg of DNA. Bisulphite conversion was performed using the Epi proLung® BL Preparation Kit (Epigenomics AG). The samples were prepared in sets of 22 samples. A positive and a negative control from the Epi proLung® BL Work Flow Control Kit (Epigenomics AG) were also included in each set.
Real-time PCR. Real-time polymerase chain reaction (PCR) was performed using an Applied Biosystems® 7500 Fast real-time PCR instrument (Life Technologies Corporation, Carlsbad, CA, USA) and the Epi proLung® BL real-time PCR Kit (Epigenomics AG). The assay was designed to detect lung cancer in bronchial aspirates. All samples were measured in triplicates. A sample was declared test-positive if the validity criteria of the SHOX2 test kit were met and if the methylation level was higher than or equal to the cut-off as predefined by the assay (ΔΔCt≤9.5). The methylation level was calculated using the ΔΔCt method (10, 11).
Data and statistical analysis. To enable a comparison between the two possible assay results and the three possible results of the morphological cytological diagnoses, an artificial construction was used. The equivocal results were counted as positive for the calculation of the sensitivity and the specificity, similar to the previous study (7). Therefore, the diagnostic accuracy of cytology in the presented study is not directly comparable with previously quoted literature (4). All statistical results were calculated using the R-Project for Statistical Computing (Version 3.0.1) and the R packages bdpv (Version 1.0) (12) and pROC (Version 1.5.4) (13). Confidence intervals (CI) were calculated at 95%.
Results
The DNA concentration of the leftover material was determined. A total of 169 specimens had enough DNA for the performance of the SHOX2 assay. These samples had the following morphological cytological diagnoses on bronchial aspirates: 47 tumor-positive, 56 equivocal and 66 tumor-negative. All 169 samples were measured using the SHOX2 assay. No valid results could be achieved for 51 samples. The patients’ characteristics are displayed in Tables I and II. The Standards for Reporting of Diagnostic Accuracy (STARD) diagram presents the flow of patients through the study (Figure 1) (14).
Valid and invalid measured specimens were inspected for significant differences. Age was tested using the unpaired t-test; the other characteristics were compared using the Chi-square test. The level of significance (p≤0.05) was adjusted to p≤0.0083 using the Bonferroni correction. Neither the cytological diagnosis (p=0.33), nor the follow-up result (p=0.38), gender (p=0.82), age (p=0.80), histological tumor type (p=0.94) or tumor localization (p=0.38) (exclusively peripheral versus otherwise localized) presented significant differences between the valid and invalid measured specimens.
Out of the 118 valid SHOX2 assay results, 49 were classified positive and 69 were classified negative. With 48 positive cases out of the 75 cases, the test achieved a sensitivity of 64% (95% CI=52%-75%). With 42 true-negative results out of 43 controls, a specificity of 98% (95% CI=88%-100%) was attained. The area under (AUC) the receiver operating characteristic (ROC) curve was calculated on a non-smooth curve (Figure 2). The AUC value was 0.92 (95% CI=0.88-0.97).
Furthermore, the sensitivity for the detection of different primary lung carcinomas was examined. At a sensitivity of 82% (9/11) (95% CI=48%-98%), small cell lung cancer (SCLC) was identified best, followed by squamous cell carcinoma at 81% (13/16) (95% CI=54%-96%), adenocarcinoma at 65% (17/26) (95% CI=44%-83%) and non-small cell lung cancer (NSCLC), not otherwise specified, at 50% (6/12) (95% CI=21%-79%). Other lung tumors and metastases were identified at a rate of 30% (3/10) (95% CI=7%-65%), including sarcoma and two out of the five cancer of unknown primary (CUP) syndromes with pulmonary manifestation. Both CUP syndrome tumours may have originated from the lung. While the different identification rates for SCLC and NSCLC were not significantly different (p=0.28), SHOX2 DNA methylation was significantly different between these groups (SCLC and NSCLCs) and other lung tumors and metastases (p=0.03).
The SHOX2 assay detected centrally localized tumors at a rate of 68% (32/47) (95% CI=53%-81%), and peripheral ones at 56% (10/18) (95% CI=31%-78%). Nonetheless, this difference was not statistically significant (p=0.34).
Table III presents how the SHOX2 assay performed for the different morphological cytological diagnostic groups. SHOX2 identified 66% (14/21) (95% CI=43%-85%) of cancers subsequent to an equivocal cytological diagnosis. The sensitivity and specificity of the morphological cytology (evaluated only on SHOX2-valid specimens) and the combination of cytology with subsequent SHOX2 test on equivocal and negative specimens are presented in Table IV.
The cytological examination achieved a sensitivity of 72% (54/75) (95% CI=60%-82%) and a specificity of 63% (27/43) (95% CI=47%-77%), on valid SHOX2 measured specimens. The cytology identified squamous cell carcinoma best, with a detection rate of 88% (14/16) (95% CI=62%-98%), followed by NSCLC (NOS) with 75% (9/12) (95% CI=43%-95%), adenocarcinoma with 73% (19/26) (95% CI=52%-88%), and SCLC with 64% (7/11) (95% CI=31%-89%) detection rate. Other lung tumors and metastases were identified at a rate of 50% (5/10) (95% CI=19%-81%), including breast carcinoma, sarcoma and three out of the five lung manifestations in CUP syndrome. The cytological examination was able to detect 67% (12/18) (95% CI=41%-87%) of the peripheral tumours, and 74% (35/47) (95% CI=60%-86%) of those tumors localized centrally. The difference in sensitivity was significant neither for tumor type (p=0.13) nor for localization (p=0.55).
In a previous study (7), a QMSP-based marker panel was applied to the same group of patients as in the present study. The marker panel focused on three different methylation sites. The studied sites were gene promoters of adenomatous polyposis coli 1A (APC), cyclin-dependent kinase inhibitor-2A (p16INK4A) and Ras association domain family protein 1 (RASSF1A). The sensitivity and the specificity for 160 possible combinations of these biomarkers with the SHOX2 assay and the cytological examination were calculated in order to test the combinations for complementarity. The performance values were calculated on the 118 valid SHOX2-measured specimens. A selection of these combinations and their sensitivity and specificity values are presented in Table V. The SHOX2 biomarker had by far the best sensitivity as a single biomarker. While RASSF1A (52%) and a combination of RASSF1A and p16INK4A (59%) enhanced the sensitivity of the cytological examination without reducing the specificity (100%); the enhancement of sensitivity by SHOX2 in addition to the cytological examination was larger (69%) but at a small cost of specificity (98%). The combination of cytological examination and SHOX2 was complemented by the RASSF1A biomarker to a sensitivity of 72% without a decrease in specificity. A further addition of the p16INK4A biomarker did not complement this combination any further. The application of APC instead of p16INK4A to this combination enhanced the sensitivity by only 1% but reduced the specificity by 3%. The marker panel of the previous study achieved a sensitivity of 51% (38/75) (95% CI=39%-62%) and a specificity of 98% (42/43) (95% CI=88%-100%). Evaluating the SHOX2 results in combination with this QMSP marker panel on valid SHOX2-measured specimens, a sensitivity of 71% (53/75) (95%CI=59%-81%) and a specificity of 95% (41/43) (95% CI=84%-99%) were achieved. Up to this point, a combination was considered test-positive if at least one of the markers gave a positive test result. If a stricter criterion is applied and a panel is only considered test-positive if at least two of the markers gave a positive result, the p16INK4A marker is more useful, as it complements a panel of SHOX2, RASSF1A and APC [sensitivity of 39% (29/75), 95% CI=28%-51%] to give a sensitivity of 47% (35/75) (95% CI=35%-59%) without reducing the specificity [100% (43/43), 95% CI=92%-100%].
Discussion
SHOX2 methylation was introduced in 2010 as a biomarker with the ability to distinguish between malignant and benign lung disease with a sensitivity of 68% and a specificity of 95% in a case-control study with 523 valid measured samples (15). In 2011, the performance of the SHOX2 biomarker on blood plasma was reported to have a sensitivity of 60% and a specificity of 90% in a case-control study with 343 valid measured specimens (16). In 2012, the SHOX2 assay was evaluated on bronchial aspirates, with a sensitivity of 78% and a specificity of 96% in a case-control study with 250 patients (8). In 2013, the assay’s performance on MPE was reported, with 40% sensitivity and 96% specificity, evaluated on 802 valid measurements with a modified cut-off criterion (9). In contrast to case-control studies on bronchial aspirates, our patient collective was accrued prospectively. Furthermore, our patient collective was also examined using other biomarkers. Therefore, it is possible to estimate the performance of combinations of these biomarkers with the SHOX2 biomarker.
According to the validation study of the SHOX2 assay (8), the test validity is strongly inversely proportional to the age of the sample. The material they had stored for up to 12 years resulted in 62% (154/250) valid results. With 70% (118/169) valid results from our material, which had been stored for up to six years, our result meets all expectations. Furthermore, we tested if the cytological diagnosis, the follow-up result, gender, age, tumor type or tumor localization influence the validity of the measurement, but none of these factors displayed a significant influence. We hence assume the 30% invalid measurements to be a result of the materials’ storage.
Comparing the sensitivity values of the SHOX2 assay between the presented study (64%, 95% CI=52%-75%) and the SHOX2 assay validation study (78%, 95% CI=69%-86%) (8), the sensitivity result of the validation study could not be reproduced. However, with a specificity of 98% (95% CI=88%-100%) and an AUC of 0.92 (95% CI=0.88-0.97), the presented results are very similar to the specificity value (96%, 95% CI=90%-99%) and the AUC value (0.94, 95% CI=0.91-0.98) of the validation study (8). We consider the tumor type to influence the difference between studies. SCLC was detected best in both studies, but in our study, the proportion of SCLC in the valid measured cases was only 15% compared to 30% in the validation study. However, as the detection rate for each of the lung tumor types (SCLC and NSCLC) was lower in our study than in the validation study, this is not the only reason for the lower overall sensitivity. As cigarette smoke is a very powerful environmental modifier of DNA methylation (17), the different smoker ratio could also have been of influence. The cases in our study consisted of at least 78% smokers (known smokers) and 91% smokers at maximum (smokers plus unknown smoking status), while in the validation study, 98% of the cases were smokers. Furthermore, the 95% CIs for the sensitivity values of both studies overlap, so it is plausible to assume the real sensitivity value to be somewhere in between these. This assumption is further supported by the fact that the first examination of SHOX2 methylation as a biomarker on bronchial aspirates achieved a sensitivity of 68% (15).
Lung cancer was identified with a significantly higher detection rate than other tumor types (69% vs. 30%). Therefore, it may be concluded that the SHOX2 assay has a tendency to detect primary lung carcinomas more frequently than other malignant neoplasms of the lung; on the other hand, however, it is not specific for lung cancer. This is consistent with the fact that use of the SHOX2 methylation biomarker is primarily focused on the diagnosis of lung carcinomas on bronchial aspirates (15). However, this effect of lung cancer specificity seems to depend on the origin of the specimen. SHOX2 methylation was able to detect breast cancer (55%) at an even higher detection rate than primary lung carcinomas (44%) in MPE. Furthermore, there was no significant proof (p=0.85) that there is a difference in the detection rate between lung cancer and other types of tumour including breast cancer, mesothelioma, lymphoma/leukaemia and gastrointestinal cancer on MPE (9).
Cytological examination and the SHOX2 assay are not directly comparable. The SHOX2 assay dichotomizes the results. The cytological examination has tumor-positive, equivocal and tumor-negative results and delivers information about composition, condition and histogenetical origin of the cells. In order to compare the results, we also had to dichotomize the results of the cytological examination. Therefore, we counted the equivocal results as test-positive. An effect of this construct is an increased sensitivity and a reduced specificity. In Table IV, this construct displayed a specificity for cytology of only 63%. But in fact there was no false-positive cytological tumour diagnosis in the group of patients (compare Table III). The false-positive cases in this construct originate from equivocal results. The alternative for the dichotomization, namely to count the equivocal results as negative, would not consider the equivocal result to be a strong indication that the patient must be examined further. In a nutshell, data for cytological examination as shown in Table IV and data for sensitivities regarding different tumor types or tumor localizations can be used for a first impression of the ability of the cytological examination, but it should be kept in mind that these values are influenced by the artificial construct used.
Table III shows that the SHOX2 assay complements the equivocal cytological diagnosis group best. Although the SHOX2 results are best in the group of specimens diagnosed as cytologically positive, it provides no additional information on the cytological examination. Therefore, its application is not necessary in the diagnostic routine if tumor cells are identified cytologically. Negative cytological results could also benefit from complementation by the SHOX2 assay, but it seems to be less effective and at the expense of specificity. With us, the detection rate in specimens which were diagnosed as negative for cytological tumor was only 24%, whereas the validation study of the SHOX2 assay achieved a tumor detection rate of 63% of the cases in the same group (8). As the SHOX2 assay is expensive, the usage of the SHOX2 assay on all specimens which were diagnosed as negative for cytological tumors is questionable with regard to cost effectiveness, based on our data. The SHOX2 reflex assay competes with the FISH analysis and the DNA image cytometry, which our Institute and other specialized centres currently use as reflex test for equivocal cytological test results. The FISH analysis has been reported to have a sensitivity of 76% (95% CI=66%-84%) and a specificity of 98% (95% CI=94%-100%) (7). The DNA image cytometry had a sensitivity of 77% (95% CI=68%-85%) and a specificity of 98% (95% CI=94%-100%) (7). From the SHOX2 assay values displayed in Table IV, the SHOX2 assay seems to be inferior. However, the major difference between the SHOX2 assay and these two methods is that the latter require a large amount (DNA image cytometry) or a small amount (FISH) of suspicious cells recognized during the cytological examination. In contrast, the SHOX2 assay is applicable even if there are no recognizable suspicious cells.
Schmiemann and co-workers developed a methylation assay for the diagnosis of lung cancer on bronchial aspirates in 2005 (18). They examined a marker panel of three different methylation sites (APC, p16INK4A and RASSF1A) utilizing QMSP. Their QMSP marker panel achieved a sensitivity of 63% (22/35) for central tumours and of 44% (21/48) for peripheral tumours. The SHOX2 assay in the present study detected centrally localized tumours at a rate of 68% (32/47) (95% CI=53%-81%) and tumours localized in the periphery at a rate of 56% (10/18) (95% CI=31%-78%). Although in both studies the difference in the detection rate was not significant [p=0.09 (own calculation) and p=0.34, respectively], the methylation-based assays seem to detect peripheral tumours less effectively than central ones. This might be attributed to the fact that bronchial aspirates contain more cells from central than peripheral regions of the lung due to the washing of primarily central airways.
Schmiemann and co-workers suggested that their assay could be enhanced by addition of complementary markers. Their marker panel (APC, p16INK4A and RASSF1A) was applied to our patient collective in a previous study (7) and achieved a sensitivity of 50% and a specificity of 98%. In a direct comparison, the SHOX2 assay was superior to this QMSP marker panel of the previous study as it had 14% higher sensitivity and the same specificity. The combination of the marker panel with the SHOX2 assay resulted in a further increase of sensitivity (71%) and a slight decrease of specificity (95%). Therefore, we conclude that the QMSP marker panel can complement the SHOX2 assay. The combination of the cytological examination, the SHOX2 assay and the RASSF1A biomarker was more useful. Further application of the p16INK4A or the APC biomarker was not useful. The p16INK4A biomarker did not complement this combination, and the APC biomarker reduced the specificity in that context and merely increased the sensitivity. If a marker panel is only considered positive when at least two of the markers give a positive result, the application of APC and p16INK4A increases the accuracy of the panel and is therefore useful in this regard.
Conclusion
Although we were not able to reproduce the high sensitivity of the validation study, the SHOX2 assay could complement the cytological diagnosis well. If applied subsequently to equivocal or negative cytology (sensitivity=69%, specificity=98%), the combined diagnostic accuracy is enhanced. Additional application of the RASSF1A biomarker could lead to further enhancement. Moreover, the SHOX2 assay has the tendency to detect primary lung carcinomas in bronchial aspirates better than other tumour types.
Acknowledgments
The Epi proLung® BL Reflex Assay was kindly provided by the Epigenomics AG (Berlin, Germany). Furthermore, the Authors would like to thank Baldur Eberle (Life Technologies, Darmstadt, Germany) for providing the AB7500 Fast Real-time PCR System.
Footnotes
-
Declaration of Interests
All Authors are employed at the Department of Cytopathology at the Heinrich Heine University of Düsseldorf in Germany. The employment of P. Ilse was granted by the research commission of the medical faculty of the Heinrich Heine University of Düsseldorf (Grant no. 9772432). The Epigenomics AG had no influence on the study except for the provision of the SHOX2 assay.
- Received September 9, 2014.
- Revision received September 22, 2014.
- Accepted September 22, 2014.
- Copyright© 2014, International Institute of Anticancer Research (Dr. John G. Delinasios), All rights reserved