Instructions

The information in the module is taken from a pilot study that assessed the validity and reliability of using a self-lavaging device for cytology and HPV testing for cervical cancer screening. Please review the study design and results and answer the questions in the module INTRODUCTION

Approximately 50% of women enrolled in one of seven large private insurance plans and diagnosed with cervical cancer between 1995 and 2000 in the United States (US) had not been screened in the three years prior to diagnosis [1]. Under-screening is likely to be higher among uninsured women. Finding innovative ways to screen for cervical cancer could improve uptake among under-screened populations. Offering self-sampling, in place of a pelvic examination, which some women find embarrassing or uncomfortable [2,3], could increase screening uptake. A number of self-sampling methods such as tampon and swab provide valid specimens for humanpapillomavirus(HPV) tests [4-6]; however, to date, self-sampling has not been found to provide adequate specimens for cytology [7].

HPV tests will be increasingly used as the primary screening test[8], but cytology will continue to play a role. HPV tests are not useful for women under age 30 [8], given their high prevalence of transient infections [9]. Further, cytology can be used for triage of HPV-positive cases with its higher specificity until new diagnostics are fully developed [10]. Identifying a self-sampling method that can be used for both HPV testing and cytology is thus important for optimizing self-sampling options.

We piloted the validity and reliability of using a self-lavaging device, the Delphi ScreenerTM (Delphi Bioscience, Scherpenzeel, Netherlands), for cervical cytology by comparing paired self- and clinician-obtained specimens using liquid-based cytology (LBC) among 198 women, and high-risk HPV in a sub-sample.

MATERIALS AND METHODS

The protocol for this study and supporting Standards for reporting of Diagnostic Accuracy (STARD) checklist are available as supporting information (see Protocol S1 and Checklist S1).

Recruitment and study visit

From December 1, 2008 to August 31, 2009, we enrolled 198 women who had attended one of three ambulatory clinics at the New York Presbyterian Hospital (NYPH) for cervical cancer screening. Clinicians at these three clinics asked women receiving cervical cancer screening as part of standard clinical care for permission to share their contact information with recruiters.

Recruiters phoned a convenience sample of consenting women to schedule study visits or record reasons for ineligibility or declining participation. During the pre-study index visit, women underwent a standard pelvic examination with clinician-collected liquid based cytology (LBC), and HPV testing if ordered by the clinician. Recruiters scheduled study enrollment visits 1-3 months after the index visit. One month after the index cytology results should have allowed sufficient time for cervical cells to replenish, with an upper limit of three months to minimize the potential for true changes in cervical cytology. By design, half of the participants had normal and half abnormal cytology results at the index visit to achieve the required sample size of 100 in each group. We used atypical squamous cells of undetermined significance (ASC-US) or worse to define abnormal cytology as these diagnoses trigger clinical follow-up [11]. Women whose index cervical screening results included co-testing with HPV were also co-tested with HPV using the Delphi Screener specimen. To be eligible, women needed to be at least 18 years old. Women were excluded if they reported current pregnancy, breastfeeding, hysterectomy, or discomfort reading on their own in Spanish or English. Women were asked not to schedule an appointment if they were within 4 days of the first day of their menses, or if they had undergone colposcopy prior to the study visit, as colposcopy and biopsy could change cytological results.

All participants provided written informed consent and received a packaged Screener for use in a private room at the clinical research site along with pictorial user instructions (which were also on the wall of the room in poster format). Study interviewers reviewed instructions with participants before they used the Screener on their own in a private room. The Delphi Screener is a sterile, plastic, syringe-like device containing 5 mL of buffered saline for self-lavage. The Screener currently does not have US Food and Drug Administration clearance. Columbia University's institutional review board approved use of the device as investigational with non-significant risk (ClinicalTrials.gov NCT00702208). Study implementation followed the ethical standards of the Declaration of Helsinki. Study visits during which women collected the self-lavage specimens took approximately 30-60 minutes and women received $30 to reimburse them for their time. Women were not reimbursed for follow-up visits that were part of their standard clinical care.

Specimen handling

Staff delivered specimens to the laboratory within 24 hours. Laboratory staff centrifuged specimens at 1700 rpm for five minutes, pouring off the supernatant, and pipetting the cell pellet into a Cytyc PreservCyt ThinPrep vial with 20 mL of PreservCyt (Hologic, Marlborough, MA). Stability testing on nine pre-test specimens found no decrease in cellular integrity at 24 or 48 hours after receipt in the laboratory. Based on this stability testing, centrifugation and fixing was batched and completed within 72 hours of self-collection (up to 24 hours for delivery to the laboratory plus up to 48 hours from receipt to processing). After suspension in PreservCyt, specimens were processed per manufacturing guidelines for ThinPrep specimens. Participants whose index screening included co-testing for HPV were co-tested using the Screener specimen with the Hybrid Capture II (Qiagen, Hilden, Germany) test per manufacturer specifications with a cutoff of one relative light unit.

Cytotechnologists used the 2001 Bethesda classification system [12] and recorded amount of fluid collected, cellularity, and presence of transformation zone cells (defined as endocervical and/or metaplastic cells). Each sample was reviewed by two cytotechnologists blinded to one another's diagnosis. These same cytotechnologists diagnosed all of the specimens collected using the self-lavage specimen. Abnormal slides were referred to a cytopathologist for final diagnosis. Cytotechnologists and cytopathologists were blinded to index results.

Clinical follow-up

Participants with abnormal cytology result either from the prestudy index visit with standard clinician collection or using the selflavaging device as part of the study were invited for colposcopy. Ascertainment bias occurs when only people with positive screening results are brought back for further diagnosis. Ten women (10%) with normal cytology results using both specimens were also invited for colposcopy to estimate ascertainment bias; these women received an additional $30 for this visit. Colposcopy was performed at three sites. All sites collected biopsies of acetowhite lesions. One site routinely collected endocervical curettings (ECCs), one collected ECC if no biopsy was taken, and one collected ECC if visualization of the transformation zone was incomplete, reflecting the lack of consensus on ECC utility when the transformation zone is fully visualized [13]. Women were followed through January 25, 2010.

Sample size

The sample size was originally calculated using the index cytological smear as the gold standard; 98 women with normal/abnormal results would detect a sensitivity/specificity of 80% with a lower 95% confidence limit of 65% [14]. As cytology is known to have limited sensitivity [15], however, results from two or more biopsies should be used to define true precursors to cervical cancer [16]. Our pilot, proof-of-concept study was not powered based on biopsy results; seven participants had histologically confirmed high-grade lesions resulting in 29% power to detect a sensitivity of 80% with a lower 95% confidence limit of 65%.

Statistical analyses

We calculated kappa statistics for clinician- and self-collected specimens for cytology and for HPV and estimated sensitivity and specificity, using colposcopy endpoints as the gold standard to define true cases. For this gold standard, true positives were defined as histologically confirmed high-grade cervical intraepithelial neoplasia (CIN2+) from biopsy or ECC. True negatives were defined as women who: 1. had normal cytology with both specimens but no colposcopy (n =83); 2. had normal findings during colposcopy visit (defined as negative or low-grade biopsy/ECC or satisfactory colposcopy with no acetowhite lesions visualized, n= 70); or 3. missed colposcopy, but had normal repeat cytology after 6 months or more at the index clinic if ASCUS or low grade squamous intraepithelial lesions (LSIL) at index visit (n =7). None of the ten women with normal results who underwent colposcopy to estimate ascertainment bias had abnormalities. Given that colposcopy was not used to confirm all negatives, we include adjusted estimates assuming that 1% of women [17] who had normal cytology with both specimens were missed cases of CIN2+. Additionally, we calculated secondary sensitivity and specificity estimates including CIN1 as true positives. We calculated 95% confidence intervals for sensitivity and specificity estimates using exact confidence intervals based on binomial probabilities.

We compared index cytology result by clinic, age and sexual activity. We compared age and initial cytology result of women who enrolled to those who refused or missed scheduled appointments and of the 167 women with validity results to those missing results. We used Fisher's exact tests to compare categorical outcomes and the Kruskal-Wallis test for non-parametric data to compare continuous outcomes. Analyses were conducted in Stata (StataCorp, College Station, TX).

RESULTS

Below are some of the results from the study

During recruitment months, 5,509 women underwent cytological screening at participating clinics and 5,479 had sufficient samples. Of these women, 198 (3.6%) eligible women enrolled, 202 (3.7%) declined participation, 122 (2.2%) missed their study appointment, 18 were ineligible (0.3%) and most (4,939, 90.1%) were never invited to participate in this convenience sample (Figure S1). Thus 38% of those invited declined participation, primarily citing lack of time as their reason, and an additional 23% did not come to their study visit.

The majority of women were Latina (85%) and sexually active in the last month (76%) with a median age of 31 (Table 1). Nineteen percent of the participants were post-menopausal. Women with normal index cytology results using clinician collected specimens were older than women with abnormal cytology (median age of 42 versus 24 years respectively, p,.001, Table 1). This pattern remained using the self-lavage cytology results; the median age for women with normal results on self-lavage was 32 versus 24 years for women with abnormal cytology, p = 0.01. Among the 98 women with abnormal cytology results, 54 (55%) had LSIL, 38 (39%) ASC-US, 4 (4%) high grade squamous intraepithelial lesion (HSIL), one (1%) atypical squamous cell cannot rule out HSIL (ASC-H) and one (1%) atypical glandular cells (AGC). The distribution of all abnormal results from participating clinics during recruitment months (n= 870) was similar: 46% LSIL, 48% ASC-US, 3% HSIL, 2% ASC-H and 1% AGC. Women who refused or missed their appointment were similar to participants for index cytology results, HPV results, and age. Women with missing versus definitive results (Figure S1) were also similar by age, index HPV, and index cytology. Results from colposcopy were used for definitive diagnosis for 46% (77/167) of women. The median time between index visit and colposcopy was 127 days (interquartile range, IQR: 69, 260).

Specimen collection and quality

Only one morbidly obese participant was unable to self-lavage, resulting in 197 specimens. The median fluid collected was 1.0 mL, ranging from 0.1 to 5.0 mL. Cytotechnologists coded 75% of specimens as having moderate to excellent cellularity, 23% low cellularity and 2% scant. Two of the Screener specimens (1%) were unsatisfactory for cytology, compared to 0.5% (30/5509) of clinician-collected specimens from participating clinics during the same time period (z-test, p = 0.51). A total of 195 self-lavage specimens were collected and readable for cytology.

The median number of days between clinician- and self-collected specimens was 60 days; the median for women whose index cytology was normal was longer than women with abnormal results (65 versus 55 days respectively, p = 0.02). Transformation zone cells were present on 93% of clinician-collected specimens compared to 18% of Screener specimens (p,.001). Transformation zone cells were less likely to be present using clinician-collected specimens for post-menopausal women: 81% of 38 postmenopausal women had transformation zone cells compared to 96% of 159 menopausal women (p = 0.006). However, the reverse was true using self-lavage specimens, 26% of the post-menopausal specimens showed transformation zone cells compared to 16% of the menopausal women (p = 0.01).

For QUESTION 14:

Discussion

Delphi Screener specimens showed moderate to high cellularity and comparable rates of sufficient specimens to clinician-collected specimens for cytology and HPV. The main difference in specimen quality was the presence of transformation zone cells: over 90% of clinician-collected specimens contained these cells compared to 18% of Screener specimens. Cross-sectional studies have found an association between presence of these cells, especially endocervical cells, and concurrent abnormal cytological findings [18]. Longitudinal studies, however, have found no association between the absence of transformation zone cells among women with negative cytology and subsequent high grade lesions [19,20]. Furthermore, the validity of HPV testing is independent of the presence of transformation zone cells [21]. Therefore, the absence of these cells may not be an important indicator of specimen quality for use in cervical cancer screening. However, the utility of transformation zone cells for women 30 years of age and under has not been well-studied.

Agreement between the clinician- and Screener-collected cytology result was low (kappa =0.36), primarily due to women with ASC-US and LSIL from clinician-collected specimens diagnosed as normal using Screener specimens. Agreement for cervical cytology with clinician-collected specimens also tend to be low; one study reported kappas of 0.26 to 0.40 among six cytologists reading the same 70 slides for specimen adequacy alone [22]. Another study of 117 abnormal slides, found kappas of 0.39 to 0.57 among seven cytologists, depending on classification system used [23]. The fairly low kappa in this study may be an indicator of low reliability for cervical cytology in general rather than low reliability from using the Screener specimen, although a larger study is needed to ensure that high grade cases are not systematically missed.

As expected women who had abnormal cytology results (primarily ASC-US and LSIL) were, on average, younger than women with normal cytology results. The kappa for cytology improved with older age, as would be anticipated given fewer abnormal findings. Future studies should be designed to produce separate estimates of the performance of self-lavage for women aged 30 years and under versus over 30.

Agreement of high-risk HPV testing between the Screener and clinician-collected specimen was moderate (kappa = 0.66), and comparable to an earlier study using the GP5+/6+ polymerase chain reaction HPV test with the Screener (kappa = 0.71) [24]. These findings are also similar to estimates for self-collected tampons or swabs for HPV testing in a recent meta-analysis (kappa = 0.66) [6], suggesting the Screener is comparable to other self-sampling methods for high-risk HPV testing.

Clinically, more important than agreement between the two specimens is their ability to detect abnormalities in women with CIN2+. Our preliminary point estimate of the sensitivity of the Screener using cytology was good even after adjusting for possible ascertainment bias (75.0%), although the 95% confidence interval was wide with a lower bound of 35%. The point estimate for the specificity of the Screener specimen for cytology was higher than that of clinician-collected specimens. While these preliminary estimates are promising, a larger study is needed to better estimate the extent to which high grade cases might be missed before relying on a Screener specimen alone for cytology.

The idea to use self-lavage for cytology is not new. In the 1960s, a number of studies tested the use of the Davis cytopipette or irrigation smear for self-lavage for cytological smears [25-29]. While initial results were promising [26], ultimately high proportions of unsatisfactory specimens, as high as 37% [29], coupled with low sensitivity, prohibited further use of the device in cervical cancer screening. In one study, 5 of 13 histologically confirmed cervical cancer cases tested negative using the irrigation smear [29]. In the current study, however, six of the 7 histologically confirmed cases of CIN2+ and 11 of the 15 histologically confirmed cases of CIN1 had abnormal cytology results using the self-lavage, suggesting further study of the Screener for cytology is warranted. Further, most women (79%) reported preferring self-lavage over clinician-collection for future cervical cancer screening [30].

The present study had a number of limitations. The study was not sufficiently powered based on colposcopy results for endpoints; this study tested proof-of-concept for use of Screener self-lavage specimens for cervical cytology. Colposcopy was not used on all women, and sixteen women who received colposcopy did not have a specimen taken for histology. Additionally women were not followed for a sufficiently long time to produce robust specificity estimates; clearly some high grade lesions may have been missed. Missing more than 1% of negatives is highly unlikely in this low-risk population however, and adjusted sensitivity and specificity calculations for the Screener specimen remain reasonable. Finally, while the same two cytotechnologists diagnosed all of the self-lavage specimens, index clinician-collected specimens were diagnosed by a larger pool of cytotechnologists available at the hospital laboratory; some disagreements in cytological findings could be caused by this difference in cytotechnologists.

Despite the low reliability of cytology between clinician- and self-collected specimens, most high-grade lesions were caught and reliability of HPV testing was good. A larger study to estimate the performance of the Screener for HPV testing with cytology triage or co-testing is warranted and could result in identifying an important new tool to increase cervical cancer screening uptake in hard-to-reach populations. Cost of the device would also need to be considered prior to large-scale introduction.

Summary

In summary, the aim of this study was to compare the Delphi Screener self-lavaging device's validity and reliability to specimens obtained by clinicians using a liquid-base cytology and HPV that tests for cervical cancer. One hundred and ninety-eight women were included in the study which lasted from December 1, 2008 until January 25, 2010 (13 months). Participants were a convenient sample of women attending three obstetrical clinics at Columbia University. The inclusion criteria among others, included women at least 18 years of age, not pregnant, not breastfeeding did not have a hysterectomy and were able to read English or Spanish well. For the purposes of recruitment, a case was defined as a person with atypical squamous cells of undetermined significance (ASC-US) or worse. The study adhered to ethical standards by the clinicians asking their patients for permission to share their contact information with recruiters, they provided the participants with written informed consent and the study received IRB approval from Columbia University. The investigators controlled for bias by ensuring that the specimens were processed within a specific time-frame that maintained the integrity of the specimen to reduce the likelihood of false negatives, double blinding was used in the laboratory testing and a subset of women with normal cytology were invited for colposcopy to prevent ascertainment bias. The authors justified their sample size by indicating that they based the sample size of the index cytology that had a sensitivity of 80% and a lower 95% confidence interval of 65% and the study was powered at 29%. In the statistical analyses, colposcopy was used as the gold standard to identify true cases. Using this as the gold standard, the clinician-collected cytology had a sensitivity of 7/7=100% and a specificity of 93/160=58.1% while the Screener-collected cytology had a sensitivity of 6/7=85.7% and specificity of 126/160=80.0%. When the Kappa coefficient was comparedfor cytology results obtained via clinician-collected specimens and Screener-collected specimens the kappa was fair at 0.39.

Jones HE, Mansukhani MM, Tong G-X, Westhoff CL (2013) Validity and Reliability of Using a Self-Lavaging Device for Cytology and HPV Testing for Cervical Cancer Screening: Findings from a Pilot Study.PLoS ONE8(12): e82115. doi:10.1371/journal.pone.0082115

ANSWERS:

A.) I have not read the discussion and summary

B.) I have read the discussion and summary

What was the duration of the study? The study lasted from December 1, 2008 until August 31, 2009 (8 months) The study lasted from December 1, 2008 until January 25, 2010 (13 months] August 31, 2009 until January 25, 2010 (4 months) Question 4 7.14 pts What type of sample did the investigators select? A simple random sample A stratied sample A convenient sample A multistage sample Question 5 7.14 pts What was the inclusion criteria for the study? Please check all that apply O Able to read English or Spanish Breastfeeding Not pregnant At least 18 years of age Had a hysterectomy Question 6 7.14 pts What case definition was used for abnormal cytology as a condition for recruitment into the study? Low grade squamous intraepithelial lesions (LSIL) or worse Colposcopy with acetowhite lesions O Atypical squamous cells of undetermined significance (ASC-US) or worse O Abnormal endocervical curetting (ECC)Question 7 7.14 pts How did the investigators ensure that the study adhered to ethical standards? Only persons at least 18 years Of age were recruited The study received IRB approval from Columbia University The clinicians asked their patients for permission to share their contact information with recruiters. Patients were given written informed consent and the protocol was approved by Columbia University's IRB Participants were provided with written informed consent Question 8 7.14 pts How did the investigators control for biases in the study? (Check all that apply) Blinding was used in the laboratory Specimens were processed within a specic timefra me Bias was not an issue in this study Inviting a subset of women with normal cytology along with women with abnormal cytology for colposcopy Question 9 7.14 pts Did the authors justify (explain how they arrived at) their sample size? x"'\\. Yes /_'\\. N0 Question 10 7.14 pts What was the gold standard that was used in the statistical analysis? Cytology Colposcopy Question 11 7.14 pts Please review the table below and calculate the sensitivity and specificity for the clinician-collected cytology Colposcopy/repeat cytology (n=167) High grade (CIN 2+) Normal Clinician-collected cytology Abnormal 7 67 Normal O 93 Total 7 160 Sensitivity/specificity (85.7%/80.0%) O Sensitivity/specificity (100%/58.1%) Sensitivity/specificity (3.6%/76.6%) Sensitivity/specificity (4.2%/55.6%)Question 12 7.14 pts Please review the table below and calculate the sensitivity and specificity of the screener collected cytology Colposcopy/repeat cytology (n=167) High grade (CIN 2+) Normal Screener-collected cytology Abnormal 6 32 Normal 1 128 Total 7 160 O Sensitivity/specificity (80.0%/85.7%) O Sensitivity/specificity (100%/58.1%) O Sensitivity/specificity (85.7%/80.0%) O Sensitivity/specificity (3.4%/76.6%)Question 13 7.14 pts The kappa coefficient between cytology result using clinician-collected specimens and cytology result using Screener specimen was 0.39. Would this be considered to be a poor agreement or not? O a fair response O a poor response