Skip to main content

Can HPV Selfy be considered as a clinically validated HPV test for use in cervical cancer screening?

In primary cervical cancer screening, it is crucial to use only hrHPV tests that are clinically validated according to international guidelines in order to reduce the risks of missing relevant disease and of over-treatment. In the recent J Transl Med [1] paper, Avian et al. concluded that the HPV Selfy assay (Ulisse BioMed, Trieste, Italy) fulfils international validation criteria for hrHPV testing on clinician-collected cervical samples (Meijer guidelines) [2] as well as by extension on self-collected vaginal samples (VALHUDES) [3]. Our perception is that the study by Avian et al. has certain limitations that are worthy of consideration and which may call into question certain conclusions.

Validation requires an appropriately composed study population comprising a sufficient number of diseased subjects, derived from a continuous screening population or from a clearly described selection of CIN2+ cases and < CIN2 controls [4]. Avian et al. compiled cervical specimens for testing with HC2 (standard comparator test) and with the new HPV Selfy (index test) [1], but it remains unclear how the study population was composed. With 98 CIN2+ and 791 ≤ CIN1 subjects it was obviously not a continuous screening population, so more granularity on this would have been welcome. Additionally, detail on how non-disease was defined, which is essential for the evaluation of clinical specificity, was lacking. The reported absolute sensitivity for CIN2+ of the HC2 comparator test was 82.7%, which was substantially lower than the sensitivities observed in validation studies following the VALGENT or Meijer protocols included in a meta-analysis (Fig. 1) [5]. This may rise suspicion of a certain degree of histological over-classification. Nonetheless, we verified the data matrices in Table 2 in Avian et al. [1] and confirm the correctness of the non-inferiority statistics (Table 1).

Fig. 1
figure 1

Sensitivity of the HC2 assay (standard comparator used in validation of hrHPV assays) in studies included in the 2020 list of HPV assays validated for cervical screening [5] that applied the Meier [2] or VALGENT [4] validation protocols (on top) or included in the study of Avian et al. [1] (at the bottom)

Table 1 Computation of the relative specificity to exclude cervical intra-epithelial neoplasia of grade 2 or worse of Selfy on self-samples (SS) vs clinician-taken samples (clin) and non-inferiority statistics

The claim that HPV Selfy on self-samples was non-inferior to clinician-collected samples was flawed by critical statistical errors. The number of subjects with discordant self+ /clinician− and self−/clinician+ results (b and c cells in Table 4, in Avian et al. [1]) in the recommended formula for comparison of matched proportions were switched yielding reported p values < 0.05. Correct data entry would have generated non-inferiority p values 0.35 and 0.81 for sensitivity and specificity, respectively. The corresponding relative sensitivity and relative specificity for CIN2+ and 95% confidence intervals (not reported by authors) were 0.92 (95% CI 0.81–1.00) and 0.97 (95% CI 0.95–0.99), respectively, indicating non-significantly lower sensitivity and significantly lower specificity of HPV Selfy on self-versus clinician-collected samples.

Collaborations between science and industry are instrumental to advance clinical research, however contractual independency of researchers and autonomy of publication enhance scientific credibility. We observe that sixteen of thirty six authors (including the first and last) of the JTM paper are affiliated with the manufacturer of the assay. In the 2020 list of validated HPV assays [5], assays evaluated by test developers were down-graded to “partially validated” if all other validation criteria were fulfilled. This principle may also apply on the HPV Selfy assessment [1]. We recommend test developers, HPV experts and collaborating epidemiologists or statisticians to design validation studies according to internationally established protocols and evaluation methodologies. Journal editors should take this advice into account as well.

References

  1. Avian A, Clemente N, Mauro E, Isidoro E, Di Napoli M, Dudine S, Del Fabro A, Morini S, Perin T, Giudici F, et al. Clinical validation of full HR-HPV genotyping HPV Selfy assay according to the international guidelines for HPV test requirements for cervical cancer screening on clinician-collected and self-collected samples. J Transl Med. 2022;20:231.

    CAS  Article  Google Scholar 

  2. Meijer CJLM, Castle PE, Hesselink AT, Franco EL, Ronco G, Arbyn M, Bosch FX, Cuzick J, Dillner J, Heideman DA, Snijders PJ. Guidelines for human papillomavirus DNA test requirements for primary cervical cancer screening in women 30 years and older. Int J Cancer. 2009;124:516–20.

    CAS  Article  Google Scholar 

  3. Arbyn M, Peeters E, Benoy I, Vanden Broeck D, Bogers J, De Sutter P, Donders G, Tjalma W, Weyers S, Cuschieri K, et al. VALHUDES: a protocol for validation of human papillomavirus assays and collection devices for HPV testing on self-samples and urine samples. J Clin Virol. 2018;117:52–6.

    Article  Google Scholar 

  4. Arbyn M, Depuydt C, Benoy I, Bogers J, Cuschieri K, Schmitt M, Pawlita M, Geraets D, Heard I, Gheit T, et al. VALGENT: a protocol for clinical validation of human papillomavirus assays. J Clin Virol. 2016;76(Suppl 1):S14–21.

    Article  Google Scholar 

  5. Arbyn M, Simon M, Peeters E, Meijer C, Berkhof J, Cuschieri K, Bonde J, Ostrbenk Valencak A, Zhao FH, Rezhake R, et al. 2020 List of human papillomavirus assays suitable for primary cervical cancer screening. Clin Microbiol Infect. 2021;27:1083–95.

    CAS  Article  Google Scholar 

Download references

Acknowledgements

Not applicable.

Funding

M. Arbyn, and M. Poljak were supported by the Horizon 2020 Framework Programme for Research and Innovation of the European Commission, through the RISCC Network (Grant No. 847845), Scienanso, the employer of M Arbyn received funding from the European Society of Gynaecological Oncology, and the VALGENT and VALHUDES projects [3, 4]. J Bonde’s institution has received research funding or consumables at reduced price or for free to support research from BD Diagnostics, Agena Bioscience, Genomica SAU, LifeRiver Biotech and QIAGEN. He has received honoraria for lectures from BD Diagnostics and Hologic Ltd. JB is appointed member of the National Danish Cervical Screening Committee by the Danish Health Authority.

Author information

Authors and Affiliations

Authors

Contributions

MA wrote the initial manuscript, all other authors critically revised the manuscript. All authors approved the last version of the submitted manuscript. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Marc Arbyn.

Ethics declarations

Ethics approval and consent to participate

Not applicable.

Consent for publication

Not applicable.

Competing interests

K. Cuschieri declares no personal conflict of interest; her institution has received research funding or gratis consumables to support research from the following commercial entities in the last 3 years: Cepheid, Genomica, LifeRiver, Euroimmun, GeneFirst, SelfScreen, Qiagen, Hiantis, Seegene and Hologic.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Arbyn, M., Bonde, J., Cushieri, K. et al. Can HPV Selfy be considered as a clinically validated HPV test for use in cervical cancer screening?. J Transl Med 20, 422 (2022). https://doi.org/10.1186/s12967-022-03627-w

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s12967-022-03627-w

Keywords

  • Human papillomavirus
  • Cervical cancer screening
  • Diagnostic test accuracy
  • Validation of tests