Skip to main content

Evaluation of the diagnostic accuracy of a new point-of-care rapid test for SARS-CoV-2 virus detection

Abstract

Background

The easy access to a quick diagnosis of coronavirus disease 2019 (COVID-19) is a key point to improve the management of severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) and to contain its spread. Up to now, laboratory real-time PCR is the standard of care, but requires a fully equipped laboratory and significant infrastructure. Consequently, new diagnostic tools are required.

Methods

In the present work, the diagnostic accuracy of the point-of-care rapid test "bKIT Virus Finder COVID-19" (Hyris Ltd) is evaluated by a retrospective and a prospective analysis on SARS CoV-2 samples previously assessed with an FDA “authorized for the emergency use—EUA” reference method. Descriptive statistics were used for the present study.

Results

Results obtained with the Hyris Kit are the same as that of standard laboratory-based real time PCR methods for all the analyzed samples. In addition, the Hyris Kit provides the test results in less than 2 h, a significantly shorter time compared to the reference methods, without the need of a fully equipped laboratory.

Conclusions

To conclude, the Hyris kit represents a promising tool to improve the health surveillance and to increase the capacity of SARS-CoV-2 testing.

Background

The infection from severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2), and the resulting coronavirus disease 2019 (COVID-19) present important diagnostic challenges [1, 2]. In particular, improved access to diagnostics is a key point to counteract the spread of the virus [3]. Different diagnostic strategies are now available to identify or exclude a current infection, identify people in need of care escalation, or to test for past infection and immune response [4, 5].

Laboratory real-time (RT)-PCR represents the standard of care for the detection of the SARS-CoV-2 infection [6, 7], but this technique is time-consuming (up to 24 h for the result), not always available, and the technical requirements usually can only be met by centralized diagnostic laboratories.

Point-of-care diagnostics tests to detect current SARS-CoV-2 infection have the potential to allow an earlier detection of infection, compared to laboratory-based diagnostic methods, thus contributing to the reduction of household and community transmission [8,9,10]. A rapid diagnosis is also crucial for setting up a good therapy [11, 12]. In addition, they can be operated in near-patient settings rather than in the laboratory, which are expected to be more easily accessible and to relieve laboratory workload. For this reason, these kinds of rapid tests were identified by a WHO expert group as one of the priorities in response to the COVID-19 outbreak [13].

The aim of the present study is to evaluate the diagnostic accuracy of the point-of-care rapid test "bKIT Virus Finder COVID-19" produced by Hyris Ltd (hereafter, termed Hyris Kit) for the detection of the SARS-CoV-2 virus, against the used reference method (SARS-CoV-2 RNA research by the RT-PCR method). The Hyris Kit is designed to be easy and quick to use in emergency condition or in delocalized areas; thus, this work is intended also to ensure that the time required to run the test, providing the same sensitivity and specificity, is less than the reference method.

Material and methods

Study design and sample collection

The diagnostic accuracy of the Hyris Kit was evaluated by a retrospective and a prospective analysis on SARS-CoV-2 samples previously assessed (and validated positive and negative) by an FDA “authorized for the emergency use—EUA” reference method. The study was conducted in the Department of Laboratory Medicine of the National Institute for The Study and Treatment of Cancer IRCCS "Fondazione G Pascale" (IRCCS Pascale) of Naples (Italy).

For the retrospective analysis, SARS CoV-2-positive nasopharyngeal swab samples were taken by during the pandemic peak period (from March to June 2020) and were provided by the Azienda Ospedaliera Universitaria Federico II (Federico II University Hospital) and the Azienda Ospedaliera dei Colli—Ospedale Domenico Cotugno (Cotugno Hospital) of Naples (Italy). SARS-CoV-2-negative nasopharyngeal swab samples were collected in July 2020 by the IRCCS Pascale.

For the prospective analysis, new positive samples were prepared by Federico II University Hospital starting from pools of one or two positive samples, diluted in a negative sample (pooled samples).

All participant patients were aged  ≥ 18 years; they understood and signed the informed consent form. The study protocol was approved by the Ethics Committee of IRCCS Pascale.

Reference analysis methods

The reference methods, used to verify the positivity or negativity of collected samples for the retrospective analysis were: (1) Abbott Kit: Real-Time SARS-CoV-2 (#09N77-095), EUA by the FDA, where a complete analysis run lasts for 6 h on average, which detect the RdRP and the N gene of the SARS-CoV-2; and (2) Roche Kit: LightMix®Modular SARS-Cov-2 (COVID19) RdRP, where a complete analysis run lasts for 3 h on average, which detects the RdRP and the E gene of the SARS-CoV-2. Using the Roche Kit, samples are rated negative if they have a late amplification beyond 35 threshold cycle (Ct, Kit cut-off) or no amplification for the target gene.

Positive samples by the Federico II University Hospital and negative samples by IRCCS Pascale were analyzed with the Abbott Kit following the manufacturer’s instructions. Positive samples by Cotugno Hospital were analyzed with the Roche Kit following the manufacturer’s instructions.

The Roche Kit, which needs 200 μL of sample, was used in place of the Abbott Kit if the sample volume was not enough for the specifications of the Abbott Kit, which requires 800 μL. The prospective analysis on pooled samples was first carried out with the Hyris Kit, and then with the Abbott kit.

All samples were analyzed in compatible soil (universal transport media/viral transport medium) and then stored at − 80 °C.

Hyris Kit point-of-care test

The Hyris Kit is intended for use on the bCUBE instrument, a miniaturized real-time PCR for the in vitro qualitative detection of SARS-CoV-2 nucleic acid in nasopharyngeal swabs or nasal swabs specimens (Fig. 1). No sample extraction is needed prior to the PCR amplification step. The Hyris Kit contains reagents that allow the sample direct amplification in the analysis well. These reagents are designed to reduce the impact of the RNase contained into the clinical specimen, as well as to promote the nucleic acid extraction before the retro transcriptase step. To allow a quick detection of the disease, without necessarily going through an analysis laboratory, all the solutions required for the analysis process are premixed within the kit. After the nasopharyngeal or nasal swab collection, a small aliquot of the swab transport medium (5 μL), which contains the specimen, can be directly added to 15 μL of the reaction mix, previously loaded into the bCUBE cartridge well (16 samples on each standard cartridge, 36 samples cartridge will be soon available). The sample will be amplified as per the amplification kit procedure. During the RT step of the RT-PCR thermal protocol, the viral RNA will be released into the qPCR master mix. The direct amplification RT-PCR assay is performed in the bCUBE. The bCUBE is designed to be fully portable and to work in network and the embedded automatic results interpretation delivers a test output readable by non-experts. At the end of the analysis, a “detected” or “not detected” result can be assigned to each sample for positive and negative samples, respectively. The Hyris Kit automatic interpretation algorithm classifies all those samples that do not show amplification for viral targets and very late amplification for endogenous control, namely the human gene RPP30 (i.e., the ubiquitous gene that encodes for RNase P), as "indeterminate". Late amplification of the RPP30 gene may depend on a poorly performed sample, this can alter the outcome of the analysis in the case of a patient with a viral load very close to the limit of the detection of the method, causing cases of "false negativity". Therefore, for the present study, samples with an "indeterminate" outcome are excluded from the trial.

Fig. 1
figure1

Hyris Kit schematic of the workflow

Samples analyzed with the Hyris Kit that have the amplification of only one of the two targets of the SARS-CoV-2 virus are classified as "inconclusive". Samples with "inconclusive" results are excluded from the trial for the present study.

In case of positive samples that showed a no amplification signal or showed partial amplification during retrospective analysis with the Hyris kit, analyses were repeat with the Roche Kit.

Determination of the lower limit of detection of the Hyris kit

Two positive samples provided by the Federico II, analyzed with the Hyris Kit and evaluated positive, were diluted and tested in technical duplication in order to identify a concentration of analyte beyond the limit of detection of the method of analysis. All dilutions where one or more replicated were not properly amplified (both targets for both replicated) have been discarded. Having defined the dilution to be tested, the samples were processed with the Roche Kit to identify the correlation between the Ct obtained with Hyris Kit, and the Roche Kit, in order to exclude all samples that have Ct values greater than the threshold value identified, which corresponds to 33 Ct.

Blinded analysis

To perform a blinded analysis with Hyris kit, a selection of positive and negative samples was prepared and by IRCCS Pascale staff and provided anonymously for analysis to Hyris kit staff, who carried out the analysis. At the end of the tests, IRCCS Pascale's staff disclosed all the information for data analysis.

Statistical analysis

Descriptive statistics were used for the present study.

Results

Study samples

For the retrospective analysis, a total of 37 positive samples were provided by the Federico II University Hospital. A total of the 206 negative samples were provided by IRCCS Pascale.

A total of 20 positive samples were provided by Cotugno Hospital and were used for the blind trial test. Baseline characteristics of the relative patients are summarized in Table 1.

Table 1 Baseline characteristics of patients

For the prospective analysis, 30 new positive samples were provided by the Federico II University Hospital, by using the pool method.

Evaluation of Hyris Kit diagnostic accuracy

Retrospective analysis

Of 37 positive samples provided by Federico II, three samples gave “inconclusive” results with Hyris kit, meaning one in two viral targets was not amplified correctly. In total, 11 samples gave no amplification signals with the Hyris kit. These 14 samples that failed the Hyris Kit test were retested with Roche Kit, as it was assumed that the samples could be degraded due to repeated thawing, the time and the conditions of storage. Analysis with Roche Kit confirmed that these 14 samples had failed, so they were not considered in this study.

The remaining 23 positive samples provided by Federico II were tested positive by the Hyris Kit (see Additional file 1: Table S1 for details of the results).

Considering the negative samples, 203 were classified as negative with the Hyris Kit. Three samples, while being negative for virus markers, were interpreted by the Hyris Kit automatic interpretation algorithm as "indeterminate", consequently were not considered in the present study (see Additional file 1: Table S2 for more details).

Prospective analysis

All 30 positive samples obtained from pools of different samples resulted positive with the first Hyris Kit test and subsequent Abbott Kit test, performed to confirm the result (see Additional file 1: Table S3 for details).

Blind trial test

Of the 20 positive samples provided by Cotugno Hospital, four samples were excluded before the start of the trial because they met one or more of the exclusion criteria.

Consequently, in the blind trial 60 samples were analyzed: 16 positive samples provided by the Cotugno Hospital, 14 positive pool samples and 30 negative samples.

Of the 16 positive samples tested from the Cotugno Hospital, six were excluded. In particular, two positive samples were excluded from the trial due to Ct greater than the determined threshold value (33 Ct), two samples because of negative results, and two because they resulted as “inconclusive” with the Hyris Kit.

The results obtained in the tests conducted with Hyris Kit on the remaining 54 samples are equivalent to those obtained with the reference methods (see Additional file 1: Table S4 for more details).

Discussion

Even if current medical knowledge on the management of COVID-19 patients and experimental treatments are still evolving, different protocols to minimize the risk of infection among the general population [14], patients and healthcare workers have been approved and diffused by International Health Authorities [15]. In this context, the development of point-of-care rapid tests to consent an earlier detection of SARS-CoV-2 infection is an urgent need, to allow an effective and rapid plan of diagnosing and to prevent further transmission [16].

This study was designed to evaluate the diagnostic accuracy of the point-of-care rapid test Hyris Kit.

In total, 63 positive samples and 203 negative samples were included in the study. Results show that the point-of-care Hyris Kit provides the same results of standard laboratory-based RT-PCR methods for all the analyzed samples. Observing the comparative results (see Additional file 1), the Ct are different among the different methods of analysis. The several reasons that explain these data includes: the absence of the purification of nucleic acids step for the Hyris kit (direct amplification); the different genes analyzed by the different methods, which are expressed at different stages of the pathology; the possible process of degradation that the samples may have encountered during the period between the analysis performed with the reference method and the test conducted with Hyris kit.

Of note, the Hyris Kit allows to get the results in less than 2 h, much quicker than the reference methods evaluated, maintaining the same sensitivity and specificity.

Therefore, the Hyris Kit diagnostic accuracy has been confirmed, along with its advantage in terms of time to results.

Another key advantage of this point-of-care platform is that it is a fully automated direct sample-to-answer platform, removing the need for the laboratory infrastructure required for traditional RT-PCR.

Therefore, this method can be used also in all medical or pediatricians’ offices since the availability of an equipped laboratory or other technological tools is no longer required.

Consequently, the results of the present study have a considerable socio-economic impact. Indeed, the Hyris Kit represents a method to promote a quick and easy health surveillance in all contexts in which a fully equipped laboratory is not accessible and rapid results are required (i.e., cruise ships, flights, patients to be hospitalized, sports teams) and responds to the request for the development of to use point-of-care assays, due to the urgent clinical and public health need to increase SARS-CoV-2 testing capacity.

For instance, the Hyris Kit has been recently approved as a point-of-care test in Canada for human diagnostics as an Interim Order.

Conclusion

The COVID-19 pandemic requires the set-up of solid and reliable diagnostic tests in order to ease the management of the pandemic spreading and its consequences. This work demonstrates the diagnostic accuracy of the point-of-care rapid test "bKIT Virus Finder COVID-19" (Hyris Ltd), sustaining the use of this promising tool to improve the health surveillance and to increase SARS-CoV-2 testing capacity.

Availability of data and materials

The datasets used and/or analyzed during the current study are available from the corresponding author on reasonable request.

Abbreviations

SARS-CoV-2:

Severe acute respiratory syndrome coronavirus 2

COVID-19:

Coronavirus disease 2019

RT:

Real-time

Hyris Kit:

BKIT Virus Finder COVID-19

Ct:

Threshold cycle

References

  1. 1.

    Zhou P, Yang XL, Wang XG, Hu B, Zhang L, Zhang W, Si HR, Zhu Y, Li B, Huang CL, Chen HD, Chen J, Luo Y, Guo H, Jiang RD, Liu MQ, Chen Y, Shen XR, Wang X, Zheng XS, Zhao K, Chen QJ, Deng F, Liu LL, Yan B, Zhan FX, Wang YY, Xiao GF, Shi ZL. A pneumonia outbreak associated with a new coronavirus of probable bat origin. Nature. 2020;579(7798):270–3. https://doi.org/10.1038/s41586-020-2012-7 (PMID: 32015507).

    CAS  Article  PubMed  PubMed Central  Google Scholar 

  2. 2.

    Coronaviridae Study Group of the International Committee on Taxonomy of Viruses. The species severe acute respiratory syndrome-related coronavirus: classifying 2019-nCoV and naming it SARS-CoV-2. Nat Microbiol. 2020;5:536–44.

    Article  Google Scholar 

  3. 3.

    Corman VM, Landt O, Kaiser M, Molenkamp R, Meijer A, Chu DK, Bleicker T, Brünink S, Schneider J, Schmidt ML, Mulders DG, Haagmans BL, van der Veer B, van den Brink S, Wijsman L, Goderski G, Romette J-L, Ellis J, Zambon M, Peiris M, Goossens H, Reusken C, Koopmans MP, Drosten C. Detection of 2019 novel coronavirus (2019-nCoV) by real-time RT-PCR. Euro Surveill. 2020;25(3):2000045. https://doi.org/10.2807/1560-7917.ES.2020.25.3.2000045.

    Article  PubMed Central  Google Scholar 

  4. 4.

    Erensoy S. COVID-19 pandemisinde SARS-CoV-2 ve Mikrobiyolojik Tanı Dinamikler [SARS-CoV-2 and microbiological diagnostic dynamics in COVID-19 pandemic]. Mikrobiyol Bul. 2020;54(3):497–509. https://doi.org/10.5578/mb.69839.

    Article  PubMed  Google Scholar 

  5. 5.

    D’Cruz RJ, Currier AW, Sampson VB. Laboratory testing methods for novel severe acute respiratory syndrome-coronavirus-2 (SARS-CoV-2). Front Cell Dev Biol. 2020;8:468. https://doi.org/10.3389/fcell.2020.00468.

    Article  PubMed  PubMed Central  Google Scholar 

  6. 6.

    Anahtar MN, Shaw B, Slater D, Byrne EH, Botti-Lodovico Y, Adams G, Schaffner SF, Eversley JS, McGrath G, Gogakos T, Lennerz J, Marble HD, Ritterhouse LL, Batten J, Georgantas NZ, Pellerin R, Signorelli S, Thierauf J, Kemball M, Happi C, Grant DS, Ndiaye D, Siddle KJ, Mehta SB, Harris JB, Ryan ET, Pierce VM, LaRocque RC, Lemieux J, Sabeti P, Rosenberg E, Branda J, Turbett SE. Development of a qualitative real-time RT-PCR assay for the detection of SARS-CoV-2: a guide and case study in setting up an emergency-use, laboratory-developed molecular assay. medRxiv. 2020. https://doi.org/10.1101/2020.08.26.20157297.

    Article  PubMed  PubMed Central  Google Scholar 

  7. 7.

    Liu R, Han H, Liu F, Lv Z, Wu K, Liu Y, Feng Y, Zhu C. Positive rate of RT-PCR detection of SARS-CoV-2 infection in 4880 cases from one hospital in Wuhan, China, from Jan to Feb 2020. Clin Chim Acta. 2020;505:172–5. https://doi.org/10.1016/j.cca.2020.03.009.

    CAS  Article  PubMed  PubMed Central  Google Scholar 

  8. 8.

    Younes N, Al-Sadeq DW, Al-Jighefee H, Younes S, Al-Jamal O, Daas HI, Yassine HM, Nasrallah GK. Challenges in laboratory diagnosis of the novel coronavirus SARS-CoV-2. Viruses. 2020;12(6):582. https://doi.org/10.3390/v12060582.

    CAS  Article  PubMed Central  Google Scholar 

  9. 9.

    Green K, Graziadio S, Turner P, Fanshawe T, Allen J. Oxford COVID-19 Evidence Service Team. Oxford COVID-19 Evidence Service: molecular and antibody point-of-care tests to support the screening, diagnosis and monitoring of COVID-19. Oxford COVID-19 Evid Serv Team, 2020. https://www.cebm.net/wp-content/uploads/2020/04/POCT-Covid19.pdf. Accessed 9 Dec 2020.

  10. 10.

    Dinnes J, Deeks JJ, Adriano A, Berhane S, Davenport C, Dittrich S, Emperador D, Takwoingi Y, Cunningham J, Beese S, Dretzke J, di FerranteRuffano L, Harris IM, Price MJ, Taylor-Phillips S, Hooft L, Leeflang MM, Spijker R, Van den Bruel A, Cochrane COVID-19 Diagnostic Test Accuracy Group. Rapid, point-of-care antigen and molecular-based tests for diagnosis of SARS-CoV-2 infection. Cochrane Database Syst Rev. 2020;8:013705. https://doi.org/10.1002/14651858.CD013705.

    Article  Google Scholar 

  11. 11.

    Cantini F, Goletti D, Petrone L, Najafi Fard S, Niccoli L, Foti R. Immune therapy, or antiviral therapy, or both for COVID-19: a systematic review. Drugs. 2020. https://doi.org/10.1007/s40265-020-01421-w.

    Article  PubMed  PubMed Central  Google Scholar 

  12. 12.

    Tortorici MA, Beltramello M, Lempp FA, Pinto D, Dang HV, Rosen LE, McCallum M, Bowen J, Minola A, Jaconi S, Zatta F, De Marco A, Guarino B, Bianchi S, Lauron EJ, Tucker H, Zhou J, Peter A, Havenar-Daughton C, Wojcechowskyj JA, Case JB, Chen RE, Kaiser H, Montiel-Ruiz M, Meury M, Czudnochowski N, Spreafico R, Dillen J, Ng C, Sprugasci N, Culap K, Benigni F, Abdelnabi R, Foo SC, Schmid MA, Cameroni E, Riva A, Gabrieli A, Galli M, Pizzuto MS, Neyts J, Diamond MS, Virgin HW, Snell G, Corti D, Fink K, Veesler D. Ultrapotent human antibodies protect against SARS-CoV-2 challenge via multiple mechanisms. Science. 2020;370(6519):950–7. https://doi.org/10.1126/science.abe3354.

    Article  PubMed  Google Scholar 

  13. 13.

    WHO. COVID 19 public health emergency of international concern (PHEIC) global research and innovation forum: towards a research roadmap. Geneva: World Health Organization, 2020. https://www.who.int/. Accessed 9 Dec 2020.

  14. 14.

    Santacroce L, Charitos IA, Ballini A, Inchingolo F, Luperto P, De Nitto E, Topi S. The human respiratory system and its microbiome at a glimpse. Biology (Basel). 2020;9(10):318. https://doi.org/10.3390/biology9100318.

    Article  Google Scholar 

  15. 15.

    Charitos IA, Ballini A, Bottalico L, Cantore S, Passarelli PC, Inchingolo F, D’Addona A, Santacroce L. Special features of SARS-CoV-2 in daily practice. World J Clin Cases. 2020;8(18):3920–33. https://doi.org/10.12998/wjcc.v8.i18.3920.

    Article  PubMed  PubMed Central  Google Scholar 

  16. 16.

    Pham VH, Gargiulo Isacco C, Nguyen KCD, Le SH, Tran DK, Nguyen QV, Pham HT, Aityan S, Pham ST, Cantore S, Inchingolo AM, Inchingolo AD, Dipalma G, Ballini A, Inchingolo F. Rapid and sensitive diagnostic procedure for multiple detection of pandemic Coronaviridae family members SARS-CoV-2, SARS-CoV, MERS-CoV and HCoV: a translational research and cooperation between the Phan Chau Trinh University in Vietnam and University of Bari “Aldo Moro” in Italy. Eur Rev Med Pharmacol Sci. 2020;24(12):7173–91. https://doi.org/10.26355/eurrev_202006_21713.

    CAS  Article  PubMed  Google Scholar 

Download references

Acknowledgements

Editorial assistance was provided by Simonetta Papa, PhD and Aashni Shah (Polistudium SRL, Milan, Italy). This assistance was supported by internal funds.

Funding

Not applicable.

Author information

Affiliations

Authors

Contributions

LM and AO conceived, designed and supervised the project. GP, LA, RC and FMB assisted with data collection. FL and EC performed the tests. PC and GDF performed the statistical analysis. All authors analyzed and interpreted the data. LM wrote the manuscript draft. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Leonardo Miscio.

Ethics declarations

Ethics approval and consent to participate

All participants signed the informed consent form. The study protocol was approved by the Ethics Committee of IRCCS Pascale (deliberation n. 732 – 15/07/2020) and is registered with no. 45/20.

Consent for publication

Not applicable.

Competing interests

The authors declare that they have no competing interests.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Additional file 1: Table S1.

Results obtained by analyzing the 37 positive samples provided by Federico II University Hospital and their inclusion or exclusion from the trial. Table S2. Results obtained by analyzing the 206 negative samples included in the trial, provided by IRCCS Pascale and their inclusion or exclusion from the trial. Table S3. Results obtained by analyzing the 30 pooled samples prepared by Federico II University Hospital and their inclusion or exclusion from the trial. Table S4. Results obtained by the blind analysis of the 60 samples and their inclusion or exclusion included from the trial.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Miscio, L., Olivieri, A., Labonia, F. et al. Evaluation of the diagnostic accuracy of a new point-of-care rapid test for SARS-CoV-2 virus detection. J Transl Med 18, 488 (2020). https://doi.org/10.1186/s12967-020-02651-y

Download citation

Keywords

  • SARS-CoV-2
  • COVID-19
  • Point-of-care rapid test
  • Health surveillance