Table 1 Critical Appraisal of Methodological Quality and Quality of reporting (CRIME-Q): categories, descriptions, and potential bias/impact on studies

1X

Peer review

QoR

Did the paper undergo peer review prior to publication? Peer review might be useful for detecting errors or fraud. Yes/No

With peer review: Bias against negative studies

Without: potential for errors and/or fraud

2X

Bench-top/laboratory work related to establishing model—reporting

QoR

Was the study’s bench-top protocol sufficiently described (transparent, reproducible)? If e.g., cells are involved, did the study e.g., present incubator settings, description in detail of how the cells were treated (transfection, irradiation, etc.)? or describe how the cells were handled? Or for instance, did the study describe how to obtain a certain genetic model given there is no commercially available animal model? Yes/Partly/No

In vivo results are highly influenced by in vitro/bench-top part of the study. If not transparent and reproducible this lessens the usability of the study

2Y

Bench-top/laboratory work related to establishing model—methodology

MQ

Was the bench-top protocol feasible and well performed in relation to the experiment? Was it likely that the intended aim could be obtained based on the bench-top method? Yes/Partly/No. Studies with poor reporting (2X) will have difficulty gaining a high 2Y because of low transparency and the ability to assess method quality

If the bench-top protocol was not feasible, TTRs may be misleading, and readers should use papers with caution

3X

Animals—reporting

QoR

Were the animals used in the experiment sufficiently described? Were all parameters: Type, breed, age, weight, and manufacturer sufficiently described? If type, age, and weight were sufficiently described OR if weight was missing, but the manufacturer was included a Yes was given. If only partly described, then Partly. and if we were not able to correctly identify the animals No was given. Yes/Partly/No

The importance of the description of animal type cannot be understated since the immunological profile differs from strain to strain, which influences results

3Y

Animals—Methodology

MQ

Did the study use similar baseline characteristics for the animals (age, weight, type)? Yes/Partly/No. Studies with poor reporting (3X) will have difficulty gaining a high 3Y because of low transparency and the ability to assess method quality

If animals were not homogenous, study results might vary, for instance, low weight might result in poorer survival, which skews results

3Z

Selection bias (baseline characteristics) (SYRCLE Item 2)

RoB

Was the distribution of relevant baseline characteristics balanced between groups? I.e., was the distribution of e.g., male:female ratio, species, strain, age, and weight equally distributed throughout groups? Yes/No/Unclear/NA. Not applicable to studies using only one group

Unequal groups in intervention studies can skew results – introduces variables that potentially affect study results

4Y

Sample size

MQ

Did the study include a calculation of sample size? Describe how it was calculated – at what power? Was it appropriate and well performed? Yes/Partly/No

Studies may prove under-/overpowered in terms of drug efficacy if too few/many animals were used

5X

In vivo design and performance—reporting

QoR

Description of the in vivo study part: Were the surgery, implantation/injection method, and duration (whole experiment) sufficiently described? Is the study transparent and reproducible? Yes/Partly/No

The results will be difficult to replicate if the study is poorly described. Meaning the study is difficult to properly be assessed as a useful base for further research

5Y

In vivo design and performance—methodology

MQ

Did the method seem feasible and well performed concerning the study’s aim and outcome and in contrast to other known literature? Is it likely that the in vivo study design influences the results—incomprehensive/insensible method? Yes/Partly/No

Studies with poor reporting (5X) will have difficulty gaining a high 5Y because of low transparency and the ability to assess method quality

A poor methodology can skew results making conclusions in relation to aims obsolete

5Z(1)

Selection bias (Sequence generation) (SYRCLE Item 1)

RoB

Was there a description of allocation (the process by which experimental units are assigned to experimental groups)—And was it appropriate? Not applicable to non-intervention studies. Yes/No/Unclear/NA

Unequal groups in intervention studies can skew results—introduces variables that potentially affect study results

5Z(2)

Performance bias (Random housing) (SYRCLE Item 4)

RoB

Were the animals randomly housed during the experiment? Yes/no/unclear. Not applicable to non-intervention studies. Yes/No/Unclear/NA

Some types of experiments are influenced by the location of housing, hence random assignment of placement could negate these issues

5Z(3)

Detection bias (Random outcome assessment) (SYRCLE Item 6)

RoB

Were animals randomly selected for outcome? For instance, If human endpoints (i.e., poor conditions, weight, etc.) were met and the investigators were not blinded, then the outcome cannot be assessed randomly. Not applicable to non-intervention studies. Yes/No/Unclear/NA

Bias toward assessing intervention effect size

6X

Compliance with animal welfare regulations

QoR

Did the study comply with any animal welfare regulations? Yes/Partly/No

Assurance of proper animal care throughout the study. Also important in terms of survival studies (human endpoints vs. death)

7X

Blinding

QoR

Was the study blinded in any way? Was the outcome assessed in a blinded fashion? Were the animals randomly selected across all groups of e.g., intervention? Were the investigator or animal handlers blinded?

More specific blinding is listed below in 7Z(1–3) Yes/Partly/No

Unblinded administrator of intervention can skew results in both a positive and negative direction

7Z(1)

Performance bias (Blinding) (SYRCLE Item 5)

RoB

Describe all used means, if any, to blind trial caregiver and researchers from knowing which intervention each animal received. Yes/No/Unclear/NA. Not applicable for non-intervention studies, however, it could be applicable for instance in xenograft studies, where multiple patient samples were used

Animal handling may be affected by unblinded study design

7Z(2)

Allocation bias (allocation concealment) (SYRCLE Item 3)

RoB

Could the investigator allocating the animals to intervention or control group not foresee assignment? Yes/No/Unclear/NA. Not applicable to non-intervention studies. Yes/no/unclear/NA. This could be applicable for instance in xenograft studies, where multiple patient samples were used

In relation to 7Z (1). Selection, handling, and treatment of animals may be affected if allocation concealment was not adequately performed

7Z(3)

Detection bias (blinding) (SYRCLE Item 7)

RoB

Was the outcome assessor blinded? and could the blinding have been broken? Describe all measures used, if any, to blind outcome assessors from knowing which intervention each animal received. Were the outcome assessment methods the same in each group? Yes/No/Unclear/NA. This could be applicable to instance in xenograft studies, where multiple patient samples where used

Measurement of the outcome can be over-/underestimated if proper blinded outcome assessment was not performed

8X

Congruency between methods and results

QoR

Did the study present all their findings based on the methods described? Is there congruency between the method and results sections? Yes/Partly/No

Presenting results in which methods are not described is not transparent and replicable and should be interpreted with caution

8Z(1)

Attrition bias (incomplete outcome data) (SYRCLE Item 8)

RoB

Describe the completeness of outcome data including attrition and exclusions from the analysis and were incomplete outcome data adequately described? Were all animals included in the analysis and if not, was it described why they were not included? Yes/No/Unclear

Attritions and/or exclusions should be clearly described, i.e., the number of animals used. If not, study results become difficult to assess. Poor replicability and transparency

8Z(2)

Reporting bias (Selective outcome reporting) (SYRCLE Item 9)

RoB

Was the study protocol available (require a description of protocol location in the record) and were all of the study’s pre-specified primary and secondary outcomes reported in the manuscript? Was the study protocol not available but was it clear that the published report included all expected outcomes (i.e., comparing methods and results sections)?The study report fails to include results for a key outcome that would be expected to have been reported for such a study, i.e. tumor-take rate in transplantation experiments. Yes/No/Unclear

Congruency between results and methods should be carefully described to avoid reporting bias. If key outcomes for a certain method were not described, study validity, transparency, and replicability become difficult

9X

Presentation of limitations

QoR

Did the study contain a section of limitations, or did they comment on the limitations of the study in relationship to in vitro and/or in vivo subparts? Yes/Partly/No

No study is without limitations, and it is paramount to present them to the reader for transparency’s sake

10X

Statement of potential conflict of interest

QoR

Did the study contain a statement of potential conflicts of interest? Yes/No

Potential conflicts of interest can skew results i.e. if an investigator has a method patent or is paid by a certain pharmaceutical company, hence it is important for transparency’s sake to include it in the study

10Z

Publication bias (influence) (SYRCLE Item 10)

RoB

Inappropriate influence of funders or biased by companies. Was the study free of inappropriate influence from funders or companies supplying drugs or equipment? Did the authors declare a direct conflict of interest in relation to the study? Yes: Conflict of interest statement with no conflicts of interest. Yes/No/Unclear

Publication bias – Negative results will be less likely to be published if inappropriate influence of funder or biased companies occur