|
 |
| RESEARCH ARTICLE |
|
|
|
| Year : 2019 | Volume
: 51
| Issue : 1 | Page : 11-16 |
| |
A study to assess the methodological quality of in vivo animal experiments published in Indian journal of pharmacology: A retrospective, cross-sectional, observational study
Sandeep Kumar Gupta
Department of Pharmacology, Heritage Institute of Medical Sciences, Varanasi, Uttar Pradesh, India
| Date of Submission | 13-Oct-2018 |
| Date of Acceptance | 01-Feb-2019 |
| Date of Web Publication | 19-Mar-2019 |
Correspondence Address:
Dr. Sandeep Kumar Gupta
Department of Pharmacology, Heritage Institute of Medical Sciences, Varanasi - 221 311, Uttar Pradesh
India
 Source of Support: None, Conflict of Interest: None  | Check |
DOI: 10.4103/ijp.IJP_536_18
BACKGROUND: Good reporting of experimental research is a vital part of research process. Although the reporting guidelines such as Animal research: Reporting in vivo experiments (ARRIVE) require the authors to stick to a standard format, they do not guarantee study reports' validity. For assessing the study reports validity, critical appraisal tools are required.
OBJECTIVE: The evaluation of the methodological quality of in vivo animal studies of Indian Journal of Pharmacology (IJP) published between 2011 and 2017 through critical appraisal was the primary objective. The secondary objective was to evaluate the adherence of selected studies to the ARRIVE guidelines.
MATERIALS AND METHODS: All in vivo animal studies published as full-text articles in IJP from January 2011 to December 2017 and satisfying the inclusion norms were included. A checklist based on the underlying principles of ARRIVE statement was applied to in vivo animal research (AR) published in IJP. For critical appraisal of reports, risk-of-bias domains were also applied on studies from in vivo AR.
RESULTS: One hundred and sixty-one studies satisfied the inclusion criteria. Seventy-three articles (45.34%, 95% confidence interval [CI]: 0.37–0.53) discussed about randomization procedure. Eighteen articles (11.18%, 95% CI: 0.07–0.16) gave details about blinding when assessing results. None of the studies provided details of sample size calculation. Eight articles (4.97%, 95% CI: 0.02–0.09) commented on the study limitations.
CONCLUSION: It was found that adherence to only some criteria of ARRIVE guidelines was subpar. There is a need for optimal reporting of random distribution of animals to experimental groups, concealment of allocation, blinded outcome evaluation, computation of sample size, and attrition of animals for improving the validity of AR.
Keywords: Animal research: reporting in vivo experiments guidelines, critical appraisal, in vivo animal research, methodological quality, risk-of-bias tool
How to cite this article:
Gupta SK. A study to assess the methodological quality of in vivo animal experiments published in Indian journal of pharmacology: A retrospective, cross-sectional, observational study. Indian J Pharmacol 2019;51:11-6 |
How to cite this URL:
Gupta SK. A study to assess the methodological quality of in vivo animal experiments published in Indian journal of pharmacology: A retrospective, cross-sectional, observational study. Indian J Pharmacol [serial online] 2019 [cited 2023 Jun 3];51:11-6. Available from: https://www.ijp-online.com/text.asp?2019/51/1/11/254587 |
| » Introduction | |  |
Replication of preclinical studies' results in humans has been disappointing. There may be two likely explanations behind this. To begin with, animals are intricate living creatures. Second, the methodological nature of animal research (AR) may be deficient, causing ambiguous outcomes.[1] There is considerable corroboration that reporting of experimental research is less than optimal.[2] Moreover, the low replication of ARs can be ascribed to insufficient reporting. Thus, optimal reporting of experimental research is an essential part of any research.[3] In view of these confirmations, there have been demands for improved reporting in AR.[4] The Animal research: Reporting in vivo experiments (ARRIVE) statement in 2010 epitomized a key step to ameliorate the reporting standard of AR.[5],[6] The ARRIVE guidelines recommend addition of the methodological factors that have been found to be inadequately reported in animal experiments. Application of ARRIVE may result in enhanced reporting quality of AR (though there is a report contrary to that).[7]
Although the reporting guidelines require the authors to stick to a standard format, they do not guarantee the study validity.[5] The poor reporting of animal experiments, due to compromised internal and external validity, has been implicated in the divergences between the results of preclinical and clinical studies.[8] Validity of the report can be evaluated by critical appraisal tools.[9] It is imperative to appraise the internal as well as external validity to judge whether outcomes of the study represent the source and target population. Internal as well as external validity can be assessed through critical appraisal.[10]
Indian Journal of Pharmacology (IJP) is a globally recognized and highly respected journal. Majority of studies published in IJP consist of in vivo AR. It was thus sensible to evaluate in vivo animal studies adherence to ARRIVE guidelines and to evaluate the methodological quality of the in vivo animal studies through critical appraisal.[11],[12]
Objective
The evaluation of the methodological quality of in vivo animal studies published in IJP from 2011 to 2017 through critical appraisal was the primary objective. The secondary objective of was to evaluate the adherence of the published in vivo animal studies to the ARRIVE guidelines.
| » Materials and Methods | | |
Study design
This was a retrospective, cross-sectional, observational study to evaluate the methodological quality of in vivo animal studies through critical appraisal and to examine their adherence to ARRIVE guideline. The ethical committee sanction was not obligatory because this study completely relies on the information accessible in the public domain and is a quality audit not incorporating sensitive information retrieval.
Data extraction
All in vivo animal studies published as full-text articles in IJP from January 2011 to December 2017 and fulfilling the eligibility criteria were eligible for inclusion. Eligible articles were identified by screening of titles, abstracts, keywords, and methodology.
Eligibility criteria
Inclusion criteria
(1) Animal studies with treatment intervention, (2) only original research articles, and (3) studies published between January 2011 and December 2017.
Exclusion criteria
(1) In vitro experiments, (2) studies on animals that were dead before inception of the study, (3) studies where no intervention was performed, (4) studies with no control group, and (5) review articles, commentaries, communications or correspondences, and short communication.
Assessment of adequacy of reporting and critical appraisal of evidences
The assessment of reporting of the in vivo animal experiments was done by guiding principles on the ARRIVE statement. A checklist was prepared dependent on the ARRIVE statement [Table 1], [Table 2], [Table 3], [Table 4], [Table 5].[6],[13] Some items on the checklists were added based on a study by Muhlhausler et al., 2013.[5] The checklist was applied to in vivo AR published in IJP. For critical appraisal of reports, risk-of-bias domains were also applied on studies from in vivo AR.[10],[14] The in-depth appraisal of AR's methodology, management, and analysis was done to determine its validity. | Table 1: ARRIVE checklist for title (item number 1), abstract (item number 2), introduction (item number 3), objectives (item number 4), and methods (item number 5 and 6)
Click here to view |
 | Table 4: ARRIVE checklist for methods (item number 12 and 13), results (item number 14)
Click here to view |
 | Table 5: ARRIVE checklist for results (item number 15, 16, 17), discussion (item number 18, 19 and 20)
Click here to view |
Data analysis
An exploratory and descriptive statistical analysis was done. Descriptive statistics was applied to illustrate the reporting of studies, as well as for the key methodological items. The SPSS statistical software package (SPSS for Windows, Version 16.0. Chicago, SPSS Inc.) was utilized for statistical analysis. Criteria's were demarcated as “yes” if they were unambiguously reported or “no” if they were indistinct or not stated at all. The lower and upper limits of the 95% confidence interval (CI) for the proportions was calculated.
| » Results | | |
Overall, 1185 articles from IJP between January 2011 and December 2017 were screened for suitability. [Figure 1] explains the retrieval and selection of articles in the present study. Out of the total 1185 articles, the screening of title and abstracts excluded 637 articles. These 637 articles included Drug watch (190); Letters to editor or correspondences (144); Short communications (118); Education forum and review articles (54); Editorials (46); and others (n = 85), i.e., Abstracts (29), Book reviews (18), Case reports (13), Obituaries (11), Errata (7), Announcements (4), Special article (2), and Journal club (1). Now, 548 articles were left and these articles were further evaluated.
One hundred and sixty-nine articles were omitted from 548 articles after full-text review. These 169 articles included randomized control trials (30); cross-sectional studies (25); pharmacokinetic/pharmacodynamics studies (22); drug utilization studies (20); questionnaire studies (19); observational studies (16); nonrandomized clinical trials (10); randomized uncontrolled clinical trials (10); systematic review/meta-analysis (6); case–control/cohort studies (5); drug safety studies (4); and pharmacoeconomic studies (2). Now, 379 preclinical articles were left, and these articles were further evaluated. A total of 218in vitro and other types of preclinical studies were omitted after evaluation of complete text. Only rest of 161 studies satisfied the inclusion prerequisites and thus incorporated in the study [Figure 1].
Out of 161 included articles, 33 articles (20.50%, 95% CI: 0.14–0.27) explained in what way the animal type being utilized can justify the methodical objectives. All the included articles clearly described objectives of the study. One hundred and forty-five articles (90.06%, 95% CI: 0.84–0.93) mentioned about the ethical permissions. Seventy-three articles (45.34%, 95% CI: 0.37–0.53) discussed about randomization procedure. Eighteen articles (11.18%, 95% CI: 0.07–0.16) described any type of blinding [Table 1]. One hundred and fifty-three articles (95.03%, 95% CI: 0.90–0.97) provided specifics of the animals used [Table 2]. None of the studies provided specifics about computation of sample size [Table 3]. All the articles provided details of statistical methods used [Table 4]. Eight articles (4.97%, 95% CI: 0.02–0.09) commented on the study limitations. Fifty-five articles (34.16%, 95% CI: 0.27–0.41) listed all funding sources [Table 5].
| » Discussion | | |
In critical appraisal, the most common characteristic contemplated is internal validity.[15] If the outcomes derived from study population are indicative of the source population, then the study is assumed to be valid internally. The major hazard to internal validity is the risk of bias.[10],[14] In in vivo AR, selection bias arises when nonrandom factors impact the distribution of animals to study interventions. Selection bias happens if detected variances between the groups in the study are attributable to the confounder rather than interventions. Randomization is the optimum tool for distribution of confounders across the intervention groups in equal proportion.[10] In this study, 45.34% of articles gave details about randomization. In a study by Kilkenny et al. 2009, 12% of all studies reported randomization of animals to experimental groups.[3] In a review by Hess 2011, 28% studies reported about randomization of animals.[16] In a methodological study of AR by Bara and Joffe 2014, 61% of all studies gave details about randomization.[1]
Performance bias happens when there are logical differences between experimental groups in susceptibility to determinant apart from the study intervention.[10] Allocation concealment can be used to prevent performance bias.[17] Random housing of the animal can also be used to prevent performance bias.[14] In this study, none of the studies reported allocation concealment mechanism. In a study by Bara and Joffe 2014, 6% of the studies reported allocation concealment.[1] Whereas in a study by Hirst et al. 2014, 15% of studies conveyed it.[18]
Detection bias can ensue if there are variances in outcome assessment methods between treatment groups.[10],[17] It can be avoided by blinding of the persons evaluating the outcome.[10] In this study, 11.18% of the study reported blinding. In a study by Bara and Joffe 2014, 40% of all studies stated about blinding.[1] Macleod et al. 2015 reported about blinding in 29.5% of animal studies.[19] In a study by Kilkenny et al. 2009, only 14% of all included study reported about blinding.[3]
Attrition bias is a problem in clinical research but the likelihood for discretionary attrition bias in in vivo AR is practically nil. However, nondiscretionary attrition can unintentionally introduce bias.[10] The commonly used tool to prevent attrition bias is reporting dropouts. In this study, animal number in each experimental group included in the analysis was given by all studies. None of the article in this study gave details about any dropout or explain why they were not included in the analysis. In the report by Hotwani et al. 2017, 91% of the study reported the animal number in each experimental group included in the analysis.[12]
External validity defines whether outcomes of the study represent the target population. O'Connor and Sargeant stated that if the study results are internally valid, then it will represent the target population.[10] There are no guidelines for assessing external validity of AR.[10] This study does not offer any insight into the external validity of the studies analyzed. This study had several limitations. The included studies were diverse with respect to the study outcomes, animals used, and disease types. Some studies may have failed to report some methodological details when in fact these were used. This study only assesses the reporting quality of assessed studies and did not aim to evaluate the quality of conduct.
| » Conclusion | | |
Overall, this study demonstrated good adherence of in vivo AR studies published in IJP with respect to the ARRIVE statement with few exceptions. It was found that adherence to only some criteria of ARRIVE guidelines was subpar. There is a necessity for optimal reporting of random distribution of animals to experimental groups, concealment of allocation, blinded outcome evaluation, computation of sample size, and reporting of attrition of animals for improving validity of AR.
Financial support and sponsorship
Nil.
Conflicts of interest
There are no conflicts of interest.
| » References | | |
| 1. | Bara M, Joffe AR. The methodological quality of animal research in critical care: The public face of science. Ann Intensive Care 2014;4:26.  |
| 2. | Steward O, Balice-Gordon R. Rigor or mortis: Best practices for preclinical research in neuroscience. Neuron 2014;84:572-81. |
| 3. | Kilkenny C, Parsons N, Kadyszewski E, Festing MF, Cuthill IC, Fry D, et al. Survey of the quality of experimental design, statistical analysis and reporting of research using animals. PLoS One 2009;4:e7824. |
| 4. | Landis SC, Amara SG, Asadullah K, Austin CP, Blumenstein R, Bradley EW, et al. Acall for transparent reporting to optimize the predictive value of preclinical research. Nature 2012;490:187-91. |
| 5. | Muhlhausler BS, Bloomfield FH, Gillman MW. Whole animal experiments should be more like human randomized controlled trials. PLoS Biol 2013;11:e1001481. |
| 6. | Kilkenny C, Browne WJ, Cuthill IC, Emerson M, Altman DG. Improving bioscience research reporting: The ARRIVE guidelines for reporting animal research. PLoS Biol 2010;8:e1000412. |
| 7. | Leung V, Rousseau-Blass F, Beauchamp G, Pang DS. ARRIVE has not ARRIVEd: Support for the ARRIVE (Animal research: Reporting of in vivo experiments) guidelines does not improve the reporting quality of papers in animal welfare, analgesia or anesthesia. PLoS One 2018;13:e0197882. |
| 8. | Sena ES, Currie GL, McCann SK, Macleod MR, Howells DW. Systematic reviews and meta-analysis of preclinical studies: Why perform them and how to appraise them critically. J Cereb Blood Flow Metab 2014;34:737-42. |
| 9. | van der Worp HB, Macleod MR. Preclinical studies of human disease: Time to take methodological quality seriously. J Mol Cell Cardiol 2011;51:449-50. |
| 10. | O'Connor AM, Sargeant JM. Critical appraisal of studies using laboratory animal models. ILAR J 2014;55:405-17. |
| 11. | Aiman U, Rahman SZ. Standard of reporting animal-based experimental research in Indian journal of pharmacology. Indian J Pharmacol 2015;47:514-7. [ PUBMED] [Full text] |
| 12. | Hotwani JH, Deshmukh KR, Mehta MN. Adherence to ARRIVE guidelines in animal research articles published in a National and International Pharmacology Journal. Int J Basic Clin Pharmacol 2017;6:1696-700. |
| 13. | Liu Y, Zhao X, Mai Y, Li X, Wang J, Chen L, et al. Adherence to ARRIVE guidelines in Chinese journal reports on neoplasms in animals. PLoS One 2016;11:e0154657. |
| 14. | Hooijmans CR, Rovers MM, de Vries RB, Leenaars M, Ritskes-Hoitinga M, Langendam MW, et al. SYRCLE's risk of bias tool for animal studies. BMC Med Res Methodol 2014;14:43. |
| 15. | Bailoo JD, Reichlin TS, Würbel H. Refinement of experimental design and conduct in laboratory animal research. ILAR J 2014;55:383-91. |
| 16. | Hess KR. Statistical design considerations in animal studies published recently in cancer research. Cancer Res 2011;71:625. |
| 17. | Higgins JP, Altman DG, Gøtzsche PC, Jüni P, Moher D, Oxman AD, et al. The cochrane collaboration's tool for assessing risk of bias in randomised trials. BMJ 2011;343:d5928. |
| 18. | Hirst JA, Howick J, Aronson JK, Roberts N, Perera R, Koshiaris C, et al. The need for randomization in animal trials: An overview of systematic reviews. PLoS One 2014;9:e98856. |
| 19. | Macleod MR, Lawson McLean A, Kyriakopoulou A, Serghiou S, de Wilde A, Sherratt N, et al. Risk of bias in reports of in vivo research: A Focus for improvement. PLoS Biol 2015;13:e1002273. |
[Figure 1]
[Table 1], [Table 2], [Table 3], [Table 4], [Table 5]
| This article has been cited by | | 1 |
Intraperitoneal drug delivery systems releasing cytostatic agents to target gastro-intestinal peritoneal metastases in laboratory animals: a systematic review |
|
| Anne G. W. E. Wintjens, Geert A. Simkens, Peter-Paul K. H. Fransen, Narcis Serafras, Kaatje Lenaerts, Gregor H. L. M. Franssen, Ignace H. J. T. de Hingh, Patricia Y. W. Dankers, Nicole D. Bouvy, Andrea Peeters | | Clinical & Experimental Metastasis. 2022; | | [Pubmed] | [DOI] | | | 2 |
Periodontal Tissue Reaction Consecutive Implantation of Endodontic Materials and Subsequent Integration of Complex Oral Rehabilitation Treatments |
|
| Liana Aminov, Aurelian Sorin Pasca, Eusebiu Viorel Sindilar, Maria Antonela Beldiman, Bogdan Petru Bulancea, Ovidiu Stamatin, Iulian Costin Lupu, Irina Croitoru, Silvia Teslaru, Sorina Mihaela Solomon, Georgeta Liliana Foia, Laura Elisabeta Checherita | | Applied Sciences. 2022; 12(3): 1464 | | [Pubmed] | [DOI] | | | 3 |
A Study to Assess the Quality of Reporting of Animal Research Studies Published in PubMed Indexed Journals: A Retrospective, Cross-Sectional Content Analysis |
|
| Kruttika R Chitnis, Ayesha C Shah, Sharmila V Jalgaonkar | | Cureus. 2022; | | [Pubmed] | [DOI] | | | 4 |
The safety and efficacy of
Momordica charantia
L. in animal models of type 2 diabetes mellitus: A systematic review and meta-analysis
|
|
| Emanuel L. Peter, Prakash B. Nagendrappa, Anita Kaligirwa, Patrick Engeu Ogwang, Crispin Duncan Sesaazi | | Phytotherapy Research. 2021; 35(2): 637 | | [Pubmed] | [DOI] | |
|
|
|
|
|
|
|
|
