Introduction

Among the quantitative research designs, randomized clinical trials are known as the gold standard for evaluating the effectiveness of new treatments or comparison of treatment approaches, as well as the cornerstone of evidence-based medicine (1-4). Evidence-based medicine itself is the basis of decision making in clinical and health policy (5), in which research evidence are systematically analyzed to be used in the practice. Indeed, clinical trials are one of the most valuable sources to provide robust and reliable research evidence (6). Therefore, to conduct clinical trials, in addition to the scientific expertise of the researchers with respect to the subject under study and the steps involved in a carefully conducted clinical trial, the report of the method and results in an article, should also have the necessary accuracy and transparency in order to prove the validity and quality of the conducted research (2, 4). Providing a clear and detailed report for a clinical trial helps to facilitate its understanding and interpretation for health care providers (4). Whereas, the report of clinical trials with poor quality can mislead the health practitioners for effective decision-making at all clinical levels (7). A clinical trial which is considered appropriate in terms of design, implementation and reporting provides unbiased data about the outcomes for the readers (3).

Critical appraisal as the cornerstone and integral parts of evidence-based medicine is a systematic process to assess the merit, trustworthiness and relevance of clinical research in a particular context and to identify
potential threats to the validity of the research findings (8). Such evidence allows researchers and clinical staff to use research evidence in more valid and effective way (8-9). Critical appraisal of clinical evidence and then making decisions about the use of high quality and valid evidence in the clinic will enhance the quality of care and provide optimum clinical care for the patients and ultimately lead to the improvement of patients' outcomes (7-9).

In order to critically evaluate the evidence, special tools are designed for this purpose. The CONSORT checklist is one of the reliable tools for critical appraisal of clinical trials and measuring the quality of these studies (10-14). In 1993, a committee of 30 experts including editors of medical science journals, clinical trial researchers, epidemiologists and methodologists met in Ottawa, Canada aimed to set specific standards for evaluating the quality of clinical trials (2, 15). At the same time, another expert group called "The Alisomar Working Group on Recommendations for Reporting Clinical Trials in Biomedical Literature" convened in Asilomar, USA (15). Finally, with the suggestion of JAMA editor, the representatives of the two groups met in Chicago in 1996. They combined the results of their work and thus the Consort Checklist was introduced in 1996 (2, 15). The Consort checklist was revised in 2001 and 2010, and was also translated into different languages ​​(2, 3, 16). The Consort checklist provides a tool for authors, reviewers, and editors to ensure the integrity and optimal quality of clinical trials, and adherence to this tool will improve the quality of articles (2, 4).

Evaluating the compliance of clinical trials by Consort checklist is important, since it can be effective in monitoring the implementation of ongoing researches as well as providing the policies for selecting articles with the least bias for publication in journals. Finally, it leads to improve the overall quality of published reports of clinical trials (3).

While more than 580 journals have considered adherence to the Consort checklist as part of their review process of clinical trials and this checklist is freely available to all researchers, various studies which appraised clinical trial studies in different fields, have reported average or low level of compliance with the principles of the Consort checklist in reporting clinical trials (2,4,17-23). This itself emphasizes on the necessity of critical evaluation of published clinical trials in order to identify the strengths and limitations of the reports.

It is very important to select the correct and evidence-based approach in ovulation stimulation in order to maintain the health of egg donors. Egg donation has been performed since 1980s, and nearly 40 years have passed since the first live birth using donated egg (24, 25). Today, with the increase in the indications of gamete donation, the demand for donated eggs has increased (25-27). According to the data of the European Society of Human Reproduction and Embryology in 2016, nearly 74,000 assisted reproductive technology cycles with donated eggs had occurred in Europe (28, 29). The important point in egg donation is to pay attention to the fact that the egg donors are healthy and potentially fertile women who voluntarily and without medical indication are subjected to the process of ovulation stimulation (24, 25). Ovulation stimulation and egg retrieval process is associated with short and long-term physical complications, including ovarian hyper-stimulation syndrome, ovarian torsion, painful drug injections, and egg retrieval-related complications such as intra-pelvic bleeding and infection (25, 30-32). Therefore, it is very important to select the right approach in ovulation stimulation in order to achieve the best outcome for egg recipients and more importantly to maintain the safety and health of donors (25). In order to achieve the best outcome for patients, it is necessary to use the best and most reliable clinical evidence in evidence-based medicine to identify and apply the most effective and safest treatment methods (33).

As aforementioned, in order to use the evidence obtained from clinical trials, the first step is to critically appraise the evidence and ensure their optimal quality. To the best of our knowledge, no critical review has been done on clinical trials in the field of ovulation stimulation approaches in egg donors; therefore, this study was performed to critically appraise the randomized clinical trials published in relation to the effect of ovarian stimulation approaches on the outcomes of egg donation in donors using the Consort checklist 2010.

Materials and Methods

In this critical review, to access the articles, the electronic databases of Web of Science, Scopus, Cochrane, PubMed and EMBASE were searched by two researchers (EI, AY) independently, using the MESH keywords of ovulation induction, Ovarian stimulation, oocyte donation and ovum donation using AND/OR operators without time limit until June 2022. Persian databases of Magiran and SID were also searched. Then, the title and abstract of the articles were checked by the two researchers in terms of content relevance. Inclusion criteria were all randomized human clinical trials in the field of pharmaceutical approaches of ovulation stimulation on the outcomes of donation in egg donors in English or Persian. Semi-experimental studies, letters to the editor, review studies, as well as articles in languages ​​other than Persian and English were excluded. Also, having no access to the full-text of articles and non-compliance to the basic principles of conducting randomized clinical trials were considered as exclusion criteria. References of related articles were also checked manually in order to retrieve the related studies. Finally, 19 articles were critically evaluated using the Consort checklist 2010 (Figure 1).

Consort checklist has six sections with 25 topics and 37 items. The title and abstract section with two items, the introduction section with two items, the method section (including subjects of designing the trial, participants, interventions, outcomes, sample size, randomization, blinding and statistical methods) with 17 items, the results section (comprising different subjects of flow of participants, patient's selection, basic data, number of analyses, outcomes and estimations, sub-analyses and risks) with 10 items, the discussion section (including subjects of limitations, generalizability and interpretation) with three items and finally the section of other information (containing subjects of registration, protocol and financial resources) with three items are the six sections of Consort checklist 2010. Items of 3b (Important changes to methods after trial commencement with reasons), 6b (Any changes to trial outcomes after the trial commenced, with reasons) and 7b (When applicable, explanation of any interim analyses and stopping guidelines) from the method section, item 14b (Why the trial ended or was stopped) from the result section and item 24 (Where the full trial protocol can be accessed, if available) from the other information section were excluded from scoring and evaluation because they were not applicable to any of the studies which were included in the study. Finally, 32 items were scored. It should be noted that previous studies which critically evaluated clinical trials have also excluded the non-applicable items from their evaluation checklist (3, 4, 23). Each item was given a score of one if it was observed in the article and a score of zero if it was not observed. The range of scores was 0-32. The minimum score in each section was zero and the maximum score was two for title and abstract sections, two for introduction, 14 for method, nine for results, three for discussion and two for other information. According to the agreement of the research team, the quality of articles was divided into four categories: poor (less than 50% of the total score), average (50-70% of the total score), good (71-90% of the total score) and excellent (more than 90% of the total score). In this study, two authors (EI, AY) individually appraised 19 studies and shared their results at the end of each evaluation. In case of disagreement, the third author (RLR) were reviewed and appraised the articles.

It should be noted that the ethical issues in conducting a critical review of evidence, including the removal of overlapping articles, accuracy in the evaluation process, participation of all authors in the review process, clarifying the role of authors and sources of financial support of the study, avoiding bias in study selection, exclusion or interpretation of the results obtained from the retrieved articles, preserving the intellectual property by correct citation and avoiding plagiarism, as well as refraining from creating data (5), were considered by the research team.

Data were analyzed by SPSS software (version 22) using descriptive statistical methods including mean ± standard deviation and number (percentage), and the independent T-test.

Results

Following the initial search, 2104 articles were obtained from PubMed (828 articles), Cochrane (89 articles), Embase (158 articles), Scopus (810 articles) and Web of Science (219 articles). After removing duplicate articles, the title and abstract of 750 remained articles were examined by two researchers, independently. After excluding the articles that did not meet the inclusion criteria, 25 articles remained. Two articles were also obtained from the manual search of the retrieved studies references. Out of a total of 27 articles, eight articles were excluded from the study due to the lack of access to the full-text (abstract presented in conferences). Finally, 19 articles, of which 15 (79%) were from Spain (34, 35, 44-48, 36-43) and one article from each country of Vietnam (49), Turkey (50), Finland (51) and Greece (52) were critically appraised. All articles were accepted and published in reputable journals. The details of the articles, including the name of the first author, year of publication, country where the research was carried out, number of authors, publishing journal, as well as journal indexing and quartile were extracted (Table 1). A total of 18 articles (95%) were indexed in Web of Science (34-35, 44-45, 47–52, 36–43) and one article (46) was indexed in PubMed database. From the articles published in the Web of Science database, 15 articles (79%) were in the first quartile (48, 35, 34-36, 41-52, 38-47, 44) and three (16%) were in the third quartile journals (39, 40, 45). Also, 48% of the articles (eight articles) were published after 2010 (42-49) and among them, three articles (16%) (46-48) were related to the recent five years. All articles were written in English.

The scores obtained by each article in each six sections and compliance percentage of the articles with the items of the Consort 2010 checklist can be seen in Table 2 and Figure 2, respectively. The mean total score of the reviewed articles was 21.31 ± 4.76. One article obtained the full score (47). The lowest score obtained was 14 (34, 43) and the highest score was 32 (47). All articles obtained the full score in the introduction section. Based on the mean scores obtained in each section, the articles have the lowest score in the sections of other information (21.05%), results (61.9%), method (66.1%), discussion (78.9), and title and abstract (84.2%), respectively. The mean quality score of 11 articles published before 2010 (36-41, 50-51, 34, 35, 52) was 20.27±3.74. Also, eight articles were published after 2010 (42-49) and the mean quality score of these articles was 22.75±5.84. The difference between the mean scores of articles published after and before 2010 was not statistically significant (P=0.27). The quality of articles were classified as follows: two articles poor (34, 43), nine articles moderate (35-36,38–40,42,45-46,52), seven articles good (37,41,44,48–51) and one article excellent (47). In total, the appraised articles in this study are considered of moderate quality by achieving the mean percentage of 66.59 from the total score.

In the method section, the lowest score were related to trial design and sample size, respectively, with scores of 0.42 and 0.47 out of the maximum score of one. Interventions with a score of one out of a maximum score of one were reported in all the studies. In the results section, the weakest parts were ancillary analyses, and in the findings section the weakest parts were related to harms, outcomes, and estimation. In the discussion section, the articles scored less than 40% of the limitation section score. Providing information related to the trial registration and source of funding were the weakest parts in the evaluated articles (Table 3). Less than 20% of the articles adhered to reporting the items 17b (For binary outcomes, presentation of both absolute and relative effect sizes is recommended) and 25 (Sources of funding and other support (such as supply of drugs, role of funders) (Figure 2).

Discussion

The present study was conducted to critically review the randomized clinical trials published on the effect of pharmacological approaches of ovulation stimulation on the outcomes of donation in egg donors. To the best of our knowledge, this is the first study that critically reviewed the randomized clinical trials in this field. For this reason, it has been tried to use articles with the same methodology and as much as possible in the same field for the discussion around the results. Also, review studies in the field of ovulation stimulation were used. In total, the evaluated articles obtained the mean score of 21.31 ± 4.76 (66.59%) and the overall quality of the articles was considered as moderate.

In the previous studies which critically appraised randomized clinical trials to review the quality of reporting articles in different fields based on the Consort checklist, the quality of articles has been reported as different. In several critical appraisals conducted in the recent years, the overall quality of randomized clinical trials was reported as moderate or good (4, 17-22, 53). For example, Partsinevelou and Zintzaras (2009) found that the quality of the published trials in the field of polycystic ovary syndrome was lower than the desired level (54). Of course, it is important to mention that due to the fact that the Consort checklist does not have a scoring system or quality ranking for articles, the quality classification of the articles has been done by the authors. Sharifi et al. (2021) in a critical review of randomized clinical trials focused on the impact of complementary and alternative medicine on pregnancy rate of infertile women reported that the overall quality of the articles was moderate with the compliance rate of 50% with the Consort checklist and the mean obtained score of 22.68 ± 6.17 (22).

Alirezaei and Latifnejad Roudsari (2022) in their study "critical appraisal of the published clinical trials on the effect of herbal medicine on striae gravidarum" reported the overall compliance of the articles with the Consort checklist (2018) was 46% and its mean score was 20.86 ± 7.18 (21). Sarayloo and Latifnejad Roudsari (2018) in their study critically appraised the published clinical trials on the effect of complementary medicine on menopausal symptoms, and reported that the mean Consort score is 23.93 and the quality of the articles is moderate (17). Also, Abdollahpour et al. (2020) in their study critically appraised clinical trials regarding the impact of midwifery interventions on post-traumatic stress in postpartum period and reported the quality of the articles as moderate and its average score as 25.1 ± 3.6 (19). In the present study, articles with 50-70% compliance with the consort checklist were considered of moderate quality, which is consistent with the above studies.

In the present study, the parts of findings and method as well as other information had the least compliance with the Consort checklist items. This finding is contrary to the results of the study by Sharifi et al. (2021), which reported the most weaknesses in the discussion, title and abstract sections. Of course, this can be attributed to the different fields of research in the two studies. In addition, in the study by Sharifi et al., since non-pharmacological approaches were evaluated, the 2017 Consort checklist was used. This checklist is used for clinical trials with non-pharmacological interventions and has 44 items, while the Consort 2010 checklist has 37 items. Irani et al. (2017) in critical review of clinical trials on the effect of massage on the intensity of labor pain found that the method section with mean obtained score of 7.30 was one of the weakest sections of the evaluated studies (18). Alirezaei and Latifnejad Roudsari (2022) also found that the sections of stakeholders' participation and method have the lowest score. They reported mean score of 8.14 in the method section (21). In the current study, also the method section with mean obtained score of 9.26 was one of the most problematic sections of the studies. Abdollahpour et al. (2020) reported the lowest score in the discussion and other information sections; they reported mean score of 0.50 in the other information section (19). In the current study, the other information section with a score of 0.42 had the lowest compliance with the Consort checklist.

In the method section of the evaluated articles, the lowest score were related to the trial design and sample size. Salehian and Karimi (2022), Manochehri et al. (2020), Sarayloo and Latifnejad Roudsari (2018) and Irani et al. (2017) in their critical appraisal also found that the reporting of method to estimate sample size in the method section of most reviewed studies was missed (17-18, 20, 55). In some other critical studies, randomization and blinding of the study was not clarified (4, 53, 56). In the current study, the rate of compliance with the Consort checklist in randomization and blinding sections was 59% and 68%, respectively. This finding is consistent with the results of a systematic review comparing two approaches of ovulation stimulation in egg donors. Badri et al. (2011) reported that 75% of randomized clinical trials reviewed by their research team provided sufficient explanations on randomization (57). Partsinevelou and Zintzaras (2009) evaluated the quality of published trials in the field of all interventions performed on women with polycystic ovary syndrome and found that the trials conducted in the field of fertility-related issues had better accuracy and quality compared to the trials which studied other fields of polycystic ovary syndrome treatment (54). This can justify the appropriate quality of the evaluated articles in the blinding and randomization sections in our study.

According to the findings of the present study, the overall quality of the articles written after 2010 has improved compared to the articles published before 2010, although it was not significant. Perhaps the reason for this improvement in quality can be attributed to the use of Consort 2010 checklist for evaluating articles. However, it should be noted that the articles written before 2010 also had access to the older versions of the Consort checklist. Several studies, which evaluated and compared the quality of randomized clinical trials in different time periods have reported that the quality and accuracy of reporting randomized clinical trials has improved over time (53-55). This indicates the effectiveness of reporting clinical trials according to the critical appraisal tools such as Consort checklist, to improve the quality of published articles.

One of the limitations of this study is not evaluating the non-Persian or non-English databases. The authors of this article were aware of the names of the authors, the name of the journal and the publishing base of the evaluated articles. Nevertheless, it was tried to evaluate the articles without any bias towards the researchers and/or the publishing journals. Another limitation is the use of zero and one scoring system, because in this scoring method, the value and importance of all items are considered the same. Extensive search of related texts in reliable databases based on the selection process of PRISMA flowchart 2020 for conducting systematic reviews, which enables the reproducibility of the search process, as well as the critical evaluation of studies that were mostly published in international high ranked and first quartile journals was the strength of the present study. In addition, conducting the review by two researchers who independently evaluated and scored the articles, as well as the use of the senior (third) researcher in cases of disagreement between the two researchers, was other strength of this research. Since this study evaluated the articles that their population was egg donors, it is recommended that future studies critically evaluate clinical trials on ovulation stimulation approaches in infertile women.

Conclusion

The overall quality of the articles reviewed in this study was moderate. Among the main sections of the articles, the lowest score were related to the findings and method sections, respectively. Considering that the findings obtained from clinical trials are the basis for evidence-based medicine, the higher quality of the clinical trial report leads to the more reliable interpretation and application of it in the practice. Currently, although an improvement has occurred in the overall quality of reporting randomized clinical trials compared to the past, there are still weaknesses in published articles even in reputable journals. Therefore, training researchers as well as reviewers and editors about the critical appraisal tools can be effective in removing these deficiencies.

Acknowledgements

The authors express their gratitude and appreciation to the authors whose articles were reviewed in this study. This research has been approved by the ethics committee of Mashhad University of Medical Sciences with the ethics code of IR.MUMS.NURSE.REC.1401.052. The Ethics Committee of Mashhad University of Medical Sciences as well as the Nursing and Midwifery Care Research Center of Mashhad University of Medical Sciences, which provided financial support for this research, are hereby appreciated.

Conflicts of interest

Authors declared no conflicts of interest.