Do Croatian open access journals support ethical research? Content analysis of instructions to authors.

INTRODUCTION: The aim of our study was to investigate the extent to which Instructions to authors of the Croatian open access (OA) journals are addressing ethical issues. Do biomedical journals differ from the journals from other disciplines in that respect? Our hypothesis was that biomedical journals maintain much higher publication ethics standards.
MATERIALS AND METHODS: This study looked at 197 Croatian OA journals Instructions to authors to address the following groups of ethical issues: general terms; guidelines and recommendations; research approval and registration; funding and conflict of interest; peer review; redundant publications, misconduct and retraction; copyright; timeliness; authorship; and data accessibility. We further compared a subset of 159 non-biomedical journals with a subset of 38 biomedical journals. Content analysis was used to discern the ethical issues representation in the instructions to authors.
RESULTS: The groups of biomedical and non-biomedical journals were similar in terms of originality (χ2=2.183, P=0.140), peer review process (χ2=0.296, P=0.586), patent/grant statement (χ2=2.184, P=0.141), and timeliness of publication (χ2=0.369, P=0.544). We identified significant differences among categories including ethical issues typical for the field of biomedicine, like patients (χ2=47.111, P<0.001), and use of experimental animals (χ2=42.543, P<0.001). Biomedical journals also rely on international editorial guidelines formulated by relevant professional organizations heavily, compared with non-biomedical journals (χ2=42.666, P<0.001).
CONCLUSION: Low representation or absence of some key ethical issues in author guidelines calls for more attention to the structure and the content of Instructions to authors in Croatian OA journals.

Introduction

Over the last decade, scientific publishing has sky-rocketed to almost two million articles a year (). This overwhelming growth is accompanied by a strong shift toward open access (OA) publishing that has forced for-profit publishers to gradually replace the subscription model with the author-pay model in which article-processing charges (APC) are covered by authors or affiliated institutions. This, in turn, boosted the emergence of many new journals that promise quick peer-review and high manuscript acceptance for an average fee of USD 1,418 for online journals and USD 2,727 for hybrids, which publish both online and in print (). At the same time, researchers are pressured to “publish or perish” (), as their academic recognition, career advancement, and research funding depend on how much they publish and get cited. In the environment of hypertrophied scholarly publishing, it is not easy to identify low-quality or even false research. Some claim that the responsibility for maintaining public trust in research integrity lies as much with editors and journal policies as with researchers. Marušić et al. believe that the “greatest power of journal editors is their responsibility and privilege to formulate and implement editorial policies to ensure the validity, objectivity, fairness and transparency of the publishing process in science” (). While this may be true for well-established journals supported by big publishers, does this apply to editors of small journals, who usually work pro bono, alongside their research and teaching careers, and in small scientific communities that may not be so keen on pursuing research integrity and other ethical standards? The answer is, it does, but the question is how? With insufficient institutional support staff, and budget, it looks as though small journals cannot afford to deal with ethical issues the way top journals do. They more often receive manuscripts which are plagiarised or hide some other type of misconduct, because their authors are reluctant to take their chances with more prominent journals.

Publication ethics is indispensable for all scientific disciplines, but the majority of articles on this topic have been published in biomedical journals (–). Medical information is sensitive and sloppy research and publication, as well as research fraud, may lead to loss of human life. Because “biomedical journals […] are supposed to be more than purely scientific publications: they are also medical journals and as such their articles potentially influence medical practice and are likely to contribute to the improvement of public health“ (), the most ethical guidelines are designed for the field of biomedicine. In its nearly 30-year history International Committee of Medical Journal Editors (ICMJE) issued different versions of its guidelines for the submission of manuscripts, establishing ground for journal editors and their associations, publishers, and other major players in the scholarly publishing world to create numerous standards, guidelines and recommendations on different aspects of scholarly communication. Adopting the best practices, policies, recommendations, codes, and guidelines which share the responsibility for research integrity between authors, editors, and publishers, journals can respond to potentially low-quality submissions.

Guidelines on publication ethics or, more often, instructions to authors have an important role in “setting the rules”. Instructions to authors should mirror editorial policies, including the ones on publication ethics. Furthermore, journals and editors should be educating their communities about ethical issues in publishing. Several studies have assessed instructions to authors of mostly biomedical journals in terms of reporting ethical issues (–, –). As far as we know, editorial policies of Croatian biomedical or other journals have not been analysed in terms of publication ethics. An exception is the recent study by Broga et al. which compared the publication ethics policies of biomedical journals published in Central and Eastern Europe, which did include a few Croatian biomedical journals ().

The aim of our study was to investigate Croatian Open Access (OA) journals from all disciplines in terms of instructions given to authors that address ethical issues. We also wanted to see how biomedical journals differ from the journals from other disciplines in that respect. Our hypothesis was that biomedical journals maintain much higher publication ethics standards.

Common ethical issues addressed by guidelines and recommendations

Scholarly publishing is rather complex process and authors, reviewers, editors and publishers have different roles and responsibilities at different stages. According position statements developed at 2nd World Conference on Research Integrity discussed by Wager and Kleinert (), authors should adhere to publication requirements that submitted work is original and has not been published elsewhere in any language. Copyright laws and conventions should be observed; previous work and publications should be properly acknowledged and referenced; data, text, figures or ideas originated by other researchers should be properly acknowledged; and authors should respond to reviewers’ comments in a professional and timely manner. To carry out responsibility to their readers, editors are responsible for ensuring the accuracy of the material they publish (). Reviewer role is crucial and reviewer is expected to be prompt and systematic in review, protect confidentiality of information and ideas obtained from the manuscript, ensure objective and unbiased assessment, and disclose possible conflict of interest. Publisher must be committed to ensuring that commercial income has no influence on editorial decisions.

Conflicts of interest (CoI) can inappropriately influence design, conduct, or reporting of research and are usually connected with funding. According to Pitak-Arnnop et al., financial ties or personal interests can threaten scientific integrity through biased study design and conclusions/interpretations that favour certain industry or ignore unfavourable findings. Ultimately, this can undermine patient safety and public trust in a biomedical journal (). To avoid this, CoI statement is intended to provide readers with the necessary information to make their own judgment on potential bias.

Redundant publications are those that add little new information to the work of the same author already published (). Duplicate publication is a subset of redundant publication involving the reproduction of data with nothing new contributed to the literature and is often regarded as self-plagiarism or recycling. During indexing process for example, National Library of Medicine (NLM) identifies articles with one or more authors in common that substantially duplicates other articles without acknowledgement. Submitting the same manuscript to several journals is also regarded as redundancy or dual, concurrent, simultaneous or multiple submissions.

During the last 30 years the number of retracted articles rose from 1 or 2 retractions per year to around 500 per year (), and instructions to authors should address the issue of possible retraction. Authors must know why and how their work might be corrected, withdrawn, or retracted. Committee on Publication Ethics (COPE) has defined a retraction as “a mechanism for correcting the literature and alerting readers to publications that contain such seriously flawed or erroneous data that their findings and conclusions cannot be relied upon”. Retractions are also used to alert readers to cases of redundant publication, plagiarism, and failure to disclose a major competing interest likely to influence interpretations or recommendations (). It is generally believed that retractions serve to maintain the integrity of scholarly publications, and, when properly enforced, to avert scientists from bending the rules of scientific conduct and publication ().

One of the most important journal selection criteria for the author is the likelihood of timely publication. Despite technological advancements, journals still have a problem defining the period between submission and publication. There are two main periods to be defined: time from initial submission to final acceptance, and time from final acceptance to actual publication, and it is very important to share this kind of information with potential authors.

Many international journals have addressed the attribution of authorship by requesting authors to describe individual contributions to the manuscript. According to ICMJE, authorship credit should be based on four criteria (), and authors should meet all four conditions. Determining the authorship can be difficult, but listed individuals’ contributions tells the reader who takes the credit and blame for the work (). It can also prevent authorship abuse, like the appearance of guest, gift and ghost authors ().

Raw data publishing has become increasingly important and has already been incorporated in the policies of the world’s leading research funding frameworks and organisations. Access to raw data is also welcome to improve efficacy and transparency of the peer review process, which is at the present time-consuming, biased, inconsistent, conservative, and open to abuse (). Access to the raw data allows peer reviewers to validate the findings, discussions, and conclusions.

Materials and methods

Study design

To investigate Croatian OA journals in terms of ethical issues we identified 228 journals on the Croatian repository of Open Access journals HRČAK (http://hrcak.srce.hr) which had an English version of the Instructions to authors. Thirty-one were excluded because the files were not machine readable (PDF stored as image or odd encodings), empty, or contained only a link to the journal web page. Among final 197 instructions, we also identified 38 Croatian biomedical journals according to the discipline coverage declared by their editorials. The list of all 197 journals is available as supplementary material (Supplement 1) in electronic publication.

Next, we analysed the content of each .pdf or .doc document that contained instructions to authors using the QDA Miner and WordStat software licensed by author (by Provalis Research) for text analysis. The pilot stage lasted from May to June 2014, and the final sample was collected between 15 and 17 September 2014. A non-validated categorisation scheme was developed to code text from the instructions by grouping words, phrases, and rules describing different categories of publication ethics. These categories were defined according to the published literature, the results of the preliminary testing, and content analysis of the most influential publication ethics guidelines. The text was coded automatically according to categories and subcategories, and assigned words, phrases, and rules stored in the categorisation dictionary. The dictionary consisted of 10 categories, 23 subcategories, and 146 words, phrases, and rules used for coding. Provided syntax was used to register all possible appearances of words and phrases. Table 1 shows the simplified version of the terms used for coding, and .XML and .txt versions of the whole categorisation dictionary are provided as supplementary material (Supplement 2) in electronic publication.

Table 1

Ethical issues used for the content analysis organized in categories.

Category	Subcategory	Terms used for coding
0. General
	Publication ethics	ethics
	Research integrity	integrity
	Responsibilities of editor, reviewer, publisher, author	responsibility/duty of publisher, responsibility/duty of editor, responsibility/duty of author
	Accuracy	accuracy
	Confidentiality	confidentiality, privacy
	Originality of submitted work	original, unpublished, not previously published
1. Guidelines, codes, recommendations
	General	best practice, code of, ethical guideline, policy, principle, protocol, recommendation, regulation
	Editorial guidelines	Committee on Publication Ethics (COPE), Declaration of Helsinki (DoH), European Association of Science Editors (EASE), International Committee of biomedical journal Editors (ICMJE), etc.*
	Reporting guidelines	Animal Research: Reporting of In Vivo Experiments (ARRIVE), Consolidated criteria for reporting qualitative research (COREQ), etc.**
2. Research approval and registration
	Research approval	approval, assent, committee
	Registration of clinical trials	trial registration
	Patient	patient, human
	Informed consent	informed consent
	Patient confidentiality	confidentiality (patient, human)
	Use of experimental animals	animal
3. Funding and conflict of interest
	Research funding	funding, financing
	Project/Grant	grant
	Sponsorship	sponsorship
	Conflict of Interest	conflict of interest, COI
4. Peer Review
	/	assessment, blinded, evaluation, external expert, final decision, non-blinded, peer review, reviewer, referee
5. Redundant publications, misconduct, retraction
	Redundant publications	compilation, concurrent, duplicate, multiple, recycled, redundant, simultaneous submission
	Misconduct	allegation, fabrication, falsification, fundamental error, fraud, malpractice, manipulation, misconduct, plagiarism
	Retraction	erratum, expression of concern, retraction, suspicion, withdraw
6. Copyright
	/	copyright, Creative Commons, exclusive licence, intellectual property, infringement, rights transfer
7. Timeliness
	/	publishing within period of time (e.g. number of months, weeks), reasonable speed, timelines
Category	Subcategory	Terms used for coding
8. Authorship
	/	authorship, contributorship, criteria (author, contributor), honorary authorship, ghost author, gift author
9. Data accessibility
	/	data accessibility, data management, dataset, experimental data, raw data, research data, underlying data
* Editorial guidelines included in the categorisation dictionary: Committee for the Purpose of Control And Supervision of Experiments On Animals (CPCSEA), Committee on Publication Ethics (COPE), Council Directive, Council of Science Editors (CSE), Declaration of Helsinki (DOH), European Association of Science Editors (EASE), Food and Drug Administration (FDA), Guide for the Care And Use of Laboratory Animals, Guidelines for the Use of Animals In Research (IASP), International Committee of biomedical journal Editors (ICMJE), „Uniform Requirements“ (ICMJE), Meta-analysis Of Observational Studies in Epidemiology (MOOSE), Office of Research Integrity (ORI), Reporting guidElines For randomized controLled trials for livEstoCk and food safeTy (REFLECT Statement), U.S. Federal Policy for the Protection of Human Subjects (HHS), World Association of Medical Editors (WAME).** Reporting guidelines included in the categorisation dictionary: Animal Research: Reporting of In Vivo Experiments (ARRIVE), Consolidated Standards of Reporting Trials (CONSORT), Consolidated criteria for reporting qualitative research (COREQ), Enhancing the QUAlity and Transparency Of health Research (EQUATOR), Minimum Information About a Microarray Experiment (MIAME), Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA), Standards for Quality Improvement Reporting Excellence (SQUIRE), STAndards for the Reporting of Diagnostic accuracy studies (STARD), STrengthening the Reporting of OBservational studies in Epidemiology (STROBE).

Statistical analysis

Results were expressed as frequencies and percentages for categorical variables or mean ± standard deviation for continuous variables. Associations between discipline and categorical parameters were tested using χ2-test. Fisher’s test was used for frequencies < 20. Level of significance was set at 0.05. Statistical analysis was done using statistical software WordStat (Provalis Research, Montreal, Quebec, Canada).

Results

The groups of biomedical and non-biomedical journals were similar in terms of originality (χ2 = 2.183, P = 0.140), peer review process (χ2 = 0.296, P = 0.586), project/grant statement (χ2 = 2.184, P = 0.141, F-test), timeliness of publication (χ2 = 0.369, P = 0.544, F-test), and misconduct (χ2 = 4.057, P = 0.045, F-test) (Table 2). As expected, we identified significant differences among categories including ethical issues typical for the field of biomedicine, like patients (χ2 = 47.111, P < 0.001, F-test), and use of experimental animals (χ2 = 42.543, P < 0.001, F-test). International editorial guidelines formulated by relevant professional organizations are represented in biomedical journals in significantly greater extent (χ2 = 42.666, P < 0.001, F-test), although these guidelines can be easily applied also to other disciplines. Some categories relevant to all disciplines are significantly more represented by biomedical journals, like publication ethics (χ2 = 21.311, P < 0.001), accuracy (χ2 = 13.828, P < 0.001), research funding (χ2 = 14.492, P < 0.001, F-test), copyright (χ2 = 11.383, P = 0.001), and CoI (χ2 = 33.065, P < 0.001, F-test).

Table 2

Rate of reference to ethical issues by category in instructions to authors of Croatian OA journals.

Category	Subcategory	All journals		Biomedical journals	Non-biomedical journals	Chi square value	P (Chi square test)
		N = 197	%	N = 38	N = 159
0. General		176	89%	37	139	3.186	0.074
	Originality of submitted work	166	84%	35	131	2.183	0.140
	Responsibilities of editor, reviewer, publisher, author	75	38%	23	52	10.069	0.002
	Accuracy	40	20%	16	24	13.828	< 0.001
	Publication ethics	40	20%	18	22	21.311	< 0.001
	Reporting	19	10%	11	8	22.194	< 0.001*
	Confidentiality	12	6%	6	6	7.976	0.005*
	Research integrity	7	4%	4	3	6.844	0.010*
1. Guidelines, Codes, Recommendations		82	42%	26	56	13.913	< 0.001
	General	76	39%	23	53	9.571	0.002
	Editorial guidelines	37	19%	20	17	42.666	< 0.001*
	Reporting guidelines	4	2%	3	1	8.404	0.004*
2. Research approval and registration		42	21%	21	21	32.338	< 0.001
	Patients	27	14%	17	10	47.111	< 0.001*
	Use of experimental animals	23	12%	15	8	42.543	< 0.001*
	Research Approval	16	8%	8	8	11.033	0.001*
	Informed consent	9	5%	8	1	34.130	< 0.001*
	Registration of clinical trials	3	2%	3	0	13.490	< 0.001*
	Patient confidentiality	3	2%	2	1	4.447	0.036*
3. Funding and Conflict of Interest		42	21%	14	28	6.932	0.009*
	Research funding	33	17%	14	19	14.492	< 0.001*
	Conflict of Interest	18	9%	12	6	33.065	< 0.001*
	Project/Grant	11	6%	4	7	2.184	0.141*
	Sponsorship	11	6%	6	5	9.664	0.002*
4. Peer Review		138	70%	28	110	0.296	0.586
5. Redundant publications, Misconduct, Retraction		35	18%	11	24	4.071	0.045*
	Misconduct	23	12%	8	15	4.057	0.045*
	Retraction	20	10%	8	12	6.266	0.013*
	Redundant publications	11	6%	5	6	5.207	0.024*
6. Copyright		68	35%	22	46	11.383	0.001
7. Timeliness		30	15%	7	23	0.369	0.544*
8. Authorship		28	14%	11	17	8.667	0.004*
9. Data accessibility		18	9%	9	9	11.143	0.001*
*Fisher exact test.

Among 197 Croatian OA journals the most frequent issues addressed publication ethics were the originality of the submitted work and peer review process. The term “original” and related variations were mentioned 874 times by 166 instructions to authors, describing mostly submitted work and including phrases such as “original scientific paper”, “original article”, “original contribution”, “original research paper”. In some documents the term original is related to images (“original graphics”) or research process (“results of original research”, “original laboratory techniques”).

From all generic terms related to guidelines and recommendations the most frequent were recommendation (31/197 journals), policy (28/197 journals), regulation (27/197 journals), principle (22/197 journals), and standard (19/197 journals). The most addressed publication ethics recommendations were those by COPE, referred to by 14/197 journals. ICMJE was mentioned by 10/197 journals, and its old version of the Uniform Requirements for Manuscripts Submitted to Biomedical Journals by 12/197 journals. Only one journal referred to the ICMJE’s most recent version, Recommendations for the Conduct, Reporting, Editing and Publication of Scholarly Work (). The Declaration of Helsinki () was referred to by 13/197 journals. One veterinary journal referred to the outdated version of European Community Council Directive on the approximation of laws, regulations and administrative provisions of Member States regarding the protection of animals used for experimental and other scientific purposes, even an updated version is available ().

Misconduct issues, including fabrication, falsification, and manipulation of data as well as plagiarism, were addressed by 23/197 journals, of which 8 were biomedical, and “plagiarism” was the most common term used by 18/197 journals. Retraction policy was addressed by 20/197 journals, and manuscript withdrawal by 11/197 journals, while article retraction was referred to by 9/197 journals.

The likelihood of timely publication was addressed by 31/197 journals, 23 of which spoke in terms of months and 6 in terms of weeks.

If we look at the total set of 197 instructions to authors, the average number of ethical issues addressed per journal was 7.6 ± 8.5, showing great dispersion of values (0 to 55). The median of registered ethical issues was 5. Ranked by the number of addressed ethical issues in parenthesis, the first ten journals are Medicina Fluminensis (55), Croatian Medical Journal (46), Transactions of Maritime Sciences (45), Geoadria (43), JAHR - European Journal of Bioethics (41), Biochemia Medica (37), Acta Pharmaceutica (), Nursing Journal (), Medicinski vjesnik (), and Croatica Chemica Acta (). From ten “top ranked” journals, eight are from the field of biomedicine, according the discipline assigned by journal editors.

Discussion

This study shows that Croatian OA journal editors are committed to receive original articles. To obtain originality, editors are relying strongly on peer review which is addressed by 70% of all journals, which makes our findings comparable with eastern EU journals (75% reference rate) (). At the same time research data accessibility, which can help reviewers in assessment of authors’ analyses, findings, and interpretation, is addressed only by few journals. From nine journals referring to the research data one of them is explicit in this respect: “Authors should retain raw data related to their submitted paper, and must provide it for editorial review, upon request of the editor”. Also the most editors do not specify how they intend to prevent, detect and process misconduct or redundant publications. We expected the key ethical issues from the group Redundant publications, Misconduct, Retraction to be addressed to a much greater extent, especially by biomedical journals (). Among all the terms and phrases related to redundant publications, misconduct, and retraction, “plagiarism” was most frequently addressed.

Authors’ responsibilities are often described (72/197), and they are fully responsible for the entire content of the manuscript, the accuracy of references, proof corrections, permissions to reproduce illustrations and tables from other sources, and the quality of language. The role and responsibilities of the editors (17/197), and publishers (3/197) are much less discussed. The Council of Science Editors stated that “editors of scientific journals have responsibilities toward the authors who provide the content of the journals, the peer reviewers who comment on the suitability of manuscripts for publication, the journal’s readers and the scientific community, the owners/publishers of the journals, and the public as a whole” (). Some Croatian OA journals assume the responsibility toward authors to keep disclosed conflicts of interest confidential but rare are those that profess to keep manuscript information confidential until it is published.

Ethical issues grouped in the category Research approval and registration were mostly related to the field of biomedicine. Knowing that all clinical research or animal studies must be approved by an ethics committee, we expected that Croatian biomedical journals would much more often use terms research approval, clinical trial registration or patient confidentiality, compared with journals from other disciplines. Instead, only a small number does address these issues, and some issues from this category remain completely ignored. For comparison, reference to clinical trial registration by Croatian OA journals (3/38 biomedical journals) is lower than that reported for European surgical journals (7/38) ().

In terms of CoI disclosure, Croatian biomedical journals fared worse (32%) than biomedical journals from other studies (89% of biomedical journals, 73-100% of surgical journals, 78% of paediatric journals) (, , ). Since CoI is an emerging ethical issue related to all disciplines, we expected this percentage to be significantly higher. It is possible, however, that some Croatian OA journals require that CoI is disclosed at an early stage of manuscript processing by different submission forms.

Authorship was addressed by 15% of Croatian OA journals, which is similar with the study of 20 SciELO journals (). Defining authorship is a relatively new ethical standard still not adopted by a large number of journals from countries belonging to the so called “scientific periphery” ().

Our study has several limitations related to the text analysis method we used and automatic coding approach. Although the categorization dictionary used for coding was comprehensive, it was not validated and some terms and phrases could be omitted. On the other hand, automatic coding is limited to the analysis of computer readable tokens, and could failed to exclude small percentage of the irrelevant content and it became part of the statistical data. Further, some of the hindrances were characters used in the documents that could not be recognised by the software.

The results of this study of the ethical issues included in the instruction to authors suggest that Croatian OA journals’ editors have yet not sufficiently responded to the global concerns about research integrity and misconduct, and emerging ethical issues are not well addressed by instructions to authors. As we expected, biomedical journals have performed significantly better in almost all elements of this study, and most of them are aware of the well known international guidelines. Guidelines and recommendations can help to improve publication practice, but they give editors also much of the responsibility for monitoring all possible threats to research integrity. Other reason why editors are restrained to change existing publishing models is related to the limited subsidizes which are distributed across journals according partially outdated criteria. Limited submission, mostly traditional researchers as potential authors, and rapid pace of changes in the scholarly communication area, could be additional reasons for the reluctance in the adoption of new trends related to publication ethics.

Our results show that current international recommendations aiming to improve publication practice have not yet been implemented widely in Croatian OA journals, even among the subset of biomedical journals. Adherence to the editorial and reporting guidelines has a potential to improve the quality of biomedical journals, and many elements could be applied also to other disciplines. Adoption of ethical issues like confidentiality, research integrity, misconduct, authorship, data accessibility, and CoI represent solutions for publication bias. Open access to the majority of Croatian scholarly journals is a great advantage and can enhance transparency of the whole research and publishing process and to promote science in Croatia globally.

For future research, a broad study should include also instructions to authors in Croatian language. To take the most advantage of the content analysis method, additional coding in Croatian language will enable much larger set of documents to be analysed, and give us more accurate insight about journals’ awareness of ethical issues. We also plan an expanded study that will include journal documents other than instructions to authors, which are provided at journals’ web site.