Evaluation of periodicals journals and community of medical sciences in iran.

BACKGROUND: Medical journals are scientific resources where utilization of knowledge is availed and create an environment of competitiveness. To speed up the growth and increase scientific production and in order that the rank of Iranian journals be defined regionally and globally, journal standardization is necessary. This study was done to evaluate the country's medical journals from 2004-2006.
METHODS: This is a descriptive/analytical study. Evaluation was based on the following; scientific credibility, registry and year of service, journal management, technical quality and accessibility. The number of journals evaluated during 2004-2006 were 86, 103 and 93, respectively and the process involved 3 phases 1) defining the journal's structural indicators and designing the evaluation form 2) collection, sharing and final confirmation of information with the publication staff 3) data entry, analysis and ranking
RESULTS: Improvement and growth of publication depends largely on 5 parameters and its development were based on the following; publication management (61.62 points), scientific credibility (43.80 points), quality of accessing the journals (37.05 points), quality of publication (14.80 points) and registry and year of service (0.02 points). Based on the study, an upward developmental trend of the country's medical journals can be seen and the tools were sufficient in terms of validity and reliability. A revised and more comprehensive checklist that would evaluate all aspects of a publication basing on latest indicators is developed.
CONCLUSION: Evaluations of the country's medical journals not only promote compliance to international standards but also led to more indexing of journals in accredited international indices.

Introduction

The growing range of sciences and the enormous growth of information produced in minutes and the speed in the development of the internet and network technologies, research needs, financial constraints and the need for specialized scientific groups; these factors have forced us to integrate in the immense world of information. And for the reason that countries’ develop, produce and circulate information with speed, scientific publications play an important role in the national development, in policy making and in applied research as well (1). Taking into consideration the important and valuable role of journals in producing and propagating knowledge, perhaps the most important reason for the lack of simultaneous change in the journals process of publication despite changes in information and communication technology, is the absence of a unified information system in publication. In fact, integration of information at the nationwide level is the only possible way of creating a collective agreement especially in strategies concerning changes in the process of publication printing towards electronic publishing.

What has been an issue of concern is the fact that there exist a wide gap between the developed and the developing countries in terms of biomedical publications and the possible reasons behind are; only few health researches are undertaken in the low-income countries due to lesser research priorities or lesser resources or limitation in research capacities and or problems regarding communication specially in reporting research works and the lack of observance on international standards (2).

Of the more than 130 titles approved by the medical commission, unfortunately these researches did not adhere to international standards and indicators, the difference in the choice of publication processes that would definitely result to indifference (some of these indifference is inevitable) and most importantly, the important role that the journals play in the production and dissemination of knowledge, and based on the 4th development program as undertaken aiming at achieving the first place in the Southwest Asian region in terms of economics, science and knowledge production, the need for continuous evaluation basing on local standards has been considered a necessity.

Investments in evaluation is currently on the rise and the reason behind is the belief that knowledge can be created and this in turn will enable to promote the quality and quantity of products and services provided. Evaluation of medical journal is one of the most important stages of program planning for development and promotion of knowledge in- national and international level and a useful basis for increasing the country’s share in knowledge production at a worldwide level.

Robo et al. in a study with the cooperation of CLUJ and the Pharmacist Scientific Council with the aim of providing a list of ranking on all current biomedical journals of Romania have conducted an evaluation for the purpose of promoting, improving and also awareness of the medical scientific communities and the role of medical libraries. In this study, 65 biomedical journals were evaluated based on the following criteria; peer review that includes international database, publication time interval, language used in writing the article and the abstract, specific index and the effects of the impact factor (3).

In his study in 2003 entitled Quantitative Parameters for Evaluating Scientific Journals, Petroeiyano has stressed that, “evaluating scientific journals is a hard struggle and the necessity for evaluation is for the purpose of regulating and changing every program” (4).

Also, the ISI institute has set up an indexing mechanism in order to promote evaluation of scientific articles and basing on this index, articles that received more than a number of citations in other articles and received more importance will be credited with a very high impact factor. Different methods of evaluation have been proposed but the 2 most important are; impact factor and citation. This index is an effort used to uphold the quality and reputation of scientific journals in all fields of medical specialties. The ISI Institute publishes an annual list of ISI journals sorted out based on the field of specialty (5). Bujoko, during his study in 2007 focused his attention on evaluation factors and the exchange of scientific journals with the purpose of analyzing the expanding effects of journals’ high impacts in promoting global knowledge. In this study, it was reported that journals were used as references (6). Also, in the study done on Chinese journals indexed in Medline in Chyton China Hospital, showed that the quality of RCT articles in the journals is low, therefore, Kansert’s statement must be extensively used (7).

A study conducted in 2005 showed that Iran placed 48th in producing global knowledge in the first six weeks of that same year (8). Also, results of an analysis done by IRAN DOC, the status of scientific production in Iran in the years 1999–2004 in comparison to the world ratio is 13,338 or 0.2% (9).

Another study done by the Development and Coordination of Medical Informatics Department, Undersecretary for Research and Technology, Ministry of Health, Iran, regarding Iran’s status in scientific production in the years 1996–2006 showed that the number of articles published in Pub Med is on an upward trend (10).The numbers of journals chosen to be indexed in the ISI database based on a report on 2006 were about 9600 and from this number, 5 journals were from Iran (11).

So far, from out of the 187 medical journals published in the universities, research centers and the specialist councils that were sent to the Iranian National Publication Commission of Scientific Journals, 130 titles of these journals have obtained scientific research ranking.

To speed up the growth rate of the country’s scientific production and to increase in the share of scientific production based on articles published in foreign journals and in order that the rank of Iranian journals be defined at the regional and global levels, it is necessary that the country’s scientific research journals be standardized in order that they be indexed and included in the global list of ISI.

Therefore, this study was implemented with the aim of evaluating the country’s medical journals from the years 2004–2006.

Materials and Methods

This was a descriptive/ analytical study and the subject of study included all approved medical scientific research journals of the country during the years 2004–2006 in the Iranian National Commission of Scientific Journals (INCMS).

For the reason that all journals under the study are under the approval of the INCMS, therefore, sampling was not necessary. The process of journal evaluation within the time frame of 2004–2006 were executed in this manner; 86, 103 and 93 titles of approved scientific research journals respectively. It is important to note that journals that publish complete issues per year were considered a priority and in cases that a journal publishes only 1 or more issues per year, evaluation process was based on the number of issue published. Data were gathered with the use of an evaluation form. In order that the evaluation form be developed to be a powerful tool in assessing the situation of the country’s medical journals, necessary information were initially gathered and after scheduling several meetings, these information were classified and evaluated. Several aspects of the form were discussed and exchange of views took place during the conference held for the journals’ chief editors and as a result, 5 parameters were deliberated which includes; scientific credibility, registry and year of service, publication management, quality of the publication and the quality of accessibility. Again, on several consecutive sessions, the form was reviewed and after analysis the evaluation form was finalized and was first used in 2004 during the evaluation of the country’s medical journals. The maximum number of points each year an evaluation take place, are shown in the following manner, in 2004-1130, 2005 and 2006-1175. Considering the increasing growth of new technologies, the score designated to the quality of accessibility in the evaluation form in 2004 was reconsidered and the points allotted for this item was changed from 180 to 225.

In the process of performing the evaluation, initially, the experts in the evaluation committee assessed the total number of issues published per year by using the evaluation form, and the results were entered into the software provided by the publication commission. Annual evaluation results for each journal were submitted to the INCMS in either an electronic and or printed form. Each publication, after evaluating their own journal, the designated officer sends their results to the regional office (INCMS) and on the other hand, the national commission experts compares these results with their own and after getting a unanimous result, the report will be signed and approved by the chief editor concerned then send to the INCMS for final submission. In case of any objections to the results, the person concerned must file a complaint to the INCMS. Once the complaint is verified, the commission experts will perform a re-evaluation and any changes in the score will be entered into the commission’s database software.

Lastly, the scores obtained from the evaluation were arranged accordingly from the highest to lowest and basing on the total score, we have succeeded in the evaluating the country’s medical journals. Results obtained from this evaluation process were analyzed and are presented in this paper in the form of tables and graphs.

Results

By observing the graph shown below, an elevation and growth of journals based on the 5 parameters can be observed and these are listed in accordance to the highest up to the lowest scores obtained; journal management (61.62 points), scientific validity (43.80 points) quality of accessibility to the journal (37.05) quality of printing (14.80) and registry and year of service (0.02) respectively.

Table 1 shows the total journals in accordance to their corresponding points basing on “scientific validity”, “registry and antiquity”, “journal management”, “quality of printing” and “quality of accessibility”, in the three successive years 2004–2006.

Table 1:

Publication based on the mean score “scientific validity”

	Mean
	Indicators	2004	2005	2006	Total
	Acceptance in international index	51.16	62.48	73.13	62.55
	Percentage of foreign editorial board members	4.86	5.82	6.26	5.67
	Percentage of authentic genuine articles	84.38	88.01	91.80	88.14
	Percentage of articles published outside the institute	74.57	84.48	85.02	81.66
	Total number of articles published every year	37.73	38.41	39.24	38.48
Total mean score		50.62	55.74	59.10	55.29

Results obtained showed that the parameter, scientific validity, in all indicators have increased during the last 3 yr. The highest indicator that showed the highest mean score in this parameter is the acceptance to international index. This point showed that at present, most scientific research journals have reached a point towards being indexed internationally. Being successful to be indexed in international database have shown that the process of evaluating approved journals have resulted to an improvement in their quality.

Results obtained shown in Table 2, showed that the parameter, registry and years of service of the publication during the consecutive 3 yr had lack continuous growth. This lack of continuous growth had been attributed to the indicator, timely publication. Some of the publications due to several reasons such as; budget, long review processes, management changes and etc., were the factors that contributed to untimely publication of journals. In this parameter, the indicator registry and years of journal service had also declined due to lesser contribution on the part of the publishers due to the reasons mentioned earlier.

Table 2:

Publication based on the mean score “registry and years of service”

	Mean
	Indicators	2004	2005	2006	Total
	Publication on time	18.50	18.67	18.48	18.56
	Volume of publication	14.71	14.81	14.89	14.81
	Years of publication (continuity and sequence are the criteria employed)	7.37	7.50	7.44	7.44
Total mean score		13.53	13.66	13.61	13.60

Results obtained shown in Table 3, showed that the parameter, publication management during the 3 consecutive years had a continuous growth but all indicators under the sub-category of publication management have showed only a small growth. The highest mean score in this parameter is, the time received to printing of articles and this result indicated the speed in the process of evaluating the publications in the universities publications departments.

Table 3:

Publication based on the mean score “publication management”

Parameters	Mean
Publication Management	Indicators	2004	2005	2006	Total
	Page Lay-out	8.44	8.77	8.95	8.73
	Editorial Board	3.37	4.57	4.59	4.21
	Writer’s Guidelines	2.73	3.33	3.50	3.21
	Article Presentation	5.19	5.87	6.06	5.73
	Time received to the printing of the article	18.59	36.06	44.40	33.50
Total mean score		4.45	5.62	5.97	5.38

Results obtained shown in Table 4, showed that the parameter, technical quality, together with its total indicators have increased during the 3 consecutive years. Basing on the table above, it can be observed that among the indicators mentioned, tables and graphs had the highest points and cause a major reason for this parameter to attain an increase mean. Results obtained shown in Table 5, showed that the parameter,” quality of access” had increase in the 3 consecutive years but, the growth did not include all parameters. The parameter that had the highest score is the search for full texts, this showed that website publication must be created and promoted by the publishing departments.

Table 4:

Publication based on the mean score “technical quality”

	Mean
	Indicators	2004	2005	2006	Total
	Paper Quality	4.28	5.00	5.00	4.78
	Quality of cover and binding	5.91	7.92	8.00	7.34
	Quality of printing	12.90	14.99	15.00	14.36
	Graphs and tables	23.31	29.72	29.89	27.84
	Pictures	5.52	7.00	7.00	6.55
	Design quality-appropriate page lay-outDesign quality- integrated design	50.62	55.74	59.10	55.29
Total Mean Score		8.17	9.95	9.98	9.42

Table 5:

Publication based on the mean score “Quality of Accessibility”

	Mean
	Indicators	2004	2005	2006	Total
	Web address in the print copy	3.17	3.95	4.62	3.94
	Home page in English or Persian	14.13	15.67	15.75	15.23
	Home page language- presence of a second language	11.16	17.52	14.68	14.67
	Publication web content	25.00	12.14	11.45	15.81
	Search facilities	7.56	19.67	18.98	15.77
	Free full text search	6.86	19.05	20.16	15.72
	Capability of the publication web site for Peer- review	1.16	7.14	10.22	6.34
	Capability of the publication web site for sending E-mail alerts	0.81	5.00	6.08	4.08
Total Mean Score			8.17	9.95	9.98

Results obtained shown in Table 6, showed an increase in the number of Iranian journals indexed in major indexing organizations.

Table 6:

Comparing the number of published Journal indexed in ISI, Pubmed, and Scopus in selected countries (12–14)

	ISI		Pubmed		Scopus
	No.	Per.	No.	Per.	No.	Per.
Germany	463	2.81	283	5.18	2176	7.79
Japan	175	1.06	162	2.96	679	2.43
China	81	0.49	102	1.86	680	2.44
Korea	40	0.24	18	0.32	133	0.47
Turkey	8	0.04	17	0.31	151	0.54
Iran	9	0.05	6	0.11	54	0.19

Discussion

Results of the study showed that based on the 5 parameters mentioned, there has been significant growth and promotion of approved scientific research publications observed during the 3 consecutive years (2004–2006). The growth of these journals with the corresponding fields are shown in this manner; in the field of publication management (61.62 points), scientific validity (43.80 points), quality of accessibility (37.05 points), print quality (14.80 points), registry and years of service (0.02). Also, the growth rate of publication based on the above parameters was more specific in 2005 in comparison to 2006.

In the study conducted by Robu et al. entitled Improving Standards in the Biomedical Scientific Community through Improvement in the Quality of Journals, 65 biomedical journals based on the following criteria were assessed; peer review including international database, time interval of publication, article’s and abstract’s language, specific indexes and the index factor. In this study, the necessity of evaluating journals was aimed at promoting and improving publications and also to provide information to the medical scientific community and to give attention to the role of medical libraries (3).

In another study conducted by Petroianu entitled Quantitative Parameters to Evaluate the Publication of Scientific Papers in 2003, an evaluation guideline for publishing scientific resources was presented. In this guideline, evaluation on the value of each journal was based on multiplication of 3 factors; the first factor is the value of the author corresponding to his place among the authors in the in the authorship of the work. The second factor is the classification of the type of publication and the third factor is the sum of the 2 factors multiplied by 10 times the “impact factor”. This method has been used since 1994, with excellent results and without confronting any judgmental problems. The criterion prevents subjective interference and, conflicts of interest are avoided when clearly presented from the beginning of the judgment process (4).

A study conducted in Romania regarding scientific the journals showed that, publications that publishes research results in internationally recognized journals in order to provide information regarding new researches, is considered a vital importance. Evaluation results intended to help the policy makers in decision making is of outmost importance. Criteria used in evaluating the journals include; indexed in international journal, listed in abstracts on specialty, international impact factor, journal publication outside Romania, international publication, national and international scientific council’s support, editorial board credentials, years of service, journal’s appearance, goals, publication of articles in foreign languages (English, French or German), publication of abstract in English, and the quality of scientific board advisors. Also in this study, criteria for evaluation of publications were also considered including; significant activities in publishing scientific works and academic books, scientific publication of at least 50 scientific works or academic books in the last 5 yr, received commendation to edit scientific works in the masteral and doctoral level, process of publishing national and international scientific works, quality and credentials of the advisory boards, number of series and special collections, international subscribers, and periodic cycle of international publications (5).

In 2007, Bojko during his study on evaluation factors and scientific publications cooperation had stressed the importance of the impact factor in the scientific development of a journal globally and for this purpose, the journal’s citation was used (6).

In another study to rank scientific and professional journals, ergonomics and human factors were used. A method for selecting and ranking scientific and professional journals, the manner of representation of the algorithm, and the human factors had been suggested.

This method was based on the journal issues, impact factor and references provided by the institutes for scientific information and the index of the published journal in the public domain of ergonomics. In using this method, three groups of journals were identified: journal of ergonomics, related journals and basic journals respectively. The qualities of the journals were classified in accordance to the 4 groups (ABCD levels). The method of indexing ergonomics journals has been developed in 2004. It must be noted that at present, the absence of a continuous electronic evaluation system and the absence of correspondence and documentation on the process of functioning in all publishing departments in the medical universities of the country is causing a wide gap in research in the realm of scientific perspective and competency. The presence of high quality scientific publications for the purpose of publishing and transferring results of researches for policy making purposes and for utilizing research results is a very important task.

One important approach to promote Iranian contribution to the world of science is to increase the number of Iranian articles published through foreign indexed journals. Improving the quality and quantity of Iranian journals are the best solutions for our journals to be indexed in reputable databases. Improvement in terms of quality and quantity can only be achieved through the evaluation processes. Some researchers reported a positive correlation between the qualitative and quantitative criteria in the Iranian scholarly scientific publications indexed by Web of Science (15). Journal Evaluation would also result to a strict observation of the most important factors to mention a few, the journal’s basic publishing standards that would definitely result in an increase number of journals to be indexed in reputable databases.

Evaluations of the country’s medical journals not only provide a medium of compliance to international standards but also led to further indexing of journals in accredited international indices. Presently, we could say that all of the country’s medical journals are indexed in one or more databases and this can be considered a step towards globalization. In general, we can say that the process of promoting the country’s medical journals based on the present contents of evaluation checklist, is on the rise and the needed tool has been established and has met the necessary reliability and credibility.

Unfortunately, the present evaluation checklist lacks the ability to evaluate all aspects of a publication such as; structure and contents of an article. For such reasons, it seems that revision of this checklist and creating another checklist which not only be able to completely and comprehensively evaluate journals but also evaluate journals in terms of the latest criteria agreed upon by most of the personnel of the department of the universities publications is considered a necessity. Considering the fact that the present study seems to coincide with the growing development of the journals in recent years, but in truth, the changes observed on the journals’ situation cannot be used as a tool in this regard, the reason behind is the publication staffs desire to impose the local journals to comply with international standards and to the standards defined by INCMS and also major parts of these standards are dependent of the country’s publication rules and regulations which were found to be effective.

The following suggestions are presented for improving the situation of the country’s medical sciences publications in line with the results of the research:

It is necessary that in the next phase of the evaluation process, attention must be paid to another checklist that would include different indicators aside from those mentioned in the previous checklist.

It is important that in performing the annual evaluation process, evaluation and assessments of the journal’s contents be also considered and the major points be specially assigned to this item.

In order to profit from the experiences and suggestions from the country’s publications staff, a 2 day workshop be implemented in order to develop a checklist for content evaluation.

The indicator, technical quality of a publication is completely imaginative and more dependent on one’s choice so therefore it is suggested that this item be removed from the evaluation form specially that this is not applicable in an electronic publication.

More education for the journal’s editors and members of the editorial board

Provision of an appropriate environment towards electronic publications of journals and an electronic evaluation of publications.

A comparison in the number of published medical journals between Iran and other countries is necessary and needs more assessments.

Ethical considerations

Ethical issues (Including plagiarism, Informed Consent, misconduct, data fabrication and/or falsification, double publication and/or submission, redundancy, etc) have been completely observed by the authors.