Empowerment evaluation of a Swedish gender equity plan.

BACKGROUND: Empowerment is essential for gender equity and health. The city of Malmö, Sweden, has formulated a development plan for gender equity integration (GEIDP). A 'Policy Empowerment Index' (PEI) was previously developed to assess the empowerment potential of policies.
OBJECTIVES: To pilot-evaluate the GEIDP's potential for empowerment and to test the PEI for future policy evaluations.
DESIGN: The GEIDP was analyzed and scored according to electronically retrieved evidence on constituent opinion, participation, capacity development, evaluation-adaptation, and impact.
RESULTS: The plan's PEI score was 64% (CI: 48-78) and was classified as 'enabling', ranging between 'enabling' and 'supportive'. The plan's strengths were: 1) constituent knowledge and concern; 2) peripheral implementation; 3) protection of vulnerable groups; and 4) evaluation/adaptation procedures. It scored average on: 1) policy agenda setting; 2) planning; 3) provisions for education; 4) network formation; 5) resource mobilization. The weakest point was regarding promotion of employment and entrepreneurship.
CONCLUSIONS: The PEI evaluation highlighted the plan's potential of constituency empowerment and proposed how it could be augmented.

Gender equity and women’s empowerment are social determinants of health (1). Power imbalances result in unequal health access and outcomes, knowledge, skills and employment, living conditions, and opportunities (2). Public policies have major impacts on empowerment (3–7).

Despite the European Union’s (EU) gender equity recommendations, women’s empowerment and inequities in individual, social, economic, and professional conditions remain a challenge in Europe (8, 9). Sweden has the tenth highest human development index, the highest gender equality and the lowest income inequality worldwide (10–12). Gender equity is considered fundamental in Swedish internal policy development (13), and the country has signed the EU plan for gender equity’s integration (9, 14). However, differences exist between socially and economically vulnerable groups and women and men on living conditions, education and health, power and influence in society as well as on employment and unpaid work distribution (15). In addition, Malmö has the most ethnically diverse population among Sweden’s big cities (16, 17). In 2011, the Malmö city office (henceforth, City) developed a gender equity integration development plan (GEIDP), in accordance with EU and national guidance. According to the plan, latest by the year 2013, all of the city agencies and services should be working toward equal distribution of services, resources, power and influence irrespective of gender, orientation, and background, and, as the city’s biggest employer, the City should provide equal employment opportunities, conditions, and salaries, and move toward a balanced gender distribution at all work categories (14).

An evaluation index named ‘Policy Empowerment Index’ (PEI) has been developed by Gavriilidis and Östergren (18), aiming to increase understanding on how policy planning can affect constituent empowerment. It aspires to do so by assessing the policy elements that affect empowerment through evidence reviews, stakeholder, and constituent feedback.

Evidence-based evaluations are necessary for more gender-sensitive and empowering policy planning. A few comparative evaluations have been attempted using ad hoc criteria and policy document content analyses (19–21); however, generally accepted criteria for such evaluations are lacking.

This study aimed to pilot-evaluate the Malmö GEIDP from the perspective of comprehensive empowerment of its constituents, in order to contribute with evidence-based feedback to the policy discussions and to evaluate and develop the PEI for more comprehensive and systematic future policy evaluations.

Methods

The PEI was previously developed to assess policy elements that have a potential impact on empowerment. It applies logical and discussed scores (from 0=minimum to 5=maximum) of empowerment potential (18) and confidence intervals (0 to 5). Due to lack of evidence on empowering factors’ impacts, the scores are not evidence-weighted. To make the results more visible, the PEI evaluators attach scores to logical statements. For example, <10% describes ‘a small minority’, 10–50% ‘a significant minority’, 50–75% ‘a majority’, 75–90% ‘a big majority’, and 90–100% ‘a vast majority/almost all’, as previously described (18).

A pilot evaluation of the plan was performed after a few adaptations: Questions (Q) 10 and 11 (on policy evaluation and adaptation) were fused, and Q12 was removed (investigation of related policies) for simplicity (see Appendix 1). The remaining 10 questions interrogated the policy plan on the following issues: 1) participation; 2) capacity building; 3) evaluation/adaptability. The index questions cover these policy empowerment elements as follows:

Constituent concern with the policy issue and participation in agenda setting (Q1, 2), policy planning (Q3), and implementation (Q4).

Building constituent capacities and opportunities through education/training (Q5), employment/entrepreneurship (Q6), network formation (Q7), addressing power inequalities (Q8), and resource mobilization (Q9).

Modes of policy evaluation (Q10).

The PEI questions the evaluation of the electronically retrieved evidence addressing them; the PEI standards and scores are presented in Appendix 1. Each question’s argued scores and CI of the GEIDP are averaged to generate a total score and CI, which are presented as proportions of the maximum 50 for intuitiveness. A summary and discussion of the evaluation and the final result is presented in ‘Results’ and in Table 1.

Table 1

Summary of PEI evaluation, City of Malmö, development plan for integration of gender equity

Q1. How many political constituents (residents in any way affected by the plan) are informed and concerned with the addressed problem?	Score: 3.5 (3–5)
Q2. How was the political agenda set? Did the plan in question start from a discourse in the community and local grassroots movement advocacy, or by professional experts and politicians at the City level and above (or both in interaction)?	Score: 3 (2–4)
Q3. How was the policy planned? Did peripheral agencies and interest groups contribute significantly to the planning?	Score: 2.5 (2–3)
Q4. What percentage of the development plan actions was delegated peripherally for implementation? Will the plan be implemented mainly by the central or peripheral City authorities?	Score: 5 (4–5)
Q5. Does the policy plan call for education/training of the constituents?	Score: 2 (1–3)
Q6. Is peripheral employment and entrepreneurship being strengthened? Will the plan create jobs or business opportunities for women or men?	Score: 1 (1–2)
Q7. Does the plan promote constituent participation in horizontal and vertical networks? Does the plan create links between the community members/citizens/residents and between them and the authorities of the city?	Score: 2 (1–3)
Q8. Are hard to reach, vulnerable or disadvantaged populations being considered and affirmatively protected and empowered (including vulnerable gender and age groups, socially/physically/economically disadvantaged individuals, groups and communities)?	Score: 5 (4–5)
Q9. Does the policy provide for or will there most likely be adequate financial, human and other resources?	Score: 3 (2–4)
Q10. How will the policy plan be evaluated and adapted?	Score: 5 (4–5)
Total: 32 (23–39)	Score: 64%
Policy classification: enabling (supportive–enabling)	CI (46–78)

According to the PEI, a policy can be classified as ‘Dictative’ (score 0–20%), a top–down policy that disregards constituent empowerment, or ‘Empowering’ (81–100%), an optimally focused, planned, and evaluated policy with empowerment in central focus, ‘Directive’ (21–40%), ‘Supportive’ (41–60%) or ‘Enabling’ (61–80%) being the intermediate policy categories as their names imply [see Appendix 2 for the full definitions of these policy categories, as in (18)]. These policy categories are ideal, theoretical models. Real policies may contain elements from more than one policy type, which average on the category indicated by the PEI score.

We conducted evidence and public debate/opinion searches to address each PEI question on constituent opinion and participation in plan design, implementation and evaluation/adaptation, and capacity development through education, employment, networks, and affirmative support. The following search queries were performed on Google, Google Scholar, Web of Science (WoS), and PubMed in English and Swedish using the following search queries: [‘gender equity’ AND ‘study’], [‘gender equity’ AND ‘study’ AND ‘Malmö’], [‘gender1 equity’ AND ‘study’ AND ‘women’], [‘gender equity’ AND ‘study’ AND ‘immigrant’], [[‘survey’ AND [‘gender’ OR ‘women’] AND [‘equity’ OR ‘equality’] AND [‘Sweden’ OR ‘Malmö’]], [‘survey’ AND ‘gender’ AND ‘immigrant’], [[‘gender’ AND [‘equity’ OR ‘equality’] AND Sweden]]. Only the first 100 search results from Google were examined (total 600). Relevant data such as education and employment statistics and official documents were retrieved from ‘Statistics Sweden’ (SCB), the ‘City of Malmö’ webpage, and through contact with city employees. The retrieved documents and statistics were selected according to relevance to each of the search questions and to Malmö, and broader inferences to Sweden were included in the absence of local ones.

The PEI evaluations can optimally be performed by many evaluators and with stakeholder feedback through surveys, interviews, and discussions (18); however, due to practical constraints it was only based on a literature review performed by two of the co-authors (GG and NN). The evaluation report was submitted to the ‘Commission for a Socially Sustainable Malmö’ and was included as an element in its final proposal (22, 23).

Results

A total of 778 documents were retrieved and examined for relevance on Google and Google Scholar (600), PubMed (54), and WoS (124). After examination for relevance and repetition, 70 documents from Google, 30 from PubMed, and 61 from WoS (total 161) were used for the evaluation.

The complete evaluation and bibliography are presented in Appendix 1. The questions and evaluation scores are summarized in Table 1. Below we summarize the GEIDP evaluation findings:

Constituent concern and participation

Q1. The GEIDP addresses the persistence of gender inequity in Malmö, for example, the inequities in employment, living conditions, education and health, power and influence in society, wages, expectations, unpaid work distribution and parental leave, sexual harassment, and gender-based violence. The evaluation suggests that between 50 and 90% of Malmö residents are currently concerned with gender inequality at home and work, and with associated issues such as gender-based violence.

Q2. Gender equity is of widespread concern in Sweden and internationally, and the agenda for this policy appears to have been set beyond the city level, although non-expert local constituents (e.g. party members) have raised the issue in Malmö. The EU and State roles in driving the issue toward the City Council and Board seem to dwarf the local community contributions.

Q3. This is a centrally conceived policy plan with significant but not determining peripheral feedback by local authorities (such as local councils), agencies (such as the educational, technical and cultural agencies) and interest groups (such as the teachers’ union), some of which had developed their own plans for gender equity. Q2 and Q3 could profit from further discussion and input from the policy-planners.

Q4. The plan is to a very large extent delegated to the peripheral authorities, agencies, departments and businesses for implementation. A number of city departments and agencies have already designed their adapted plans. However, the overall control, supervision and support are left to the central authority.

Building constituent capacities and opportunities

Q5. The plan relies significantly and invests on education on gender equity issues among the City employees at all levels but mainly for coordinators and mention is made for education as a means of more equitable social integration. Gender education is targeted toward students of all levels, from primary school to university and leaders. However, no specific programs for targeted training in areas where women or men are lagging behind are planned although some especially vulnerable groups are supported for job seeking by publicly supported programs.

Q6. There is some direct creation of employment through this plan, mainly in its implementation in education on gender, gender sensitive budgeting and communication and in application of gender disaggregated statistics as well as, indirectly, through developmental benefits, private sector attitude change and fairer parental leave distribution. However, there is no explicit provision for extra jobs; therefore the impact on jobs seems unlikely to exceed 1% of the city’s employable residents (i.e. aged 16–64).

Q7. The plan calls directly for the responsibility of the City to support gender equity contact persons of peripheral administrations and companies through an existing website (malmo.se/jamstalldhet) and the city intranet (KomIn). Indirectly, it aims to place new emphasis on the issue, by spreading knowledge and serving as a model for wider equity in the society. The discussions and seminars are likely to improve the communication between the many groups concerned with gender equity, institutions, agencies and businesses active in Malmö and between those and the City. Given the large number of existing groups, it is unlikely that many new groups and networks will be formed because of the plan. Therefore, the proportion of constituents that the plan will indirectly motivate to participate in these links is unlikely to exceed 10% of the resident population.

Q8. Despite living in one of the world’s most equitable societies, many women in Sweden can still be considered disadvantaged and vulnerable. Seventy two percent of all leaders and 82% of board members of publicly listed companies are men. Only 16% of professors are women and a woman’s average salary is 8% lower than a man’s. The aggregate total income of women is 70% of men’s while they contribute threw times more in unpaid services. One in five women feels that she is being treated unequal at work and one in four is unhappy with gender equity in this country. There are over 25,000 crime reports, mostly by intimate partners, and around 6,000 rapes against women every year. Malmö is 37th among 290 municipalities on the SCB’s gender equity index (Jämindex), last among the major cities. Forty percent of its population has an immigrant background and 30% were actually born abroad. Immigrant women often face even more challenges toward employment and empowerment. Men are also disadvantaged in regards to parental leaves.

The plan calls directly for improvement in the both genders situation among the City’s employees first, and through the handling of public matters to all the gender-disadvantaged residents of Malmö, clearly more than 10% of the population.

Q9. Central (City Council) funding and resources (mainly human) are called for, which are most likely to be sufficient since significant funds have already been budgeted and allocated and the City currently has sufficient financial assets and budget commitments were made. Given the local and national academic environment’s focus on gender knowledge and human resources are also most likely to be available.

Modes of policy evaluation

Q10. The plan calls for annual, gender disaggregated statistics of measurable targets and consequence reports from the city boards and administrations to be continuously created, and an annual revision update report and analysis will be presented by the city board. The City will be responsible for the overall follow-up of gender equity integration′s progress by the city administrations and also provide support and coordination (24), p. 15). In addition to the statistics the plan explicitly asks for qualitative equity analyses, in order to sample the ‘residents and users experiences through different kinds of surveys’ (24, p. 24). The City Council must approve changes in the plan. However, as adequate authority is delegated peripherally for implementation, several city departments and administrations have already formulated and adapted the plan to their situations so this is also likely to happen after the evaluations.

Evaluation conclusions

After applying the index scores to these questions (see Table 1 and Appendix 1), a total score of 64% (46–78%) best describes the policy plan as ‘Supportive’ of empowerment and ranging between the ‘Supportive’ and ‘Enabling’ PEI policy categories (see Appendices 1 and 2).

The policy plan’s strong points according to the index are: 1) the wide constituent knowledge and concern with gender inequality (Q1); 2) the peripheral implementation by the city departments, administrations, agencies and companies (Q4); 3) the protection conferred to women and men against gender discrimination by the City as employer, and the push for support by the City agencies for a more equitable society and empowerment through employment, education and fairer distribution of resources and agency (Q8); and iv) the comprehensive evaluation and adaptation procedures in the city (Q10).

The plan scored average on the following points:

A centrally set policy agenda (Q2). Although initiated by grassroots movements, the gender equity agenda in Sweden is nowadays pushed forward by international and national agencies, outside the constituent communities. One central assumption of the PEI is that policies are more empowering when their constituents own them from their conception. Obviously the plan is part of a wider policy framework, which has sprung from population movements in the mid-1900s. However, the PEI here estimates its own empowering potential, and not that of the overall gender equity policy.

The planning was done mainly by City-office experts, with no game-changing peripheral and lay contributions (Q3). Active, wide and equitable constituent participation in policy planning may be effort- and time-consuming but has a potential to empower by delegating agency and influence, and through better responding to current and consensual community needs.

The plan’s provisions for education are focused on gender issues (Q5). Additional capacity development could stem from more explicit support to skill development to overcome proficiency inequalities.

Emphasis is placed on communication, but new network formation is limited (Q7). The existing networks could always benefit from more vivid and wide participation. Maximization of contacts and engagement of different views can empower the community toward coming up with innovative solutions and faster norms’ change, benefiting from Malmö′s cultural diversity.

Resources may be adequate, but more local, creative resource mobilization, e.g. advice for promotion of volunteer work, or even a community ‘inequality tax’ could generate local funds for equity programs, promoting local ‘ownership’ of the plan’s components (Q9).

According to this evaluation, an issue where the plan can be improved from empowerment’s perspective is on promotion of specific and targeted employment and entrepreneurship (Q6). The City has the capacity to promote equitable employment not only as an employer but also through its regulations, services and programs. It can be argued that unemployment levels in the city are relatively low. However, disadvantaged groups certainly exist, such as within ethnic, age groups, or with specific professional orientations, penal and addiction histories. Part time or temporary work is also problematic for sustainable career development, economic independence and empowerment.

Discussion

We conducted a pilot evaluation of a Swedish gender equity plan using an empowerment index, based on diverse evidence sources, ranging from official documents, statistics and peer reviewed articles to published and electronic news and debates. Our aim was to provide feedback to the policymakers on the GEIDP as well as to test and develop the PEI for future policy evaluations for empowerment.

There is a clear need for a richer policy discourse and feedback on policy design (25). Public policies are most commonly evaluated after their implementation, whenever this is possible. This is easier for some policy types, such as monetary and health policies, where the impacts can often be measured. Few others have focused on various models of policy document content analysis according to predefined criteria, which apply best to gender equity policies (19–21). Thus, they have been able to compare gender equity provisions and gender mainstreaming among health policies. Such approaches are obviously valuable for policy discourse and development; however, more systematic and generally accepted and applicable, synoptic tools such as indexes should be developed to turn the art into science. To date, no evaluations of constituent empowerment by policies have been found other than our previous evaluation of the South African traditional medicine policy plan (18). That policy was found to be less empowering that the GEIDP [PEI score 42% (27–57)], mainly due to minor peripheral participation in the planning and implementation; however, comparisons among policies with such different scopes and contents using the PEI, although of interest from an overall empowerment perspective, are in need of further validation and standardization of the index and its methodology. We argue that empowerment of the constituents is important for democratic policy planning and that planning stage evaluations using standardized tools can contribute toward better-designed policies. The PEI is an effort in that direction.

This evaluation has significant limitations. Concessions were made in regards to the number of evaluators and the sampling of stakeholder responses due to time and staff availability constraints. The PEI evaluation can optimally be performed by three or more evaluators including members of the constituency, and combined with more extensive surveys.

However, an extensive review of both official and lay documents on the world’s broadest discussion forum, the World Wide Web, strengthens the validity of the findings. The use of both official and unofficial sources in the evaluation enriches the discourse about the plan, including sources that may be ignored by policymakers. We also communicated with a member of the policy-making team who responded to six out of the 10 PEI questions, suggesting a score of 80%, where our score for those questions was 66%. Similar feedback from several policy stakeholders could further validate our findings.

The PEI may be inadequate for comprehensive feedback to policy planners. However, it can be combined with existing and further developed socioeconomic, political, and health indicators such as the Gini income inequality index, OECD indicators (26), the gender equity index (Jämindex) (27), the Gender-related Development Index (GDI), the Gender Empowerment Measure (GEM), the Gender Gap Index (GGI), the Gender Status Index (GSI), The African Women’s Progress Scoreboard (AWPS), and the UNDP Gender Mainstreaming Scorecard (28, 29). More comprehensive indexes and indicators, for example, also measuring well-being and sustainability, are needed for a more holistic understanding of social welfare and progress (30).

Also, a single policy plan evaluation may be inadequate. Complementary plans and strategies may increase the overall policy’s empowerment potential. Therefore, a multi-sectorial policy ‘cluster’ evaluation is proposed as the next step.

Conclusions

An empowerment evaluation of the Malmo City’s policy plan to increase gender equity in all aspects of city life showed that the plan has a strong potential to empower its constituency, being, according to the PEI terminology, ‘supportive’ and potentially even ‘enabling’ of empowerment’ with a PEI score of 64% (46–78%) (see Appendix 1). The plan’s main strong point according to this evaluation lies in the protection against gender discrimination in employment, education and distribution of resources and agency. Proposals that emerge from the analysis are toward proactive job creation for vulnerable groups and more delegated policy conception and planning.

Comprehensive policy evaluations can contribute toward a more participatory, community enriching and effective policy planning, facilitating the achievement of developmental and public health goals. The PEI can be a part of such evaluations.