Evaluation of the process and outcomes of the Global Matrix 3.0 of physical activity grades for children and youth.

BACKGROUND/
OBJECTIVE: Participation in the Active Healthy Kids Global Alliance (AHKGA) Global Matrix initiative represents a significant work and resource investment for Report Card teams. The objective of this paper was to evaluate the process and findings of the Global Matrix 3.0 and formulate recommendations for improvement.
METHODS: The evaluation of the Global Matrix 3.0 was planned prior to its development and a list of potential process, impact, and outcome indicators were pre-identified. These indicators were informed by online surveys, direct reports, MailChimp's statistics, Google Analytics of the AHKGA website, and emails sent to the AHKGA Board of Directors.
RESULTS: Five online surveys were completed by 88%-100% of the targeted respondents. High satisfaction ratings were observed for most of the Global Matrix 3.0 methods, key steps, concepts, and the resources (e-blasts and website) provided by the AHKGA. A total of 496 open-ended comments were provided in the five surveys, including 199 comments reporting issue(s), and 38 reporting both positive feedback and issue(s). The participating Report Card teams successfully assigned a grade to each physical activity indicator, produced a Report Card document, and wrote a short Report Card article.
CONCLUSION: This evaluation process allowed for the identification of needed improvements and the formulation of recommendations for future Global Matrix initiatives. This work highlighted the need for the development of physical activity behavior assessment tools that would be internationally adopted and culturally adaptable to varying contexts to improve the standardization of physical activity surveillance at the global scale.

Introduction

The Global Matrix on physical activity for children and youth1, 2, 3 is an international initiative where multidisciplinary teams of experts from numerous countries concurrently develop national Report Cards with physical activity grades following a harmonized development process. This initiative is led by the Active Healthy Kids Global Alliance (AHKGA), an incorporated not-for-profit organization dedicated to improving the physical activity of children and youth from around the world. In 2018, the AHKGA released the Global Matrix 3.0, compiling Report Card grades from 49 countries for 10 physical activity indicators: five behavioral indicators (overall physical activity, organized sport and physical activity, active play, active transportation, sedentary behavior), one individual characteristic indicator (physical fitness), and four sources of influence indicators (family and peers, school, community and environment, government). Forty-nine short Report Card articles summarizing the main findings from each participating country, three articles discussing results from the Report Cards by predetermined Human Development Index (HDI)7, 8, 9 categories (low and medium HDI, high HDI, and very high HDI),10, 11., 12 an article integrating the findings from all HDI papers, and an article reporting the international impact of the Global Matrix and Report Card initiative were published in a special issue of the Journal of Physical Activity and Health (JPAH).

Participation in the Global Matrix represents a significant investment in both human and financial resources for all the involved researchers and stakeholders. Report Card teams have to pay registration fees and follow a specific harmonized development method to develop their Report Card. Therefore, the AHKGA designed a method to evaluate the Global Matrix 3.0 process and outcomes to carefully reflect on opportunities to work more effectively and efficiently in the future. This evaluation process involved the collection of process and outcome indicators through online surveys and reports by email at different stages of the Global Matrix 3.0 development process. The objective of this paper was to present the main findings of these evaluations and to formulate recommendations for the improvement of future Global Matrix initiatives.

Methods

Global Matrix 3.0 and harmonized report card development

The detailed methods for the development of the Global Matrix 3.03 and Report Card, have been previously published. In each participating country, one to three Report Card leaders were officially designated to manage the national Report Card development, and to ensure effective communication between the AHKGA Executive Committee and the Report Card team. Report Card teams had to compile the best available published and unpublished evidence to assign letter grades to physical activity indicators following harmonized benchmarks and grading rubric.

A mentoring system was developed for the Global Matrix 3.0: new Report Card leaders and teams were paired with an intermediate mentor who was an experienced Report Card leader from a country that participated in the Global Matrix 2.0. In addition, six regional mentors (Africa, Asia, Europe, North America, Oceania, and South America) were designated to provide help and guidance to the countries when needed and to report their progress to the AHKGA Executive Committee. The AHKGA Board of Directors coordinated the Global Matrix 3.0 development and provided a variety of resources through e-blasts, a monthly newsletter sharing resources and deadlines, and mentorship to support the harmonized development of Report Cards. The Report Card leaders were also provided with individual access credentials for the “members only” section of the AHKGA website where information on the Report Card development process and resource documents were provided. The Report Card grades from each participating country were submitted with their rationale and were audited by selected AHKGA members. Finally, a short Report Card article was written by each Report Card team following a template provided by the AHKGA and audited by selected AHKGA members before being submitted for copy-editing in JPAH.

Evaluation indicators

An evaluation plan for the Global Matrix 3.0 was designed prior its development by members of the AHKGA Board of Directors. The aim of this plan was to gather data and information to strengthen and improve the Global Matrix development process and approach, and also to assess the international impact of the Global Matrices and Report Cards. The impact indicators monitored the progress of achieving the Global Matrix 3.0 objectives. A study entirely dedicated to the international impact of the Report Cards and Global Matrices was recently published in JPAH. The reader is invited to consult this open access publication reporting and discussing the Global Matrix 3.0 impact indicators. Process outcomes meeting the SMART framework criteria (Specific, Measurable, Appropriate, Reliable, Time-consistent) were identified by reviewing the scientific and grey literature and adapted to the specificity of the Global Matrix. The evaluation plan included the possibility to collect unanticipated indicators by offering the opportunity to report any negative or positive side product of the Global Matrix process in open comments on evaluation surveys and by email to everyone involved at any stage of the development. In addition, specific outcome indicators for the expected Global Matrix 3.0 outputs were also pre-identified to assess if the objectives of the Global Matrix 3.0 were achieved. The evaluation of these indicators was informed by online surveys, direct reports, MailChimp’s statistics, Google Analytics of the AHKGA website, and emails that were sent to the AHKGA Board of Directors. MailChimp (The Rocket Science Group, Atlanta) is an email marketing platform that provides analytics on the number of recipients who received the e-blasts, opened them, and clicked on relevant links. Google Analytics (Google, Mountain View) is a Google service that tracks AHKGA website traffic and usage.

Online surveys

Five online surveys were developed using REDCap Software, a secure, web-based application designed exclusively to support data capture for research studies, for evaluation purposes and circulated by AHKGA: three sequential Report Card Leader Surveys, and two sequential Mentoring Surveys. A web link to the online surveys was circulated to the Report Card leaders through the monthly AHKGA e-blasts. Regular reminders to complete the surveys were sent to the Report Card leader in subsequent e-blasts and by e-mail. Only one Report Card leader per participating country was allowed to complete the online surveys. The Report Card Leader Surveys were circulated at three time points of the Global Matrix 3.0 development process as seen in Fig. 1, and their content evolved depending on the time point. Open- and close-ended data on satisfaction and potential issues concerning the Global Matrix 3.0 key steps, concepts and process were collected. The Mentoring Surveys were circulated at two time points (Fig. 1). The two Mentoring Surveys were anonymized, identical, and aimed to evaluate the mentorship organization for the Global Matrix 3.0. In these surveys, Report Card leaders rated (from 1 to 10) the responsiveness, engagement, and overall organization of AHKGA, their intermediate mentor, their regional mentor, or their mentee(s) (in the case of mentors completing the surveys). In both types of surveys, satisfaction rates were evaluated using three-point Likert scales.

Fig. 1

Timeline presenting the key steps of the Global Matrix 3.0 development and the time of the evaluation surveys. AHKGA = Active Healthy Kids Global Alliance; JPAH = Journal of Physical Activity and Health.

Ethics statement

This evaluation protocol was submitted to the Children’s Hospital of Eastern Ontario Research Ethics Board (CHEO REB) (#18/08X). CHEO REB determined that this project did not meet the Tri-Council Policy Statement Ethical Conduct for Research Involving Humans definition of research but corresponded instead to a quality assurance project “quality assurance/quality improvement/program evaluation”.

Results

Process indicators

Global Matrix 3.0 country and participant information

Table 1 summarizes the information on countries that participated in the Global Matrix 3.0. While the AHKGA aimed for a maximum of 75 countries to participate in the Global Matrix 3.0, 51 countries registered and had paid their registration fees by the end of January 2018. Over the development of the Global Matrix 3.0, two countries dropped out due to lack of time and lack of new data respectively. Table 2 presents descriptive statistics concerning the number of participants to the Global Matrix 3.0. Global Matrix 3.0 “participants” refers to all the people who have contributed to the development of the Global Matrix 3.0 (i.e. AHKGA Board of Directors, Report Card leaders and team members). Fourteen country Report Cards were led by one or several females, and six were co-led by at least one female. During the development of the Global Matrix 3.0, the AHKGA Board of Directors was composed of four females and six males. The Report Card leaders also reported the number of members who dropped-out their Report Card team, the number of new members who joined, and the reason(s) why in the Report Card leader survey 2 (Table 3).

Table 1

Global Matrix 3.0 country participation information.

Indicator	Total	Low and Medium HDI countries	High HDI countries	Very High HDI countries
Number of participating countriesa	49	9	10	30
Country dropout	2	1	1	0
New participating countries	15	4	4	7
Global Matrix 2.0 returning countries	34	5	6	23

Notes: HDI = Human Development Index, NA = not applicable. aA total of 51 counties registered and paid their registration fees by January 2018, but two countries dropped out and 49 fully participated in the Global Matrix 3.0 process.

Table 2

Global Matrix 3.0 participants’ characteristics.

Indicator	Total	Mean number of members (range) per Report Card team
Total number of Global Matrix 3.0 participants	513	10 (3–23)
Number of males	258 (50%)	5 (0–17)
Number of females	255 (50%)	5 (0–16)
Number affiliated with a university (%)	NA	68% (0%–100%)
Number affiliated with a governmental institution (%)	NA	10% (0%–67%)
Number of institutions affiliated with the Global Matrix 3.0 participants	283	6 (1–15)

Note: NA = not applicable.

Table 3

Number and reasons of dropped-out Report Card team members and new Report Card team members reported in the Report Card Leader Survey 2.

Report Card team members who dropped out n (%)	13 (2%)
Report Card team members drop-out reasons	Lack of time (n = 7)Moved to another position/ city/ country or to retirement (n = 4)Removed/quitted due to lack of participation (n = 2)
Number of new Report Card team members	9 (1.7%)
New Report Card team members reasons	Expertise needed for the Report Card (n = 4)Replacement of Report Card team member who dropped out (n = 2)Help needed due to large amount of workload (n = 2)Recommended by AHKGA (n = 1)

Responsiveness, engagement and organization of the Global Matrix 3.0 report card leaders

The three Report Card Leader Surveys were completed by 100% of the targeted respondents (i.e. one Report Card leader per participating country, n = 49) and the Mentoring Surveys 1 and 2 were completed by 98% and 88% of the targeted respondents (i.e. one Report Card leader per participating country and an additional regional mentor, n = 50), respectively. The intermediate and regional mentors rated (from 0 to 10) the responsiveness, engagement and overall organization of the Global Matrix 3.0 participants using Likert scales in the Mentoring Surveys. A summary of these ratings is presented in Table 4. The statistics provided by MailChimp regarding the use of the e-blasts are presented in Table 5. Finally, the number of page views in the AHKGA website member only section over the year 2018 provided by Google Analytics is presented in Fig. 2.

Table 4

Mean responsiveness, engagement and overall organization ratings of the Report Card leaders evaluated in the Mentoring Survey 1 and 2.

Indicators	Mentoring Survey 1	Mentoring Survey 2
Number of Report Card leaders rated by intermediate mentors	13	9
Number of Report Card leaders rated by regional mentors	17	34
Mean responsiveness rating (out of 10)	8.0	8.2
Mean engagement rating (out of 10)	8.0	8.1
Mean overall organization rating (out of 10)	8.1	8.4

Table 5

Mailchimp statistics concerning the use of the e-blasts circulated monthly by AHKGA.

Title of the e-blast	Send Date	Total Recipients	Open Rate	Total Opens	Click Rate	Total Clicks
Global Matrix 3.0 - January 2018 E-Blast	2018-01-26 16:23	68	75%	373	51%	106
Global Matrix 3.0 - February 2018 E-Blast	2018-02-26 12:59	70	73%	940	40%	81
Global Matrix 3.0 - March 2018 E-Blast	2018-03-29 15:39	69	75%	637	59%	157
Global Matrix 3.0 - April 2018 E-Blast	2018-04-24 10:25	67	84%	629	57%	188
Global Matrix 3.0 - May 2018 E-Blast	2018-05-14 12:03	67	82%	1448	60%	190
Global Matrix 3.0 - June 2018 E-Blast	2018-06-22 15:18	77	77%	1343	57%	181
Global Matrix 3.0 - July 2018 E-Blast	2018-07-17 13:47	77	79%	728	42%	186
Global Matrix 3.0 - August 2018 E-Blast	2018-08-27 13:39	76	72%	623	42%	132
Global Matrix 3.0 - September 2018 E-Blast	2018-09-19 8:18	76	76%	734	11%	20
Global Matrix 3.0 - October 2018 E-Blast	2018-10-14 21:30	78	73%	925	11%	16
Global Matrix 3.0 - November 2018 E-Blast #1	2018-10-31 17:05	78	74%	792	53%	142
Global Matrix 3.0 Materials Needed Reminder	2018-11-07 11:17	77	74%	744	52%	65
Global Matrix 3.0 Materials Needed Reminder #2	2018-11-13 13:57	76	78%	340	60%	80
Global Matrix 3.0 - November 2018 E-Blast #2 - FINAL ONE BEFORE LAUNCH!	2018-11-17 15:08	123	63%	635	38%	324

Fig. 2

Number of page views in the AHKGA website member only section over the year 2018 provided by Google Analytics.

Responsiveness, engagement and organization of the intermediate and regional mentors

The Report Card leaders rated the responsiveness, engagement and overall organization of their intermediate mentors and/or the regional mentors using Likert scales (from 0 to 10) in the Mentoring Surveys. A summary of these ratings is presented in Table 6. All intermediate and regional mentors reported that they spent ≤1 h per week mentoring countries.

Table 6

Mean Responsiveness, engagement and overall organization scores of the intermediate mentors, the regional mentors, and AHKGA evaluated in the Mentoring Survey 1 and 2.

Indicators	Mentoring Survey 1	Mentoring Survey 2
Number of Report Card leaders rating intermediate mentors	23 (47%)	17 (35%)
Mean intermediate mentors responsiveness score	7.9	7.9
Mean Intermediate mentors engagement score	7.7	7.7
Mean intermediate mentors overall organization score	7.5	7.5
Number of Report Card leaders rating regional mentors	31 (63%)	31 (63%)
Mean regional mentors responsiveness score	8.0	8.4
Mean regional mentors engagement score	8.0	8.3
Mean regional mentors overall organization score	8.1	8.4
Number of Report Card leaders rating AHKGA	36 (73%)	36 (73%)
Mean AHKGA responsiveness score	9.4	8.5
Mean AHKGA engagement score	9.4	8.8
Mean AHKGA overall organization score	9.1	8.8

Responsiveness, engagement and organization of AHKGA Board of Directors

A total of 12 AHKGA Board of Director meetings took place between January and November 2018 (one per month in addition to a special three-day strategic planning meeting which occurred in January), for a total of 28 h of face-to-face or teleconference meetings. For each of these meetings, an agenda was prepared and circulated by the President of AHKGA, and the attendance, discussions, and list of action items were recorded in the minutes. The President of AHKGA prepared and circulated an e-blast sent to all country leaders within a week following each of these meetings. In addition to these recurring activities, the AHKGA Board of Directors created the resource documents that guided the development of the Global Matrix 3.0; created and maintained the AHKGA website, led the audit of the Report Card grades and of the short Report Card articles; contributed to the writing of the integrated scientific articles; prepared workshops and scientific presentations for the release of the Global Matrix 3.0 at the Movement to Move event; and developed adaptable templates of a press release, key finding document, question and answer sheet, social media material, and infographics dedicated to help the Report Card leaders promote the dissemination of their findings. All Report Card leaders who were not members of the AHKGA Board of Directors were asked to rate the responsiveness, engagement and overall organization of the AHKGA using Likert scales in the Mentoring Surveys; a summary of these ratings is presented in Table 6.

Collateral Impact

A series of success stories related to the Report Card and/or Global Matrix activities was already published elsewhere. One of these stories, classified in the “capacity building” category, is a relevant unexpected side effect that resulted from the Global Matrix 3.0 development process: the development of the 2018 Ghana’s Report Card facilitated the training of senior students in physical education and sport in data monitoring and surveillance. The Report Card leader of Uruguay also reported recently that the development of their Report Card was a good training opportunity for the members of his research group and his senior undergrad students.

Satisfaction measures, challenges and issues

The satisfaction of the Report Card leaders concerning key steps and concepts of the Global Matrix 3.0 was assessed though the Report Card Leader Surveys 1, 2, and 3. A selection of satisfaction ratings organized by themes is presented in Fig. 3A–H. The Report Card leaders also reported potential issue(s) through open-ended responses in several sections of the Report Card Leader Surveys and Mentoring Surveys. A total of 496 comments were provided in the five surveys, including 103 positive comments, 156 neutral or informative comments, 199 comments reporting a/several issue(s), and 38 reporting both positive feedback and some issue(s). The number of issues mentioned in the last two categories of comments is summarized in Table 7. Finally, the Report Card leaders were also asked in Report Card Leader Survey 3 to report if they (or someone else in their country) were planning on participating in the next Global Matrix (Global Matrix 4.0). Out of the 47 who replied, 85% replied “yes”, 15% replied “unsure”, and none replied “no”.

Fig. 3

Satisfaction rates of the Report Card leaders collected in the Report Card Leader Surveys 1, 2, and 3 concerning the registration process (A), the Report Card team (B), the Report Card process (C), the final Report Card documents (D), the short Report Card articles and the integrated articles (E), the release of the Report Cards and of the Global Matrix 3.0 (F), the supportive resources provided by AHKGA (G), and the general satisfaction concerning the Global Matrix 3.0 (H).

Table 7

Summary of the challenges/issues reported in the Report Card Leader Surveys 1, 2, and 3, and in the Mentoring Surveys 1 and 2.

	Mentoring Survey 1	Mentoring Survey 2	Report Card Leader Survey 1	Report Card Leader Survey 2	Report Card Leader Survey 3	Total
Lack of new data/ good quality representative data			13	48	14	75
Difficulty to find funding			7	15	26	48
Benchmarks/indicators are unclear/not detailed enough	1		3	12	12	28
Lack of support from the mentor	7	6	1	2	1	17
Difficulty to recruit Report Card team members/to have a complete Report Card team			4	6	1	11
All the Report Card team members were not responsive/ contributing at the same level				5	4	9
Registration fees (for the Global Matrix or Movement to Move event) were too expensive			4	1	2	7
The short Report Card articles were too restrictive					7	7
Difficulty to reach a target audience/ have an impact in the media					6	6
Mentored Report Card leader was not responsive/ cooperative	1	1	1		2	5
Challenging deadline/ complain about deadline change/ unclear deadline	1		3	1		5
Benchmarks and indicator are not relevant and adapted to all context (low and medium HDI countries)			2	1	2	5

Note: This table presents the count of how many times each issue or challenge was mentioned. Several comments contained more than one mention of issue or challenge. Only the issues that were raised a minimum of five times are presented. As there were several open spaces in each survey, the same issue could potentially be mentioned several times by the same Report Card leader.

Outcome indicators

The expected outputs of the Global Matrix 3.0 included the compilation of physical activity grades, the production of Report Card documents, the publication of short Report card articles and peer-reviewed articles integrating and comparing the international findings.,10, 11., 12 The Report Card leaders were also invited to rate their confidence in each physical activity grade of their Report Card by entering a number from 1 to 10 in the Report Card Leader Survey 2. The descriptive statistics of their confidence ratings are presented in Table 8.

Table 8

Descriptive statistics of the confidence ratings for the physical activity grades reported in the Report Card Leader Survey 2.

	PA	SP	AP	AT	SB	PF	FAM	SCH	COM	GOV
Number of replies	46	46	45	46	46	44	45	45	46	46
Mean (out of 10)	8.1	7.8	6.4	8.0	7.4	6.8	6.4	7.7	7.3	7.1
Ratings >6	85%	80%	64%	83%	80%	68%	60%	84%	74%	67%
Max	10	10	10	10	10	10	10	10	10	10
Min	4	2	0	2	2	0	0	0	0	0

Notes: PA = Physical Activity; SP = Organized Sport and Physical Activity; AP = Active Play; AT = Active Transportation; SB = Sedentary Behaviors; PF = Physical Fitness; SCH = School; COM = Community and Environment; FAM = Family and Peers; GOV = Government. Some Report Card leaders specified in comments that that they decided to rate their confidence level “0” for the incomplete grades of their Report Card while some others specified that they rated them “10”.

Discussion

The collective efforts from the Report Card leaders and team members, intermediate and regional mentors, and AHKGA Board of Directors allowed us to collect a variety of quantitative and qualitative information to inform the evaluation of the process, impact, and outcomes of the Global Matrix 3.0.

Overall, high satisfaction ratings were observed concerning the majority of the Global Matrix 3.0 methods, key steps, and concepts identified in Fig. 3. Table 5 and Fig. 2 show that the resources (e-blasts and website) provided by AHKGA were used all year long by the majority of the Report Card leaders, and the Report Card leaders expressed high satisfaction with these supporting materials (Fig. 3G). The general overview of the findings presented in this article shows that despite the challenges and issues encountered, the Global Matrix 3.0 process facilitated the success of all participating Report Card teams to submit their 10 physical activity grades, publish their short Report Card articles, and potentially have a positive impact nationally or internationally. However, several points of improvement were identified to inform recommendations for future Global Matrix initiatives.

Recommendations

Findings from the Mentoring Surveys showed that the mentorship organization created for the Global Matrix 3.0 was too complex and confusing for the participants. Several participants failed to even identify their mentor or mentee country, and lack of support from the mentor was one of the main issues raised in the comments collected in the surveys (Table 7). In practice, most country leaders contacted AHKGA directly for guidance and mentoring. Consequently, it is recommended that central support should be provided by a smaller number of mentors selected by AHKGA that will guarantee their involvement and responsiveness to help guide and mentor new and requesting Report Card teams. It would also be interesting to take advantage of the expertise of all the future Report Card team members with the creation of a discussion/advice forum on the AHKGA website where anyone could post a question and receive a quick answer from available experts or share advice or resources. It is recommended for AHKGA to pursue providing additional support and resources for the development of the Report Cards through e-blasts, the “members only” section of the AHKGA website and the audit process of the grades and their rationale. In addition, the AHKGA has also provided support for the dissemination of the Report Cards through the AHKGA website and Twitter account, and with the provision of a detailed dissemination package described elsewhere. Findings presented in Fig. 3G show that Report Card leaders found these resources useful; therefore, AHKGA is encouraged to continue these efforts.

As reported by several Report Card leaders in the surveys (Table 5), the anticipated deadlines for the submission of the final physical activity grades that were provided at the beginning of the Global Matrix 3.0 development stage were very challenging. Once the registration period was closed, the Report Card teams ended with a strict four-month deadline to complete their grade assignment and a six-month deadline to write their short Report Card article. It is recommended that AHKGA aim to provide more lead time in future Global Matrices, allowing more anticipation and flexibility to the Report Card leaders in the planning of their Report Cards development.

Comments collected in the surveys (Table 5) highlighted the challenge faced by AHKGA in the choice, definition and descriptions of the indicators and their associated benchmarks. More detailed and strict criteria for the benchmarks to evaluate the indicators was requested; however, several Report Card leaders complained as well about the lack of good quality data in their country, in particular of data that fits with the benchmarks that were provided by AHKGA. It was also raised that some of the indicators or benchmarks may be too specific and more adapted to very high HDI countries, and not always relevant for the others. Some Report Card leaders suggested that AHKGA should take the leadership on the development of standardized tools dedicated to the collection of physical activity behavior data, or provide a clear recommendation of tools to use for the Report Card teams willing to collect their own data or contributing to the development of new national physical activity surveillance systems. As several Report Card leaders expressed their interest in contributing to the development of more detailed or clear benchmarks or even specific AHKGA tools to assess and analyze the public policies, a feasible approach in the first instance would be for AHKGA to lead an interactive discussion involving all interested Global Matrix participants to update physical activity indicators, their definitions and associated benchmarks. The development of physical activity behavior assessment tools that would be internationally adapted and adopted could become a parallel project to the Global Matrix initiative if adequate resources permit.

With the increasing number of countries participating in the Global Matrix initiative comes a growing need for human, material and financial resources to successfully coordinate its development. Before the initiation of Global Matrix 3.0, AHKGA decided to adopt a registration fee system that increased with the level of HDI of the countries in order to facilitate the participation of the developing countries. The lowest satisfaction ratings reported by the Report Card leaders were related to the registration fees and the availability of funding for the design of the Report Cards (Fig. 3A). And Report Card leaders from countries belonging to each category of HDI reported difficulty to find funding in the open spaces for comments of the surveys (Table 7). However, the difficulty to find funding is not a problem that is uniquely tied to the Global Matrix initiative but widespread in scientific research and knowledge translation globally. The Global Matrix 3.0 registration fees were used to pay for the overall management of the Global Matrix 3.0 initiative, to finance the publication of the Global Matrix 3.0 articles in a special issue in JPAH, develop and maintain the AHKGA website, and assist with expenses related to the release at the Movement to Move event in Adelaide Australia. It is recommended that the benefits and supports provided for the registration fee be clearly presented. It is also recommended that the AHKGA explore strategic partnerships in order to obtain financial support and reduce the registration fees for future Global Matrices and/or offer more support to low- and medium-HDI countries. Meanwhile, Report Card leaders and teams need to be creative, persistent, and support each other in their quest to secure funding to support report card developments, releases and promotions.

Despite the generally good confidence ratings concerning the Report Card grades that were reported by the Report Card leaders (Table 8), the lack of good quality data was the issue most leaders noted (Table 4). It is an important recommendation to highlight to all the future participating countries that a lack of good quality data or lack of new data since a previous Report Card has been published is still a relevant and meaningful finding to report. This is particularly true concerning some of the sources of influence indicators (school, community and environment, government). The main message of a Report Card can be to highlight the efforts or the absence of efforts that have been accomplished to tackle the population health issue of children and youth physical inactivity or to improve the surveillance within a country. Regardless of the availability of good quality data, the Report Card remains a potentially powerful tool for advocacy.

There are still parts of the world missing in the Global Matrix initiative, and the low- and medium-HDI countries and the high HDI countries are also underrepresented in comparison with the very high HDI countries. However, the AHKGA is an active network of interconnected researchers and physical activity experts from various fields, distributed across six continents. Therefore, to have more participating countries in the next Global Matrices, in particular from the low- and medium-HDI countries, it is recommended that the international network of the AHKGA keep promoting this project through publications, social media, e-mails, conferences and mouth to mouth interactions to connect with new experts and teams from missing countries, in particular the with the low- and medium-HDI countries and the high HDI countries.

Finally, it is recommended that this evaluation process be continuous. Keeping ongoing records of media impact, publications, presentations, success stories, and issues that arise will need to be proactively pursued in order to facilitate the improvement and optimize the impact of this global initiative.

Strengths and limitations

The main limitation of this evaluation was that it relied essentially on responses to surveys and on reports made by the Report Card leaders, the Report Card team members, the mentors, and the AHKGA Board members. Consequently, the results may be biased and/or incomplete, potentially making the opinion of individual who replied more consistently and reported more information weighting more than others. This evaluation plan was also designed and led by members of the AHKGA Board of Directors, constituting a potential source of personal bias. On the other side, a complete understanding of the Global Matrix and Report Card harmonized development process was essential to the design and understanding of this evaluation process. In addition, this evaluation plan allowed to collect feedback from the Report Card teams at all the stages of development of the Global Matrix 3.0, allowing to identify potential issues that were still addressable at the moment they were raised.

Conclusion

A variety of quantitative and qualitative information was collected to inform the evaluation of the process and outcomes of the Global Matrix 3.0. From these evaluations the Global Matrix 3.0 was a successful international initiative in terms of delivery of the expected outcomes - the physical activity grades for each participating country - despite several identified issues that challenged the Report Card teams. This evaluation process allowed for the identification of several improvements and the formulation of recommendations for future Global Matrix initiatives and highlighted the need for the development of physical activity behavior assessment tools that would be internationally adopted and culturally adaptable for all contexts to improve the standardization of physical activity surveillance at the global scale.

Declaration of competing interest

The authors want to disclose that they were members of the AHKGA Board of Directors during the development of the Global Matrix 3.0.