New Report: Measuring Community Engagement and Higher Education

Posted by on December 1, 2008

This report from Australia, “In Search of Evidence – Measuring Community Engagement and Higher Education” is available at:
http://www.eidos.org.au/news/items/2006/12/119393-upload-00001.pdf

EXECUTIVE SUMMARY

Recent discussions about the inclusion of the community engagement agenda as part of the National Protocols for Higher Education Approval Processes and calls for Third Stream funding have highlighted the need for some measures of university-community engagement. Such measures of engagement should offer tangible evidence of the role that universities play in the educational, social and economic wellbeing of local communities and the nation more broadly. In Search of Evidence builds on the self-audit of Victorian universities undertaken for the Department of Education and Training, Victoria in 2005, reported in Beyond Rhetoric: University-Community Engagement in Victoria (available at http://www.eduweb.vic.gov.au/edulibrary/public/highered/postsece/Beyond_Rhetoric_full_report_amended_ver_July05.pdf)

One of the recommendations made was for the development of appropriate indicators for engagement in Research and Teaching Quality frameworks.

The pilot project therefore investigates the practicality of using currently collected and published information to report against a set of indicators to determine the type and extent of community engagement activity in Victorian universities. This will involve the examination of national and international literature in order to develop a preliminary set of indicators, which will then be tested against the data available at
Victorian universities. A major condition for such an indicator set is that it does not generate yet another demand on universities to generate data, but that it can operate within the framework of the current data collected and reported by universities. The intention therefore of this report is not to suggest a new suite of indicators but to select from those that have been suggested in the literature, a set which could be applied to evaluate the nature of community-university engagement in Victoria. Or, to be more precise, to actually assess whether such an approach might work.

To test this, a two-stage approach is used. First, on the basis of a comprehensive international literature review an attempt is made to draw up a set of indicators relevant to the concept of community engagement. Second, using information available in the public domain, i.e. published reports and data contained on university websites, we investigate the extent to which the indicator set can be populated to constitute a reasonable understanding of the community engagement activities within and between universities in
Victoria.

The international literature search resulted in six sets of indicators and protocols that could prove useful in the development of a set of indicators for community engagement. These indicator sets were evaluated for their usefulness to the objective of this project based on the following criteria: Relevance, Reliability, Transparency, Availability, Auditability and Cost. This assessment resulted in the selection of the so-called Russel Group Indicators, which had the highest overall score (according to those criteria) and also provided the most comprehensive coverage of what in the broadest sense may be captured under the heading of community engagement. We opted to continue the project with that set, while adding in a number of
other relevant measures. A trial run consisting of an analysis of existing Victorian university reports and other available forms of data was undertaken to see to what extent the suggested set of indicators could
indeed be viably used to determine the extent of community engagement. The outcomes of this trial run were disappointing. Our research suggests that at present there is insufficient information reported to even superficially understand the level of community engagement being undertaken by a university. We believe that this is not the result of the indicator set selected, but is the consequence of what universities report – or not report for that matter – in the area of community engagement. Real reporting of achievements against stated intention only takes place to a very limited extent.

KEY FINDINGS

*Community engagement can be defined as the mechanism through which universities achieve the goals they have articulated in relation to specific communities in terms of their trinity of basic functions, i.e. teaching, research and service, whether at the strategic university level, or in project specific contexts.
*A number of reporting protocols have been produced to facilitate the assessment of University performance in relation to their function as institutions of teaching, research, and service.
*There seems to be a dominance of indicators to measure commercialisation, technology transfer and spin-offs.
*At the university level, typically the notion of university-community engagement is enshrined in Mission & Vision Statements and/or the Strategic Plan. Yet, these goals and objectives in general do not seem to be well supported by formal reporting.
*Anecdotal evidence suggests that many university managers are not aware of the breadth of community engagement that currently occurs within their own institutions
*If community engagement is reported upon, it is done in an inconsistent and qualitative way (which is intended here as a statement of fact rather than implying that quantitative is to be preferred over qualitative).
*Where there is more detailed information, it generally is found at the faculty or, more often, department/school levels. Yet again we need to note that this type of reporting is ad-hoc, primarily qualitative and by no means consistent across an institution.
*The present report suggests that in theory it is possible to arrive at a set of indicators that perhaps may not be comprehensive, but that is robust enough in international comparative terms to serve as a workable operationalisation of the concept of engagement.
*Examining the extent of community engagement on the basis of existing information as yet is an impossibility.

RECOMMENDATIONS

*While not asking universities to collect new data, the reporting of existing data could take better account of the university-community engagement activities; Universities may wish to consider publishing, based on their particular mission and objectives, what they consider to be their own best/good practice with respect to university-community engagement;
*University institutional planning offices may wish to explore the quantitative application of the performance indicators identified in this report to their organisations performance with respect to university-community engagement; and
*While it is apparent that successful university-community engagement involves institutional support and acknowledgement at all levels of seniority, this will only be achieved when the real benefits of engagement, particularly on the university side of the equation, are acknowledged and become a recognised part of the university culture.