Guidelines for defining and evaluating the youth affairs sector

(in Finnish: Suuntaviivoja nuorisotoimialan määrittelyyn ja arviointiin)

Anu Gretschel, Pirjo Junttila-Vitikka & Anne Puuronen

Abstract

The aim of the study was to present an overall examination of the utility of practices for evaluating youth work and the youth affairs sector and for reporting on these, and to indicate how these practices can be reshaped. This also required a fresh look at and a redefinition of the youth affairs sector and its operating environment. 

The study was carried out through network-based collaboration with young people, with operators and decision-makers in the youth sector, and with representatives of funding entities. The network-based collaboration emphasised the need to recognise that actions take place in a performance management environment and that the processes in youth work are diverse. There was also an understanding that improved evaluation practices would benefit all parties. In the proposals for new evaluation models the aim was to create models that would meet the aforementioned needs regarding the development of evaluation across a broad front. The study produced a number of youth work evaluation models that bring together the evaluation objective of: a) producing information for operators as a basis for further developing their operations; b) producing documentation for decision-makers and funding entities in response to the funding of youth work; c) producing information for the entire sector on the use of services and the functioning of the services domain; and d) offering young people opportunities to participate in determining the orientation of services, in the role of co-evaluators. 

For practical purposes the study was divided into four sections. The section on service and development centres for youth work examined the duties and essence of the centres, which fall within the performance management sphere of the Ministry of Education and Culture, and the structural evaluation challenges. The study was undertaken as a networkbased process in cooperation with the service and development centres and representatives of the Ministry. It highlighted the extensive scope of the network as a structure producing various specialist and development services as well as youth services. The most significant development areas presented include the redefinition and highlighting of the criteria for designating service and development centres, and development of the evaluation tool used by the Ministry in evaluating the work of the centres. The study also involved an evaluation of the operation of one of the service and development centres with the aid of a deliberative discussion day method. The evaluation process drew attention to the quality criteria for specialist services. It also validated the project’s presupposition that an inclusive evaluation method that involves young people and the collaborating partners will produce benefits for all participants in the process. 

In the section of the study dealing with online and local services, an evaluation was made of the extent to which six online services and the local services of one city district meet the needs of young people. The evaluators were young people, services providers and representatives of entities providing funding for the services. For the services providers and funding entities the evaluation gave rise to a number of service development ideas that were jointly discussed at length. The evaluation provided the researcher with a basis against which to compare the evaluation and reporting methods used earlier in the services and the results that emerged from these. The results of this part of the study emphasised the need for the identifiers in use today to be supplemented with elements that stress the nature of open youth work. Alongside visitor numbers the reporting should also include information on equal and unrestricted access to services, on the systematic nature of service quality monitoring, on client satisfaction and on the results concerning evaluation of the extent to which services meet young people’s needs. There also emerged a need to further develop methods that document the impact and effectiveness of open youth work and which would take account of the significance of the work not only at the individual level but also for groups, communities and society. 

In targeted services (outreach youth work, youth workshops and the Nuotta social reinforcement training for young people), the study focused on defining the form of the work and developing a multiple constituency evaluation model. The methods employed were a questionnaire survey directed at operators and a dialogue-based development process. In the internal specification of the form of the work, the way forward is seen as a recasting and reorganising of the collaborative opportunities in standard and specialised youth work. This is highlighted as a youth work goal uniting the entire sector and replacing the notion of keeping different forms of work separate. Evaluation of the impact and effectiveness of targeted services can be furthered by applying the multiple constituency evaluation method to the process evaluation of a young person. With this method, the process evaluation criteria should take account of the need to be youth-oriented, the relationship between the process and the number of young people and the local availability of youth services, and the role of multidisciplinary cooperation as part of the learning and operating environments of targeted youth work. Alongside process evaluation, the application of multiple constituency evaluation to the targeted services should take account of the young person’s previous service experience and the realisation of service needs, and subsequent process evaluation. 

The section of the study that deals with definition of the youth affairs sector draws attention to the shortcomings of the sector’s current definitions and presents a new analysis of the sector put together jointly by the researchers. The proposed new sectoral analysis draws on the extensive material collected in the network-based process and from youth work research. The new analysis model is proposed for wide use as a basis for evaluation of the youth affairs sector. 

Keywords: youth work, youth affairs sector, upbringing, public services, effectiveness, quality, accessibility, evaluation, reporting, development, identifiers, performance management, interactive evaluation, client satisfaction, equality, unrestricted access, inclusion