Feedback and evaluation
The evaluation of pilot activities was a crucial part of the Incluso project and SHMU, the Scottish pilot partner based in Aberdeen, conducted evaluations in several different ways. But it should be pointed out that in retrospect, the Incluso partners from SHMU felt the Big Picture outcomes were too big, and not detailed or focused enough. Having said that, we found strong evidence, from young people and staff, which support the Incluso objectives, and that is included into the recommendations in various chapters of the Whitebook.
Gary Dawson, senior youth worker at the Tilly Youth Project, said: "We have no choice now so we have to use social media in youth work. This is the way everything is going. If we don’t join now, we’ll have to do so in the future. I think we need to be proactive about this. I’m convinced about all this. It didn’t take much persuasion on my part."
We used the following means to gather evidence:
- We counted activities (comments etc) on the sites we created
- We made assessments of individual participation using a scorecard
- We made our own anecdotal observations of activity
Other methods used for measurement, included:
- Completion of questionnaires (prepared by partners at the Verwey Jonker Institute)
- Participation in a focus group
While we gathered our evidence and carried out our measurements as best as we could it was often difficult to do so for a number of reasons. Some young people simply refuse to engage and will walk away from discussion, refuse to speak or participate and sometimes become hostile (with outbursts of swearing and shouting) when pressed. Other young people say they will comply and participate, but never do and when asked they say they forgot or say they just haven't gotten around to the task (this can go on for months). Others will begin a task, but never finish it or find a short cut, or not complete it properly.
So, many young people refused to complete the online questionnaires (for example, not one person at the Middlefield Youth Flat completed the questionnaires. Our pilot at Tilly Youth Project started too late, but we believe that young people there would have been too 'independent' to participate). Notice how these were external projects, ie. not based in SHMU premises or run by SHMU staff. We found it was much easier to gather evidence from young people particpating in projects run by SHMU on SHMU premises.
We believe young people participating in SHMU projects on SHMU premises are likely to have better relationships with the staff who administer these questionnaires (staff who are themselves more motivated by the objectives of the Incluso project) and therefore are more likely to engage and participate. As well as this, the young people who engage with SHMU do so because they are motivated to participate in the radio station or other activity. SHMU is not a youth or drop-in centre and so there is an increased level of committment from the young people for the tasks they are asked to participate in.