Report on the Consultation on changes to the College Performance Indicators

4. The following table summarises the changes we will implement to the publication:

[1] The Credit Guidance for Colleges 2024-25 states that the threshold for full-time students on courses lasting for over 20 weeks is being active after 5 weeks from the course start date. For shorter courses the threshold is after one quarter of the course. Students leaving before these thresholds will be early withdrawals.

5. This was a public consultation in which stakeholders were invited to share their feedback on six proposals related to the published statistics.
6. Respondents from colleges were also invited to share their feedback on a further five questions related to their use of the published statistics, to help inform broader future developments to collection of data, analysis of college success and the publication.
7. All responses were received via the online form provided on the SFC website.
8. We have reviewed the responses we received to the consultation – a statistical analysis was not necessary because of the small numbers of responses. This document provides a summary of the responses received and confirms the decisions made by SFC about the future publication of the statistics.

Summary of responses and general feedback received

9. Eight responses to the consultation were received, of which six were from representatives of colleges, one from EAUC Scotland and one from Scottish Government.
10. One respondent highlighted that the title ‘College Performance Indicators’ could mislead readers as the publication focuses almost exclusively on student retention, with additional information on credit delivery and on staff qualifications.
11. Having considered this feedback, we have decided to make the following changes to the official statistics:
    • The data on credit delivery will be published in the College Statistics publication from the next release, which will be published in 2025.
    • The data on staff qualifications will be published in the College Staffing publication from the next release, which will be published in 2025.
    • The College Performance Indicators publication will be renamed the College Student Outcomes publication from the next release, which will be published in 2025.

Feedback on proposal A: Early withdrawals calculation

12. All respondents agreed with this proposal.
13. All respondents provided commentary along with their answer, the themes of which were in support of the proposal:
    • Separating early withdrawals provides a more appropriate measure of success, focusing on the success of students who’ve committed to courses.
    • The proposed change improves ability to compare outcomes across the UK and with other types of institutions including universities.
    • Continuing to report both early withdrawals will help to ensure a more complete picture of college performance.
14. One respondent proposed inclusion of commentary on the use of terminology such as ‘student retention’ versus ‘withdrawal’ while another asked us to ensure that the differences with university continuation metrics be clear. We will consider how we can ensure clarity in our publications and communications, though we note that these terms are used across the education sector in the UK by diverse organisations, employing definitions that differ based on context.
15. One respondent expressed hope that changes would be suitable narrated and special care taken to understand changes in the time series due to the threshold date change for early withdrawal. We will ensure these points are taken into consideration when preparing the next publication.
16. One respondent highlighted the need to raise awareness that the increase in the
    non-continuation metric versus the previous metric will be due to the change of definition. We will ensure that the new publication strongly emphasises that the relevant figures are not comparable to those published in previous versions due to the definition changes, and we will provide trend data wherever possible to aid understanding.

Feedback on proposal B: Withdrawals reasons reporting

17. All respondents agreed with this proposal.
18. Some respondents provided commentary along with their answer, the key points of which were:
    • Keen interest in reporting additional information on reasons for withdrawal, and especially where there is data on students withdrawing to take up employment.
    • Some respondents expressed a wish for students withdrawing to enter employment to be included as a successful outcome in the performance indicators. We outlined our position on this topic in the consultation document and the consultation responses provided no further information or evidence which would justify any immediate reconsideration of this issue.
    • Some respondents also indicated an ongoing interest in another future review of the metric, revisiting inclusion of outcomes such as transfers to other courses or taking up a place at university. We will keep the calculation of the performance indicators under regular consideration, particularly if and when new data collection and/or analysis methods become possible.

Feedback on proposal C: Contextual information

19. All respondents agreed with this proposal.
20. Some respondents provided commentary along with their answer, the key points of which were:
    • Highlighting the potential benefits of contextual information for informing reflection and understanding of the quantitative success metrics.
    • An interest in colleges continuing to receive a full journey success metric. We will explore the feasibility of continuing to provide this calculation to colleges, while the published metric will separate early withdrawals from non-completion.
    • An interest in additional detail about the alternative study choices of students withdrawing to take up another course, for example the SCQF level and institution. We will explore the feasibility of provision of this type of information.
    • An interest in additional contextual information for students on school like courses. We will explore the feasibility of provision of this type of information.
21. A wish to ensure there is recognition of the broader benefits of study at college, which may still have been realised for students who did not complete their studies. We will consider how to ensure this is recognised by stakeholders.
22. A request for SFC to review the funding of students who withdraw, start late or never attend their course, particularly mature students for whom recruitment and support costs may have been higher. Our credit guidance states that colleges can claim credits for full-time students on courses lasting over 20 weeks if they are still active after 5 weeks from the course start date. For shorter courses the required date is after one quarter of the course has been delivered. We will share the information provided in the consultation response with relevant colleagues within SFC for further consideration.

Collection of withdrawal data for internal operational or monitoring purposes

23. Several respondents mentioned their collection of more detailed data on the reasons for student withdrawal and their onward outcomes. We will work with the sector to identify good practice which could be implemented more widely. We will also reinstate a dialogue with the sector through the Student Advisory Group for Further Education (SAGE) to explore potential improvements to the information collected on reasons for withdrawal and outcomes which could be reported.
24. Most respondents mentioned that they use individual learner progress data to inform student development and support, as well as curriculum evaluation and enhancement. We support the use of individual data to improve student outcomes.

Contextual information which may help the public to understand withdrawal

25. Respondents gave quite diverse responses to this question, proposing that the publication could include:
    • Information on the work at colleges to support students and minimise withdrawals.
    • Information on progress reviews as part of evaluation of curriculum processes.
    • Analysis by demographic and course type and SCQF level
26. We will consider options for incorporating these types of information in the publication.

Current use of the performance indicator

27. Respondents use the performance indicator data for diverse internal purposes including:
    • For marketing and promotional information about courses, as well as to inform recruitment and selection and preparedness of learners.
    • To inform course reflection and evaluation, including course design and balance of assessments.
    • To determine the distribution of pastoral and academic support and discretionary funding.
28. We will continue working with colleges to ensure the performance indicators are used effectively to improve student outcomes.

Expected differential use of early withdrawal versus non-completion

29. Some respondents indicated that they already separate analysis of the early withdrawals and non-completion internally.
30. Three respondents gave some specific examples of how the early withdrawals and
    non-completion data might be considered differently:
    • Early withdrawals relate primarily to recruitment, induction and preparedness and the data can be used for review of recruitment processes and partnerships and effectiveness of marketing materials and induction.
    • Non-completion has different underlying reasons and needs to be related to feedback, engagement and progress data to identify enhancement action to improve outcomes.
31. We will continue working with colleges to ensure that the performance indicators are used effectively to improve student outcomes.