Invicta National Academy has blazed a trail in remote teaching, our teaching team boasts some of the UK's most qualified and experienced teachers. To enable parents and carers to book our teachers for 1-2-1 tuition, we have created a booking system. We are charging an introductory registration fee of £25, which gives you access to all our teachers and their expertise.
Invicta National Academy has blazed a trail in remote teaching, our teaching team boasts some of the UK's most qualified and experienced teachers. To enable parents and carers to book our teachers for 1-2-1 tuition, we have created a booking system. We are charging an introductory registration fee of £25, which gives you access to all our teachers and their expertise.
Impact Report 2021
Overview
During the summer of 2021, the Invicta Summer Academy Programme delivered free and interactive, online catch-up lessons for pupils aged 6 to 16 from KS1 to KS4. This report summarises the findings from an evaluation of this programme from lower KS2 to KS4, conducted in partnership with ImpactEd.
Short baseline and final tests were used to assess the change in academic achievement in mathematics and English, although these were not as in depth as standardised academic tests. Baseline and final validated surveys were used to assess changes in participants’ belief in their ability to do well in these subjects (self-efficacy) and their ability to persevere in the pursuit of long-term goals (grit). Statistical significance testing was conducted using t-tests, which use the average and spread of two groups of values (baseline and final scores) to determine if the mean of the groups are significantly different (i.e., likely to reflect a true difference as opposed to a chance observation).
Context
The programme was carried out during the Covid-19 pandemic which had major negative consequences on student learning. In addition, the programme took place over the summer holiday, during which students can experience learning losses due to lack of tuition, unavailability of free school meals and/or a safe and comfortable living space. Learning losses are evidenced for reading and writing within the UK. Furthermore, a recent US study of 6-13 year olds found that the average student lost 17–34% of the prior year’s learning gains during their summer break (although UK summer holidays are approximately 50% shorter than in the US so we might not expect the same magnitude of learning loss in the UK).
Impact
1
There was a statistically significant change in participants’ academic test scores, with
participants scoring 6.8 percentage points higher following the programme.
2
There was a statistically significant change in participants’ test scores in both subjects, with
participants scoring 5.2 and 8.6 percentage points higher for mathematics and English,
respectively, following the programme
3
There was an increase in participants’ test scores following the programme across all keystage-
subject combinations.
4
There were no statistically significant changes in self-efficacy or grit when considering only
participants who completed the surveys both before and after the programme.
5
There was a statistically significant increase in both self-efficacy and grit when considering all
participants (i.e., those who completed the surveys before and/or after the programme), but
this may be influenced by bias in the pool of respondents.
Methodolgy
Research questions
The evaluation approach has been designed to answer the following questions:
Does taking part in the Invicta National Academy Summer Programme increase:
â–º Academic performance in mathematics and English?
â–º Self-efficacy (a pupils’ belief in their ability to do well) in mathematics and English?
â–º Grit (a pupils’ ability to persevere in the pursuit of long-term goals)?
Evaluation design and analysis
Participants in the programme were asked to complete short questionnaires, including:
â–º Baseline and final subject-specific assessments for each subject they participated in.
â–º Baseline and final validated surveys to assess grit and self-efficacy.
Reach
Participants came from across the UK, but were predominantly from the South East of England. Many came from areas near to poor and underperforming schools.
Impact on Attainment
Key finding 1: There was a statistically significant
change in participants’ test scores, with
participants scoring 6.8 percentage points higher
following the programme.
When combining the results across all subjects and keystages, the mean academic test score increased by 6.8 percentage points from 72.4% before the programme to 79.2% after the programme. This change is statistically significant (paired t-test, p < 0.01). This means that, on average, participants answered a further 6.8% (approximately 1 in 15) of the questions
correctly following the programme.
Key finding 2: There was a statistically significant
change in participants’ test scores in both
subjects, with participants scoring 5.2 and 8.6
percentage points higher for mathematics and
English, respectively, following the programme.
When considering only participants in the mathematics lessons, the mean mathematics test score increased by 5.2 percentage points from 74.9% before the programme to 80.1% after the programme. This change is statistically significant (paired t-test, p < 0.01). This means that, on average, participants answered a further 5.2% (approximately 1 in 20) of the questions correctly following the programme.
When considering only participants in the English lessons, the mean English test score increased by 8.6 percentage points from 69.5% before the programme to 78.1% after the programme. This change is statistically significant (paired t-test, p < 0.01). This means that, on average, participants answered a further 8.6% (approximately 1 in 12) of the questions correctly following the programme.
Key finding 3: There was an increase in
participants’ test scores following the programme
across all keystage-subject combinations.
The mean test score increased for all keystage-subject combinations, with the greatest change seen for KS4 English (+13.9 percentage points). This demonstrates that the participants scored more highly in every keystage and every subject, although some keystage/subject combinations saw greater improvements than others.
Table: Change in test score by subject and keystage (where % indicates percentage points)
Participants responded differently to the programme, with some seeing large increases in assessment score whilst others saw more modest increases or decreases. More pupils saw increases than decreases, and this was true across every subject/keystage combination.
Figure: The frequency of each change in test score by subject and keystage, where all bars to the right of the
dotted line represent participants with a score increase. LKS2 = lower KS2, UKS2 = upper KS2.
Key finding 4: There were no statistically
significant changes in self-efficacy or grit when
considering only participants who completed the
associated surveys both before and after the
programme.
Self-efficacy is a measure of pupils' belief in their ability to achieve a specific task in the future. Self-efficacy is linked to higher academic achievement and persistence and contributes to pupil wellbeing. Participants were asked to complete a short, validated questionnaire for each of English and mathematics to assess their self-efficacy in these subjects. Self-efficacy is scored on a scale ranging from 1 (very low) to 7 (very high).
â–º Self-efficacy in English increased from 5.5 before the programme to 5.8 after the programme for English lesson participants, although this change was not statistically significant (paired t-test, p = 0.19).
â–º Self-efficacy in mathematics marginally decreased from 5.75 before the programme to 5.72 after the programme for mathematics lesson participants, although this negligible change was not statistically significant (paired t-test, p = 0.56).
Grit is a measure of pupils’ perseverance and passion for long-term goals. It is closely related to academic and social resilience. Pupils with high levels of grit may work persistently over years to achieve goals despite setbacks. Grit is scored on a scale ranging from 1 (very low) to 5 (very high).
â–º Participants’ grit increased from 2.4 before the programme to 2.5 after the programme, although this negligible change was not statistically significant (paired t-test, p = 0.96).
These findings suggest that the programme does not increase participants’ perseverance towards long term goals or belief in their ability to succeed in English or mathematics; however, there was a limited pool of 22 young people who responded to both the baseline and final self- efficacy and grit questionnaires. This means that, even if the programme was effective at
increasing participants’ self-efficacy and grit, we might not see a statistically significant change due to low statistical power.
Key finding 5: There was a statistically significant increase in both self-efficacy and grit when
considering all participants (i.e., those who
completed the associated surveys before and/or
after the programme).
â–º Self-efficacy in English increased from an average of 5.0 before the programme to 5.7 after the programme for English lesson participants. This change was statistically significant (unpaired t-test, p-value < 0.01).
â–º Self-efficacy in mathematics increased from an average of 5.2 before the programme to 5.8 after the programme for mathematics lesson participants. This change was statistically significant (unpaired t-test, p-value < 0.01).
â–º Participants’ grit increased from an average of 2.2 before the programme to 2.3 after the programme, and this change was statistically significant (unpaired t-test, p < 0.05).
Contrary to the previous finding, these findings suggest that the programme could increase participants’ perseverance towards long term goals or belief in their ability to succeed in English or mathematics. However, this analysis is less robust that the analysis of matched responses in Key Finding 4. This is because there is a risk that the improvements seen were due to inherent differences in the pool of respondents. For example, it could be that those participants who felt very positively about the programme were more likely to reply following the programme, which could result in a positive bias in post-programme questionnaire scores. As such, we do not know how much the improvements in these outcomes can be attributed to the programme.nificant change due to low statistical power.