The Research Excellence Framework (REF) is a peer review assessment of the quality of research in UK higher education institutions (HEIs). The review takes place every five to seven years. It is undertaken by the government's Higher Education Funding Councils and is used to:
- Provide accountability for public investment in research and produce evidence of the benefits of this investment.
- Provide the benchmarking information and establish reputational yardsticks.
- Inform the selective allocation of funding for research.
The last REF was REF 2014. All the submissions and results for REF 2014 remain available to view. The next REF exercise will be REF 2021.
For enquiries about REF at Solent contact firstname.lastname@example.org.
How the REF works
The REF assesses research excellence through a process of expert review. The reviews are undertaken by panels of experts appointed to a Unit of Assessment (UoA). There are 34 UoAs, which are split between four main panels (A-D).
Each expert panel will review all of the outputs, impact and environment for the HEI in that UoA and will produce a quality profile. The overall quality profile of the HEI will reflect all of these UoA level assessments.
The results are published at sub-panel (UoA) and main panel (A-D) level. The UoA profiles are reported as a percentage of the total submitted research activity deemed to have met each of the defined levels of quality. The overall profile comprises an aggregate of the weighted profiles produced for research outputs, research impact and research environment.
In a change to REF 2014 the weighting of the 3 different elements of the assessment exercise will be:
- Outputs (60%)
- Impact (25%)
- Environment (15%)
|Level ||Definition |
||World-leading in terms of originality, significance and rigour.
||Internationally excellent in terms of originality, significance and rigour, but which falls short of the highest standards of excellence.
||Recognised internationally in terms of originality, significance and rigour.
||Recognised nationally in terms of originality, significance and rigour.
| Unclassified (u/c)
||Falls below the standard of nationally recognised work. Or work which does not meet the published definition of research for the purposes of the assessment
Only 3* and 4* results contribute to the allocation of QR funding.
Timetable for REF 2021
View the REF timetable
1 January 2000 - 31 December 2020
Underpinning research for impact (minimum 2*)
1 August 2013 - 31 July 2020
Assessment period for impact
1 August 2013 - 31 July 2020
Assessment period for environment data
1 January 2014 - 31 December 2020
Assessment period for outputs
Publication of final UoA panel guidance and criteria
HEI codes of practice approved by Research England
31 July 2020
Staff census date
Closing date for submissions
Publication of outcomes
Publication of submissions and reports
The REF assesses research excellence through a process of expert review, informed by metrics where appropriate. HEIs submit evidence of their research outputs, research impact and research environment to UoAs, which are discipline-based expert panels who conduct the review.
The role of metrics in research assessment
Research England conducted an independent review to ascertain the extent to which metrics could be used in the assessment and management of research. This was reported in The Metric Tide report which concluded that metrics are not yet robust enough to replace peer review. The consultation proposes that metrics are used to inform peer review in some UoAs (similar to REF 2014). The Forum for Responsible Metrics, established in 2016, produced advice in 2017 for Research England (and the other HE funding bodies) on the use of quantitative indicators in the assessment of outputs in REF 2021 (with further discussion planned later for assessment of impact and environments).
The Forum defines responsible metrics as using the following principles:
- Robustness - basing metrics on the best possible data in terms of accuracy and scope.
- Humility - recognising that quantitative evaluation should support, but not supplant, qualitative, expert assessment.
- Transparency - that those being evaluated can test and verify the results.
- Diversity - accounting for variation by research field, and using a range of indicators to reflect and support a plurality of research and researcher career paths across the system.
- Reflexivity - recognising and anticipating the systemic and potential effects of indicators, and updating them in response.
Use of citation information
Previously the following UoAs used citation information as an indicator in addition to expert peer review
Main panel A - All
Main panel B - UoAs 7, 8, 9 and 11
Main panel C - UoAs 14 and 16
Main panel D - None
Peer review of research outputs will contribute to 60 per cent of the HEIs' overall REF outcome.
For a research output to be eligible for submission to the REF it should be considered against criteria such as:
- it is a product of research: a process of investigation leading to new insights, effectively shared;
- it was first made publicly available during the REF publication period (1 January 2014 - 31 December 2020);
- it is authored/co-authored by a member of staff at the HEI on the submission census date;
- it is a journal article, conference contribution, book, book chapter, physical artefact, exhibition, performance, digital artefact, web content.
In all UoAs individual staff may submit between one and five outputs, however, the overall submission must be an average of 2.5 outputs per FTE.
For example, where UoA submitted staff = 20, UoA submitted outputs = 50, and
A maximum of five outputs may be attributed to individual staff members (including those who have left)
Data on the distribution of outputs across staff in the UoA, including staff who have left, will be provided to the sub-panels for consideration in relation to the assessment of the environment
Output portability: A transitional approach is being adopted whereby outputs may be submitted by both the institution employing the staff member of the census date and the originating institution where the staff member was previously employed when the output was demonstrably generated. 'Demonstrably generated' will be determined by the date when the output was first made publicly available. This applies to the whole REF 2021 period.
Open access: The REF Open Access policy requires outputs within the scope of the policy (journal articles and some conference proceedings) to be deposited in an open access repository as soon after the point of acceptance for publication as possible, and no later than 3 months after this date from 1 April 2018. Due to concerns around deposit on acceptance, a deposit exemption will be introduced from 1 April 2018 and remain in place for the rest of the REF2021 publication period. The will allow outputs unable to meet this deposit timescale to remain compliant if they are deposited up to 3 months after the date of publication.
Peer review, including review by research end-users, will contribute 25 per cent to the HEIs' overall REF outcome.
For a UoA to be eligible for submission to REF, it must have a minimum of two impact case studies. For every 15 staff added to the first 15 staff submitted a further impact case study must be submitted.
For a research impact case study to be eligible for submission to the REF it should be considered against criteria such as:
- the underpinning research was produced in the submitting unit between 1 January 2000 and 31 December 2020;
- the impact took place between 1 August 2013 and 31 July 2020;
- the underpinning research is assessed as at least 2* in the excellence framework;
- the impact case study demonstrates 'reach and significance' supported by evidence.
A template for case studies will be issued and will include mandatory fields to be completed including the following:
- a title,
- summary of the impact,
- description of the underpinning research including key research insights which underpinned the impact, including reference to where the research was first made publicly available,
- details of the impact including how the underpinning research made a distinct and material contribution to the impact, and the nature and extent of the impact,
- documented evidence of the above.
Criteria and definitions of the assessment of impact
| Level||Definition |
||Outstanding in terms of reach and significance.
||Very considerable in terms of reach and significance.
||Considerable in terms of reach and significance.
||Recognised in terms of reach and significance.
||Little or no reach and significance; or not eligible; or not underpinned by excellent research produced by the submitting unit.
Impact in REF 2014 was defined as 'an effect on, change or benefit to the economy, society, culture, public policy or services, health, the environment or quality of life, beyond academia. Impact included, but was not limited to, an effect on, change or benefit to:
- the activity, attitude, awareness, behaviour, capacity, opportunity, performance, policy, practice or understanding;
- of an audience, beneficiary, community, constituency, organisation or individuals;
- in any geographic location whether locally, regionally, nationally or internationally'.
Impact included the reduction or prevention of harm, risk, cost or other negative effects.
Further guidance will be issued by Research England on inclusion in REF 2021 of impacts on research or the advancement of academic knowledge within the higher education sector nationally or internationally, and impacts on teaching or other activities with the submitting HEI.
Peer review of the research environment will contribute 15 per cent to the HEIs' overall REF outcome.
For research environment to be eligible for submission to the REF it should be considered against criteria such as:
- Research degrees awarded - number of doctoral degrees awarded in each academic year of the assessment period to students supervised within the submitting unit (taken from HESA data).
- Research income - total external research income (grants and contracts) in each academic year of the assessment period in the submitting unit (taken from the Financial Statistics Return (FSR) returned to HESA).
- Research income-in-kind - estimated value of Research Council facility time allocated through peer review and used by staff in submitting units (data provided by HEFCE).
A template will be issued and will include mandatory fields to be completed including the following:
- research strategy
- staffing strategy and staff development
- research students
- income, infrastructure and facilities
- collaboration and contribution to the discipline
Criteria and definitions of the assessment of the environment
|Level ||Definition |
||Conducive to producing research of world-leading quality in terms of its vitality and sustainability.
||Conducive to producing research of internationally excellent quality in terms of its vitality and sustainability.
||Conducive to producing research of internationally recognised quality in terms of its vitality and sustainability.
||Conducive to producing research of nationally recognised quality in terms of its vitality and sustainability.
||An environment that is not conducive to producing research of nationally recognised quality.
Working with the REF 2021 main panels, Research England will provide further guidance on identifying staff with significant responsibility for research. This will be published in the guidance on submissions and panel criteria. This guidance will not prescribe a fixed set of criteria that all staff would be required to meet, but will set out a 'menu' of what Research England would consider may be appropriate indicators of 'significant responsibility' for research.
Recognising that there are staff who have more significant responsibility for other activities, Research England will implement an approach whereby institutions, working with their staff and with guidelines, identify who is in scope for submission among staff meeting core eligibility criteria. Research England has defined the core eligibility criteria as
Category A eligible:
- Academic staff with a contract of employment of no less than 0.2 FTE.
- Employed by the submitting institution on the census date (31 July 2020).
- Whose primary employment function is to undertake either 'research only' (independent researchers only) or 'teaching and research'.
- Have a substantive connection to the submitting institution.
- For staff on 'research only' contracts, the eligible pool should only include those who are independent researchers, not research assistants.
Category A submitted:
- Describes the staff from the 'Category A eligible' pool who have been identified as having 'significant responsibility for research' on the census date.
Where the 'Category A eligible' staff definition accurately identifies all staff in the submitting unit with a significant responsibility for research, the unit should submit 100 per cent of staff. Where it does not accurately identify all staff in the submitting unit who have significant responsibility for research, institutions will need to implement processes to determine this and document this in a code of practice, approved by HEFCE with advice from the Equality and Diversity Advisory Panel (EDAP).
- Staff who have left the HEI by 31 July 2020 will not be eligible to be returned in the UoA, except where outputs were 'demonstrably generated' at the submitting HEI.
- QR funding is allocated on the basis of the 'assessed quality' of the submissions (research outputs, impact and environment) and the submitted volume (FTE) of submitted academic staff who were in post on the census date and were eligible for submission based on the HEIs' Code of Practice.
- Each HEI will decide which staff to submit in order to maximise both the quality (which includes the assessment of their research outputs) and the resultant funding (linked to submitted staff volume). The peer review panels do not assess the quality of individual staff.
The Code of Practice is drawn up at each HEI in accordance with guidance issued by HEFCE. The Code of Practice will set out the criteria for eligibility of staff for submission to the REF (staff volume) and the selection process for outputs. Each HEI's Code of Practice will be approved by HEFCE before implementation, to ensure decisions are justifiable and non-discriminatory
What is submitted in a UoA?
Five sections are submitted:
- Staff data
- Research outputs
- Impact case studies
- Environment data
- Environment narrative
How is the REF assessed?
Academic experts are appointed to REF UoA sub-panels and review all the research submissions. Research users also sit on panels and provide input into the assessment of impact case studies.
Back to top
What is the role of the main panel?
There are four main panels (A-D) who comprise a group of expert researchers representing a broad discipline area whose role is to lead and oversee the conduct of the sub-panels under their remit. They are responsible for setting the assessment criteria, working methods and consistency of approach and adherence to criteria across the sub-panels.