Research Excellence Framework

The Research Excellence Framework (REF) is the system for assessing the quality of research in UK higher education institutions (HEIs). The review takes place every five to seven years. It is undertaken by the Government body, Research England and is used to:

Provide accountability for public investment in research and produce evidence of the benefits of this investment.
Provide the benchmarking information and establish reputational yardsticks.
Inform the selective allocation of funding for research.

For REF 2021, Solent University has returned 57 staff across three Units of Assessment (UoA). The submission reflects our areas of research strength and expertise. Building on our REF 2014 submission, this is the foundation from which we will build towards the next REF.

Find out about our research themes

For enquiries about REF at Solent, please contact ref@solent.ac.uk.

Solent University Code of Practice

As part of the preparations for REF 2021, all UK universities submitted a Code of Practice (CoP) to the REF Equality and Diversity Advisory Panel (EDAP) for approval. The CoP outlined the process for selection of staff and research outputs. It applied to academic staff on research only and teaching and research contracts.

The Code of Practice was informed by the REF 2021 Guidance on Submissions.

Following a period of consultation (21 February - 29 March 2019), the University submitted the Solent University draft Code of practice to EDAP, which they approved in November 2019.

View the approved Code of Practice

How REF 2021 worked

The REF assesses research excellence through a process of expert review. The reviews are undertaken by panels of experts appointed to a Unit of Assessment (UoA). There are 34 UoAs, which are split between four main panels (A-D).

Each expert panel reviews all the outputs, impact, and environment information for the HEI in that UoA and produces a quality profile. The overall quality profile of the HEI reflects all these UoA level assessments.

The results will be published at sub-panel (UoA) and main panel (A-D) level. The UoA profiles are reported as a percentage of the total submitted research activity deemed to have met each of the defined levels of quality. The overall profile comprises an aggregate of the weighted profiles produced for research outputs, research impact and research environment.

In a change to REF 2014 the weighting of the three different elements of the assessment exercise in REF 2021 was:

Outputs (60%)
Impact (25%)
Environment (15%)

Overall quality star rating profile

Level	Definition
4*	Quality that is world-leading in terms of originality, significance and rigour.
3*	Quality that is internationally excellent in terms of originality, significance and rigour, but which falls short of the highest standards of excellence.
2*	Quality that is recognised internationally in terms of originality, significance and rigour.
1*	Quality that is recognised nationally in terms of originality, significance and rigour.
Unclassified (u/c)	Quality that falls below the standard of nationally recognised work. Or work which does not meet the published definition of research for the purposes of the assessment

Only 3* and 4* results contribute to the allocation of quality-related QR funding.

Open access

The REF Open Access policy requires outputs within the scope of the policy (journal articles and conference proceedings with an ISSN number) to be deposited in an open access repository as soon after the point of acceptance for publication as possible, and no later than three months after this date.

For further information about Open Access at Solent, please find some information here, see our open access policy (which is currently under review), or contact our research support librarian.

For REF 2021, peer review of research outputs contributes to 60% of the HEIs' overall REF outcome.

For a research output to be eligible for submission to the REF it should be considered against criteria such as:

it is a product of research: a process of investigation leading to new insights, effectively shared;
it was first made publicly available during the REF publication period (for REF 2021 this was 1 January 2014 - 31 December 2020);
it is authored/co-authored by a member of staff at the HEI on the submission census date;
it is a journal article, published conference contribution, book, book chapter, physical artefact, exhibition, performance, digital artefact, web content.

In all UoAs, individual staff may submit between one and five outputs, however, the overall submission must be an average of 2.5 outputs per FTE.

For example, where UoA submitted staff = 20, UoA submitted outputs = 50, and:

A minimum of one output will be required for each eligible staff member employed on the census date
A maximum of five outputs may be attributed to individual staff members (including those who have left)
Data on the distribution of outputs across staff in the UoA, including staff who have left, will be provided to the sub-panels for consideration in relation to the assessment of the environment

Output portability:  A transitional approach was adopted whereby outputs were submitted by both the institution employing the staff member of the census date and the originating institution where the staff member was previously employed when the output was demonstrably generated. 'Demonstrably generated' was determined by the date when the output was first made publicly available. This applied to the whole REF 2021 period.

Research outputs at Solent

Solent University is committed to making as much of our research open access as possible. More information about open access publishing is available on the Solent researchers' Libguide and in the Solent Open Access Policy (which is currently under review).

The Solent REF 2021 submission was extracted from the Pure research information management system. Following that submission, the way that publications and other research outputs are added to Pure is changing. From January 2022, whenever you submit work for publication, please inform the research support librarian by using the form available on the Solent Researchers' Libguide.

Please remember that we always need the date that submissions are accepted, as well as the date of publication. If the publication is not in an open access journal or book, we also need a copy of the author accepted manuscript (AAM), that is the text that was submitted for publication after any changes made as a result of peer review, but not the publisher's PDF with their formatting.

You will no longer be able to add entries to Pure yourself or amend existing ones. Any changes must be requested by emailing pure@solent.ac.uk, or by contacting the research support librarian.

This change is being made in order to reduce the administrative burden on research staff and students; ensure a more consistent approach to recording research outputs, improving data integrity; and improve our open access record.

Academics are encouraged to register for an ORCID for inclusion on their Pure profile.

Research impact

For the purposes of REF 2021, impact was defined as ‘an effect on, change or benefit to the economy, society, culture, public policy or services, health, the environment or quality of life, beyond academia. Impact includes, but is not limited to, an effect on, change or benefit to:

the activity, attitude, awareness, behaviour, capacity, opportunity, performance, policy, practice, process or understanding
of an audience, beneficiary, community, constituency, organisation or individuals
in any geographic location whether locally, regionally, nationally or internationally.

Impact includes the reduction or prevention of harm, risk, cost or other negative effects.

For a UoA to be eligible for submission to REF, it must have a minimum of two impact case studies. For every 15 staff added to the first 15 staff submitted in a UoA, a further impact case study must be submitted.

For a research impact case study to be eligible for submission to REF 2021 it had to meet the following criteria:

the underpinning research was produced in the submitting unit between 1 January 2000 and 31 December 2020;
the impact took place between 1 August 2013 and 31 December 2020;
the underpinning research is assessed as at least 2* in the excellence framework;
the impact case study demonstrates 'reach and significance' supported by evidence.

Research environment

For REF 2021, the research environment is considered against criteria such as:

Research degrees awarded - number of doctoral degrees awarded in each academic year of the assessment period to students supervised within the submitting unit (taken from HESA data).
Research income - total external research income (grants and contracts) in each academic year of the assessment period in the submitting unit of assessment (taken from the Financial Statistics Return (FSR) returned to HESA).
Research income-in-kind - estimated value of Research Council facility time allocated through peer review and used by staff in submitting units (data provided by Research England).

Assessment

The REF assesses research excellence through a process of expert review, informed by metrics where appropriate. HEIs submit evidence of their research outputs, research impact and research environment to UoAs, which are discipline-based expert panels who conduct the review.

The role of metrics in research assessment

Research England conducted an independent review to ascertain the extent to which metrics could be used in the assessment and management of research. This was reported in The Metric Tide report which concluded that metrics are not yet robust enough to replace peer review. The consultation proposes that metrics are used to inform peer review in some UoAs (similar to REF 2014). The Forum for Responsible Metrics, established in 2016, produced advice in 2017 for Research England (and the other HE funding bodies) on the use of quantitative indicators in the assessment of outputs in REF 2021 (with further discussion planned later for assessment of impact and environments).

The Forum defines responsible metrics as using the following principles:

Robustness - basing metrics on the best possible data in terms of accuracy and scope.
Humility - recognising that quantitative evaluation should support, but not supplant, qualitative, expert assessment.
Transparency - that those being evaluated can test and verify the results.
Diversity - accounting for variation by research field, and using a range of indicators to reflect and support a plurality of research and researcher career paths across the system.
Reflexivity - recognising and anticipating the systemic and potential effects of indicators, and updating them in response.

Research output assessment criteria and definitions

Level	Definition
4*	Quality that is world-leading in terms of originality, significance and rigour.
3*	Quality that is internationally excellent in terms of originality, significance and rigour, but which falls short of the highest standards of excellence.
2*	Quality that is recognised internationally in terms of originality, significance and rigour.
1*	Quality that is recognised nationally in terms of originality, significance and rigour.
Unclassified (u/c)	Quality that falls below the standard of nationally recognised work. Or work which does not meet the published definition of research for the purposes of this assessment.

Impact case study assessment criteria and definitions

Level	Definition
4*	Outstanding in terms of reach and significance.
3*	Very considerable in terms of reach and significance.
2*	Considerable in terms of reach and significance.
1*	Recognised in terms of reach and significance
Unclassified (u/c)	Little or no reach and significance; or not eligible; or not underpinned by excellent research produced by the submitting unit.

Research environment assessment criteria and definitions

Level	Definition
4*	Conducive to producing research of world-leading quality in terms of its vitality and sustainability.
3*	Conducive to producing research of internationally excellent quality in terms of its vitality and sustainability
2*	Conducive to producing research of internationally recognised quality in terms of its vitality and sustainability.
1*	Conducive to producing research of nationally recognised quality in terms of its vitality and sustainability.
Unclassified (u/c)	An environment that is not conducive to producing research of nationally recognised quality.

Further information

REF 2021

REF 2014

Research England

Five sections are submitted:

Staff data
Research outputs
Impact case studies
Environment data
Environment narrative

Please email ref@solent.ac.uk.

Information can be found on the REF2014 website.

No. Publishing an article in a well-regarded journal is not a guarantee that the research output being published will meet the definitions and quality criteria for REF 2021.

Your information librarian can advise on subject specific journals and journal rankings.

An underpinning principle of the REF is that all forms of research output are assessed on a fair and equal basis. Sub-panels will not regard any particular form of output as of greater or lesser quality than another per se.

All research outputs must meet the definition of research for the REF. For the purposes of the REF, research is defined as ‘a process of investigation leading to new insights, effectively shared’.

All submissions should provide sufficient information to allow a sub-panel to understand the research process, the research insights, and the time and manner of dissemination. Often this will be evident within the materials submitted, requiring no further information. However, where this is not evident within the submission, this may be supplemented by an up to 300-word statement or supporting evidence.

The table below sets out categories of output types under which outputs could be submitted in REF 2021, the collection formats for the different output types, and a broad definition of each category. This includes examples, which are provided for guidance only and do not represent a definitive list.

Category	Upload to submission system	Physical output (deposit to REF warehouse)	Definition
(Parts of) Books
A - Authored book	PDF	Actual book	An authored book written entirely by a single author or by joint authors who share responsibility for the whole book. Includes: scholarly books research monographs textbooks based on significant research (as defined above) by the author(s) revisions / new editions of the above, providing this includes substantial new research material novels, plays and screenplays collections of plays, poems, short stories or other creative writing by the author(s).
B - Edited book	PDF	Actual book (if the edition is in multiple volumes, submit representative volume in the first instance)	A book or volume in which individual chapters or contributions have been written by different authors. To submit a work in this category, the editor must have had sole responsibility, or be identified as having made a substantial contribution to the editing, choices for inclusion, and underpinning process of investigation. Includes: edited books or volumes textbooks or encyclopaedias where significant background research is required annotated anthologies where research informs the annotations revisions or new editions of the above, providing this includes substantial new research material literary translations, where these contain significant editorial work in the nature of research.
C - Chapter in book	PDF upload of chapter and page(s) of the book that bear the title, publisher, editor, and publication date	Actual book or hard copy of chapter including a copy of the page(s) of the book that bear the title, publisher, editor and publication date	This category includes contributions to edited books. This may include scholarly work, such as: chapters in edited books entries in textbooks incorporating significant research content entries in scholarly editions entries in revisions or new editions, providing this includes substantial new research material translations where these contain significant editorial work which constitutes research.
R - Scholarly edition	If not available in print, pdf upload of short written description of the scholarly edition, including details of how it can be freely accessed (eg, URL, DOI)	Actual scholarly edition	An edition of another author's original work or body of works informed by critical evaluation of the sources (such as earlier manuscripts, texts, documents and letters) often with a scholarly introduction and explanatory notes or analysis on the text and/or original author. This may include a translation of the original text(s) where this constitutes part of the research.
Journal articles
D - Journal article	Submit with DOI: REF team to source. If REF team is unable to source then HEI to upload pdf of article / conference contribution.	n/a (if only hard copy is available the HEI should upload a scanned pdf)	A scholarly paper, usually on a specific topic, published in an externally circulated scholarly or professional journal that has an ISSN. This may include: full research articles critical scholarly texts which appear in article form review articles, where these meet the definition of research for the REF evidence synthesis, including systematic reviews, analyses, metaanalyses, metasyntheses, where these meet the definition of research for the REF rapid communication (short papers, usually published swiftly, in scholarly journals presenting original material discussion paper (short articles in scholarly journals that critically address specific results or data provided in a published research paper) creative article, including photographic essays.
E - Conference contribution	Submit with DOI: REF team to source. If REF team is unable to source then HEI to upload pdf of article/conference contribution.	n/a (if only hard copy is available the HEI should upload a scanned pdf)	A conference paper or other contribution published in conference proceedings. The conference proceedings will usually have an ISSN or ISBN and may be published in a number of formats, such as: volume of proceedings special or normal edition of a journal book or a monograph website. Submitted outputs may include: full written papers that appear in published conference proceedings other conference contributions which meet the definition of research.
U - Working paper	Submit with DOI: REF team to source. If REF team is unable to source then HEI to upload pdf of article / conference contribution.	n/a (if only hard copy is available the HEI should upload a scanned pdf)	Research papers disseminated to encourage discussion and suggestions for revision. This may be through pre-print dissemination, lodging in an institutional repository, or self-publication for distribution.
Physical artefacts
L - Artefact	PDF upload of photographic / visual record of output, or details of how it can be freely accessed (eg, URL, DOI)	Photographic / visual record of output (paper and/or DVD/CD/USB)	Artefacts, objects or craftworks, exhibited, commissioned or otherwise presented or offered in the public domain, eg, visual arts, craft, and cultural creations. This can include (but is not limited to): illustration sculpture media installations ceramics jewellery metalwork buildings cultural artefacts, such as large permanent public sculptures.
P - Devices and products	PDF upload of photographic / visual record of output, or details of how it can be freely accessed (eg, URL, DOI)	Photographic / visual record of output (paper and/or DVD/CD/USB)	An element, system or substance developed to perform a particular function, set, or combination of functions. Incorporates developing the concept and the design and development of any chemical, mechanical, electronic and software components and, where appropriate, the overall system architecture. use may be functional, aesthetic, or commercial may be physical, including chemical or compound, ie, medicines may include digital/virtual products for particular functions, ie, gaming, analysis, display may include services, ie, transportation, energy supply, public broadcasting, healthcare systems may be associated with the manufacturing, extraction and refinement of other devices.
Exhibitions and performances
M - Exhibition	PDF upload of photographic / visual record of output, or details of how it can be freely accessed (eg, URL, DOI) and evidence of year of dissemination	Representation of the output (eg, recording or photographic/visual record) and evidence of year of dissemination (paper and/or DVD/CD/USB)	A single or series of public events or short-term, long-term or permanent installations, at which works of interest are displayed. Submissions can be: solo exhibitions curation of exhibitions contributions to collaborative group exhibitions. Submissions may include: original artistic works and/or designs historical, political, social, technical / technological or scientific research and information works exhibited in non-standard environments curating an exhibition.
I - Performance	PDF upload of photographic / visual record of output, or details of how it can be freely accessed (eg, URL, DOI) and evidence of year of dissemination	Representation of the output (eg, recording or photographic / visual record) and evidence of year of dissemination (paper and/or DVD/CD/USB)	A live or recorded first performance (by, for example, an actor, musician, dancer, conductor, artist) to an external audience. The 'author' can have one (or more) of a variety of major roles (eg, lead performer, director, writer) in the production, which should meet the REF definition of research. The role should be specified within the additional details required, with details of other participants involved in the research. Includes (but is not limited to): performance of a play, musical, opera, concert, television or radio production, performance artwork theatre productions (stage play, mime, circus, puppet show, variety act, comedy show) concerts and recitals (music or dance) broadcast performances and other modes of presentation production of an audio/visual medium (such as CD or DVD recording) artistic direction of a staged production input into a theatre production (eg, design, dramatology).
Other documents
F - Patent / published patent application	PDF upload of published patent application / granted patent	Published patent application / granted patent (paper)	Granted patents, copyrights, trade marks, or registered designs on specific products or processes. Patents can have been granted in the UK or another patent-awarding country. The patent should have been granted for the first time during the assessment period.
J - Composition	Details of how audio recording (if available) can be freely accessed (eg, URL, DOI), and PDF upload of score and evidence of year of dissemination	Audio recording (if available) and score and evidence of year of dissemination (paper and/or DVD/CD/USB)	An original published/publicly available score, first performance or first recording by a record label of a musical composition. Can include (but is not limited to): compositions created while being played,eg, electronic compositions, jazz improvisation published / publicly available score recordings sound component of a film or video, lyrics, multimedia composition commissioned works combinations or developments of the above.
K - Design	PDF upload of photographic / visual record of design or details of how it can be freely accessed (eg, URL, DOI) and evidence of year of dissemination	Photographic / visual record of design and evidence of year of dissemination (paper and/or DVD/CD/USB)	A creative research / problem-solving output in the form of design drawings, books, models, exhibitions, websites, installations or built works. This can include (but is not limited to): fashion design textile design graphic design interior design industrial design architectural design multimedia design sound design exhibition design (ie, not the content of the exhibition) theatre design other designs.
N - Research report for external body	PDF upload of report or details of how it can be freely accessed (eg, URL, DOI) bearing year of publication / dissemination	Hard copy of report and evidence of year of receipt (eg, letter, email, delivery notice)	Non-confidential reports, commissioned and/or funded by an external organisation, including reports for private companies, government departments and non-governmental organisations.
O - Confidential report for external body	PDF upload of report and evidence of year of receipt (eg, letter, email, delivery notice)	Hard copy of report and evidence of year of receipt (eg, letter, email, delivery notice)	Confidential reports commissioned and/or funded by an external organisation, including reports for private companies, government departments and non-governmental organisations. For clarity, confidential material is not in scope of the open access requirements (see main text, paragraphs 213-214 for details of in-scope outputs).
Digital artefacts
G - Software	PDF upload of written description of the software and details of how the software, and if relevant, the source code can be accessed (eg, URL, DOI)	n/a	Originally researched, created and published software (computer programs and their associated documentation, consisting of a set of instructions written by a programmer) or database products of commercial quality, which has been made publicly available. May include (but is not limited to): operating systems utilities application programs interactive multimedia video games logic systems.
H - Website content	PDF upload of content as at date of publication (eg, a date certified electronic copy of content) or details of how it can be freely accessed (eg, URL, DOI)	Content as at date of publication, eg, a date certified electronic copy of content (DVD/CD/USB) or date-stamped printout of content (paper)	A collection of material which embodies research and is undertaken on a systematic basis specifically for dissemination through a website and/or as an interactive approach to allow users to engage directly with the process or products of the research. Web content is the textual, visual, or aural content encountered as part of the user experience on websites. It may include - among other things - text, images, sounds, videos and animations. May present factual information, analysis or data, or fictional, imaginative and/or creative work, using pictorial, video, audio, etc.
Q - Digital or visual media	Details of how it can be freely accessed (eg, URL, DOI) and evidence of year of dissemination	Either a copy of the published DVD, CD or other visual output; or for outputs that were broadcast, a digital or other visual copy of the content and evidence of year of dissemination	Research outputs presented in digitised and/or audio-visual format, such as: films documentaries audio-visual presentations computer games animation Encoded in digital format, machine readable and presenting and forms of communication not limited to verbal and text-based means.
S - Research data sets and databases	PDF upload of written description of the dataset or database and details of where it can be accessed (eg, URL, DOI) where relevant. Do not submit copies of actual datasets or databases	n/a	Submissions may include: Data sets: May come in a variety of formats, for instance in spreadsheets, but also any collection of data on which analysis can be performed. Most commonly a data set corresponds to the contents of a single database table, or a statistical data matrix, where every column of the table represents a particular variable, and each row corresponds to a given member of the data set. Databases: Collections of data specifically organised and presented for the ease of viewing, retrieval and analysis. May comprise multiple data setc. Often characterised by data field structuring and searchability tools.
Other
V - Translation	PDF upload of output or description of the output, or details of how it can be freely accessed (eg, URL, DOI)	The actual output (paper or USB)	A translation of a work or body of works by another author or authors, informed by critical evaluation of the sources (such as earlier manuscripts, texts, documents and letters), and by critical analysis of the work's original cultural context for the new readership. Translations may also include a scholarly introduction and explanatory notes or contextual analysis. Translation may enhance existing understanding of the material in question, and may provide evidence of creativity in its own right.
T - Other	PDF upload of representation of the output or details of how it can be freely accessed (eg, URL, DOI) and, if not clear from the output, evidence of year of dissemination	Either the actual output or a representation of the output; and, if not clear from the output, evidence or year of dissemination (paper and/or DVD/CD/USB)	Other forms of assessable output meeting the definition of research but not captured within any of the above categories. This may include (but is not limited to): new materials structures images buildings food products and processes published geological and/or geomorphological maps creative bodies of enquiry design processes / programmes of research multi-platform project.

Academic experts are appointed to REF UoA sub-panels and review all the research submissions. Research users also sit on panels and provide input into the assessment of impact case studies.

There are four main panels (A-D) who comprise a group of expert researchers representing a broad discipline area whose role is to lead and oversee the conduct of the sub-panels under their remit. They are responsible for setting the assessment criteria, working methods and consistency of approach and adherence to criteria across the sub-panels.

Where a sub-panel member declares an interest they do not participate in the assessment of departments or institutions where it may be perceived that they could be open to external influences, for example; where they or a close family member are employed.