each year, the irish government spends a significant amount of public funds on scientific research, training and development. as with all public spending, it is both desirable and necessary to show value for money and, within this, demonstrate and articulate the impact and benefits of investing in scientific research. as ireland’s scientific infrastructure and capacity matures, there is an even greater focus on demonstrating the economic, societal and other benefits of publicly-funded scientific research to the wider society.
in support of delivering impact from the state’s investment, sfi published its strategy document, agenda 2020, in 2013, which sets out a vision in which ireland, by 2020, is the best country in the world for both scientific research excellence and impact.
further insights into sfi’s impact framework and the objectives of sfi’s impact assessment can be found below.
sfi agenda 2020 sets out a vision in which ireland will, by 2020, be the best country in the world for both scientific research excellence and impact. while sfi continues to focus on scientific excellence, we now apply an equal focus on impact across a portfolio of programmes. sfi defines impact as the “demonstrable contribution that excellent research makes to society and the economy” sfi classifies the impacts of scientific research according to 8 pillars which are underpinned by 3 thematic areas (see diagram below) sfi defined this framework as part of the small advanced economies initiative “broadening the scope of impact”.
sfi has always asked applicants to articulate the value of their research to ireland but is expanding and refining this aspect of research proposals and in the evaluation of award progress and will use experts in the translation, commercialisation and development of scientific research to evaluate research impact as an important and integral part of our review processes. the objectives of sfi’s impact assessment are as follows:
- to stimulate researchers to consider how best to maximise the impact of their research and how to maximise the engagement of users of their research
- To actively demonstrate the contributions and benefits of publicly funded research to society and the economy and, in doing so, demonstrate the value of publicly funded investment in R&D
as part of the annual reporting guidelines, all of sfi awardees are provided with a list of 10 impact ‘declarations’ or statements. at least one statement must be selected but awardees are encouraged to rank up to 5 statements, starting with the number 1 (being the most relevant). awardees are then asked to provide more details justifying the statements selected. the impact declarations help sfi to quantify the types of impacts coming from the various sfi programmes. each declaration statement can be aligned with at least one of the types of impact.
types of impact
impacts where the beneficiaries may include businesses, either new or established, or other organisations which undertake activity that creates jobs and revenue. beneficiaries may also include graduates, employees, trained scientists and the general public. the following are examples of economic impacts:
- a spinout or start-up has been created around a new product, service or licence
- research has attracted and nurtured developing businesses, for example, through the licensing of technologies.
- industry or other organisations or charitable foundations have invested in their own research and development through research collaboration.
- performance has been improved, or new or changed technologies or processes have been adopted, in companies or other organisations through highly skilled people having taken up specialist roles that draw on their research, or through the provision of consultancy or training that draws on their research.
- employment has been created or increased through the production of a highly educated and relevant workforce in demand by industry and academia.
impacts where the beneficiaries may include individuals, groups of individuals, organisations or communities whose quality of life, knowledge, behaviours, creative practices or other activity have been influenced positively. the following are examples of societal impacts:
- public debate has been stimulated or informed by research.
- the awareness, attitudes, education and understanding of the public have been enhanced by engaging them with research activities
- quality of life has been improved through improved access to healthcare.
- research has contributed to community development and regeneration
- research supports creativity and increases appreciation and/or design of cultural services, for example, museums, galleries, libraries, through improving cultural awareness or improving the design and accessibility of public facilities thereby having a positive impact on cultural life of population and/or national identity.
- mitigation of risks to public health, for example, through preventative measures for communicable and non-communicable diseases
international engagement impacts
impacts where the beneficiaries include irish based research scientists who are striving to improve their international reputation and international scientists who wish to relocate their research groups to ireland. irish businesses and irish headquarters of mncs may also benefit from increased international engagement. the following are examples of international engagement impacts:
- significant contribution to global challenges, for example in the areas of health, the environment and poverty reduction
- contribution to international relations and the international profile and reputation of ireland
- attraction of international scientists and talented people
- leveraging of international funding through industrial and collaborative research
- new connections to international expertise have been developed providing access to new markets and state-of-the art knowledge
Policy & public service impacts
Impacts where the beneficiaries may include government, non-governmental organisations (NGOs), charities and public sector organisations and society, either as a whole or groups of individuals in society. Impact can occur top-down through policy changes and bottom up, through changing behaviours at the delivery level. The following are examples of Policy & Public Service Impacts:
- implementation, revision or verification of policy to improve efficiency, efficacy and responsiveness of public services and / or government regulation
- policy decisions or changes to legislation, regulations or guidelines have been informed by research evidence
- changes to education or the school curriculum have been informed by research.
- improvements in best practice of those delivering public services, have been made
- risks to national security have been reduced
- improvements in risk management in public services/public sector
health and wellbeing impacts
impacts where the beneficiaries may include individuals (including groups of individuals) whose health outcomes have been improved or whose quality of life has been enhanced (or potential harm mitigated) through the application of enhanced healthcare for individuals or public health activities. the following are examples of health and wellbeing impacts:
- patient health outcomes have improved through, for example, the availability of new drug, treatment or therapy, diagnostic or medical technology, improvements to patient care practices or processes, or improvements to clinical or healthcare guidelines.
- public mental and social health and well-being has improved.
- increased efficiency of delivery of public health services.
- decisions by a health service or regulatory authority have been informed by research.
- animal health and welfare has been enhanced by research.
- reduction in cost for treatment for an equivalent outcome through a new drug, device or improved diagnostics
- mitigation of risks to public health, for example, through preventative measures for communicable and non-communicable diseases
- disease prevention or markers of health have been enhanced by research
- public awareness of a health risk or benefit has been raised
- improved nutrition and food security
- debate on the environment, environmental policy decisions or planning decisions have been stimulated or informed by research and research outputs
- the operations of a business or public service have resulted in the meeting of relevant environmental objectives
- new/improved technology or process has led to direct reduction in pollution and/or reduction of impact of pollutants on ecosystems and humans
- improvement in sustainable use of resources and reduced overall consumption of constrained resources
- the management of natural resources, including issues around global competition for energy, water and food resources, has been improved.
- understanding of health risks to livestock and disease risks to crops have improved, enabling improved health and increased security in food production.
- in the built environment, infrastructure or housing quality and/or longevity have been increased.
professional services impacts
- changes to professional standards, guidelines or training have been informed by research.
- practitioners/professionals/lawyers have used research findings to improve the standard of their working practices
- the quality or efficiency or productivity of a professional service has improved.
- professional bodies and learned societies have used research to define best practice.
- forensic methods/technologies have been improved as a result of research
human capacity impacts
impacts where beneficiaries cover the entire population, primary school students studying stem subjects, the general workforce including science teachers, health professionals, policy makers, business leaders in smes and mncs and the general public. the following are examples of human capacity impacts:
- the production of a highly educated and relevant workforce in demand by industry and academia
- increased productivity of the workforce through improvements in health and general work environment
- improved scientific and technical skills of current and future workforce
- increased uptake of stem subjects at secondary and university level
- public interest, discussion and engagement in science, engineering and mathematics has been stimulated
- attraction of international scientists and talented people to ireland,
- performance has been improved, or new or changed technologies or processes have been adopted, in companies or other organisations through the employment of highly skilled people having taken up specialist roles that draw on their research, or through the provision of consultancy or training that draws on their research
- increased leveraged funding though programmes such as horizon 2020 due to the increased number and level of highly skilled researchers in ireland
preparing your impact statement
view the presentation for a webinar designed to help applicants understand impact with guidance on what makes a good impact statement.
the statement should be as specific as possible and provide information that sfi and external reviewers will find helpful in assessing the potential impact of the proposed research activity. innovative and creative approaches to engaging beneficiaries and creating impact are strongly encouraged. appropriate milestones and deliverables associated with the potential impact should be indicated. sfi funds research in very disparate domains hence depending on the nature of the research, the impacts may be short-term, medium-term or longer-term. a credible implementation plan outlines the pathways to impact citing realistic timelines.
as examples, applicants should briefly outline previous indicators of the relevance of their research:
- changes to the state of knowledge within a field
- where past group members have found employment
- industrial interest in their past or current work
- collaborative projects
- vc funding obtained
- companies formed
- problems solved
- documented changes to public policy or guidelines
- improvements in public health
- who will benefit from this research?
- how will they benefit from this research?
in thinking about potential impacts, the following points should be considered:
- what is the activity’s potential impact on the development of science, technology and industry in ireland and ireland’s economy and competitiveness?
- how will industry collaborators enable increased impact? what supports are they offering? have routes to commercialisation been considered?
- what is the activity’s potential impact on society and the quality of life of ireland’s citizens?
- how will the potential impacts from your research best be realised?
- how do you propose the impact from your project could be measured?
common characteristics of high quality impact statements
- good knowledge of the relevant beneficiaries of the proposed research and the needs of the sector(s)
- clear description of how the applicant intends to reach and engage with the beneficiaries of the research, including clear deliverables and milestones
- good knowledge of national priorities and activities in the relevant areas
- applicant demonstrates clear commitment to maximising the impact of their research
common characteristics of poor impact statements
- statement is vague, lacks specificity and clear deliverables
- activities are not project specific, but are routine activities for academic research positions
- too much focus on track record rather than what will be done as part of the proposed research project
- lack of knowledge of beneficiaries, likely impacts and appropriate mechanisms for realising the potential impacts
- too focused on outputs for their own sake rather than their contribution to impact generation
- unrealistic expectations (not to be confused with setting ambitious goals)
in summary, a poor quality impact statement lacks a credible implementation plan.
what should not be included?
as outlined above, on their own the following are not impacts but are considered research outputs:
publications; presenting research at a conference
while the dissemination of research output is very important, in articulating how it will lead to the utilisation of the outputs, applicants should be specific as to why that publication or conference presentation is important, how it ensures the potential beneficiaries have the opportunity to engage with the research, and how this will be followed up.
invention disclosures; patent filings
how will impact statements and impact be reviewed and evaluated?
The requirements for Impact Statements will vary from programme to programme and will be based on the objectives of a particular programme and the specific call. Details of Impact Statement requirements and how they will be reviewed will be included in the call document for each particular programme call. In nearly all cases, SFI will use international experts with specific/documented interests in impact evaluation from other jurisdictions to review and rank the impact statements of scientifically excellent projects. These impact reviewers have included Company R&D Directors, Heads of Translational Institutes, Senior Technology Transfer Professionals, Investors in scientific/technology early-stage companies, for example.
sfi researchers (or award holders) are required to complete an annual report on the progress of their award. since 2014, the majority of sfi annual reports are managed through its online grants and awards management system, sesame. each sfi researcher or centre has a “researcher profile” in sesame which they update with their outputs continuously throughout the year. these outputs, in turn, can be used to populate the researcher’s or centre’s annual report.
the annual report template for the majority of the awards that sfi makes contains a section on “strategic impact” in which the award holder is asked to prioritise at least five relevant impact declarations from a list of options provided (see below).
sfi are using this self-assessment type approach, since it goes in some way towards quantifying the types of impact arising from the awards it makes. this approach, however, may be subject to bias and so in addition to providing outputs through the researcher profile in sesame in support of their chosen impact declarations, the award holders are required to provide narrative/details justifying the options they have selected.
as part of the annual reporting guidelines, all of sfi award holders are provided with a list of 11 impact ‘declarations’ or statements. at least one statement must be selected but award holders are encouraged to rank up to 5 statements, starting with the number 1 (being the most relevant). they are then asked to provide more details justifying the statements selected. the impact statements help sfi to quantify the types of impacts coming from the various sfi programmes. each statement can be aligned with at least one of the “types of impact” discussed here. the impact type is listed in parenthesis after each statement.
- the research conducted through my award has enabled me to leverage international funding through industry/collaborative research [economic and commercial, international]
- the research conducted through my award has resulted in the start or expansion of a company which has resulted in the creation of high value jobs [economic and commercial]
- the research conducted through my award has attracted developing and nurturing businesses [economic and commercial]
- the research conducted through my award has attracted international scientists and talented people [human capacity; international engagement]
- the research conducted through my award has resulted in a new policy being implemented and/or an improvement to the delivery of a public service [public policy and services]
- the research conducted through my award has enhanced the quality of life and health of irish citizens [Health & Wellbeing, Societal Impact]
- the research conducted through my award has improved the environment and/or the sustainable relationship between society, industry and the environment [environmental impact]
- the research conducted through my award has increased the knowledge, appreciation and understanding of science, engineering and technology amongst the general public. the research conducted through my award has developed the country’s international reputation [societal impact, international engagement]
- the research conducted through my award has resulted in the creation of employment through directly influencing and inspiring the future workforce and/or the production of a highly educated and relevant workforce in demand by industry and academia [human capacity, economic and commercial]
- 通过我的奖项进行的研究已经影响到其他领域，而这些领域并未体现在所提供的选择中，例如，通过提高爱尔兰公民的创造力 [environmental, professional services, societal]
- the research conducted through my award has not yet realised any significant impact
mid-term programme progress review
many of sfi’s awards, in particular awards of scale and investigator-led awards, are subject to a mid-term programme progress review. international experts in the relevant discipline, including those with expertise in relevant areas of industry, commercialisation and translation, are required to evaluate the progress being made against the original impact statement as submitted in the funded proposal. there is specific guidance provided to direct the reviewers to ‘score’ with reference to indicators of impact, as defined by sfi, and not simply to ‘rate’ the outputs on the award, some of which may have little relevance to impact. furthermore, all research centre awards are asked to provide targets for specific outputs which are considered in direct support of delivering impact. these are examined internally by sfi staff, and used to gauge progress and report on their progress to the sfi executive and board, and our parent department.
programme evaluation is a critical process towards ensuring that the mechanisms for providing funding are aligned with the relevant strategy du jour, with particular focus on delivering scientific excellence and impact. specifically, data collated for the annual stocktake of sfi research outputs (formerly referred to as the sfi census) and collected in the annual reporting process are used to support detailed programmatic evaluations. this ensures that the success of a particular programme in delivering against its original objectives and the collective metrics gathered in support of key performance indicators (kpis) as defined in sfi’s agenda 2020, can be determined. furthermore, the associated narrative in support of the metrics gathered (typically found in the annual report) can be used to provide case studies and additional context to this data. this is particularly important for scenarios where rationale in support of impacts arising from specific programmatic activities is required to be articulated to our parent department as justification for the investment made.
research impact links
the information on sfi’s research impact webpages is a collection of knowledge and experience from a variety of sources; sfi would like to acknowledge the small advanced economies initiative, the economic and social research council uk (esrc), the european science foundation (esf), the european commission, and the sfi research community and stakeholders for material used to build our impact knowledge base.
- small advanced economies initiative - broadening the scope of impact: defining, assessing and measuring the impact of major public research programmes
- economic and social research council impact toolkit
- european commission impact assessment guidelines
- european science foundation report on impact assessment
case studies are not impact statements. they are examples of the impact possible after years of investment. the case studies showcase longer-term impact generated from the scientific research community. society reaps many benefits from excellent research, some of which occur immediately, while others develop over years or generations. this repository of case studies will show examples from a wide spectrum of endeavour over a wide range of impact categories - economic and commercial, societal, health and wellbeing, environmental, international engagement, human capacity, public policy, services and regulation and professional services.
each case study will outline how a researcher or group used their knowledge and research ideas to help change the world. each one will contain the challenge the researcher faced, the response of the researcher to that challenge, the engagement they had with other academics, industry and/or agencies, the impact generated from their work and the next steps they plan to take.
sfi impact webinar
each year the irish government spends a significant amount of public funds on scientific research, training and development. as with all public spending it is both desirable and necessary to show value for money and within this to demonstrate and articulate the economic, social and cultural benefits of scientific research projects to the wider society.
the following webinar is designed to assist applicants to sfi programme calls in formulating their impact statement. impact can be described as the “demonstrable contribution that excellent research makes to society and the economy”. scientific excellence of both the applicant(s) and the research programme is both necessary and paramount, but is not sufficient; applicants must also demonstrate the potential impact arising as a result of the programme of research.
applicants are asked to give thoughtful consideration in relation to how the proposed research will deliver impact with emphasis on the pathways to impact as well as the milestones and deliverables within the project duration and beyond. we hope that this impact webinar will be a useful resource when planning your impact statement as part of an application to an sfi programme.