Explore as a

Share our content

Information for panellists

Information for panellists assessing Rutherford Discovery Fellowship applications

The Panellists' Guidelines are available as a PDF: RDF Panellist Guidelines 2020

Proposal guidelines for Panellists

Amendments for 2020

  • More guidance for Vision Mātauranga has been provided along with links to resources. All applicants are required to write a Vision Mātauranga statement. Applicants marking the proposal as ‘not applicable’ must provide a rationale for this decision. Panellists need to consider the quality of the Vision Mātauranga statement in their assessment.
  • Assessment of the applicant’s leadership quality has been amended to emphasise that non-discipline-specific leadership should be assessed equally to discipline-specific leadership. Examples could include leadership in Mātauranga Māori, community leadership; board and council positions/memberships, etc.

About the Rutherford Discovery Fellowships

The Rutherford Discovery Fellowships are administered by the Royal Society Te Apārangi (the Society) for the New Zealand Government.

The Fellowships will develop and foster the future leaders in the New Zealand science and innovation system[1].  They will attract and retain New Zealand’s most talented early- to mid-career researchers and encourage their career development by enabling them to establish a track record for future research leadership. It is expected that Fellows, throughout their careers, will contribute to positive outcomes for New Zealand.

Receipt of a Rutherford Discovery Fellowship is expected to have significant value in the future career development and leadership potential of a researcher.

Fellowships are awarded on a full-time basis of which at least 85% (or 0.85 FTE) of the Fellow’s time must be dedicated to the research objectives identified in the proposal, unless an exemption to this requirement has been approved by the Society. The remainder of their time may be used for other research, teaching and non-research related development opportunities.

The scheme will award a minimum contribution of $70,000 per year towards the researcher's salary, $60,000 in research related expenses, and $30,000 per year for the host organisations to support the Fellow’s research programme.

[1] This includes research in science, technologies, and humanities.

Eligibility

For the purpose of this scheme, early-to mid-career researchers are researchers whose doctoral degrees were conferred between three and eight years prior to the year in which the Fellowship is awarded. For the 2020 funding round, applicants must have a PhD conferment date between 01 January 2012 and 31 December 2017. Eligibility can be extended where applicants have an allowable career interruption, including maternity/parental leave, medical leave, part-time employment because of ongoing childcare responsibilities, or as otherwise agreed to by the Society. Applicants that are the primary caregiver of dependent children born since their PhD was awarded, are also able to extend the period of eligibility by two years per child, to account for career interruptions experienced due to being the primary caregiver for young children. The extension of two years per dependent child is inclusive of any periods of parental leave.

All applications forwarded for review fulfil the eligibility requirements.

Assessment Process (in brief)

The Society will appoint a selection panel, chaired by the President of the Society, or their nominee, to oversee the selection process.  The Chair of the panel will work with the Society’s nominated manager to determine the best process to be used. The assessment of proposals is a two stage process.

Stage one is the assessment of all proposals by three discipline-based panels. Discipline-based panellists are asked to participate in a briefing video conference prior to beginning their assessment, but do not otherwise meet. Each discipline-based panellist grades the proposals within their panel and then submits their grades on an electronic form.  All proposals are graded against three criteria and panellists are obliged to consider the three applicant-solicited referees to produce their final scores for the applicants. Panellists need to consider the quality of the Vision Mātauranga statement in their assessment. There is a space for panellist comments on Vision Mātauranga on the electronic form.

Once the overall scores from the panellists have been received, the Rutherford Discovery Fellowship Secretariat will produce an ordered list of applicants with the highest grades from each of the discipline-based panels.  These top ranking applications will form the Long List for consideration by the interview panel.  The number of applicants from each panel on the Long List will be determined by the number of proposals submitted (Table 1).

For information, discipline panellists will receive the overall panel scores and Long List following stage 2a of the process.

Stage two is in two parts: (a) the assessment of the Long List of applicants by the interview panel; and, (b) interviewing a shortlist of applicants and making recommendations for the successful Fellows.  The Chair and four member interview panel will conduct the interviews.

 Process flow image

 

  • Process flow

 

 

Panel

 

Number of proposals submitted to each discipline-based panel

Number of proposals submitted    (% of total)

Long List (number)

HSS

 

49

30%

12

LFS

 

56

35%

14

PEM

 

57

35%

14

Totals

 

162

100%

40

  • Example distribution of proposals if 162 applications were to be received.

 

Discipline-based panels (Stage one)

Each of the three research areas will have a discipline-based assessment panel.  The panel comprises researchers who are experts in their field, have a broad knowledge of the research area and are experienced in assessment.  Panel members are appointed by the Society under consent from the Chair of the selection panel.  These panels are advisory only, providing recommendations on the relative merits of proposals to the interview panel.  The three research areas are:

Humanities and the Social Sciences (HSS).

Research related to the human condition or aspects of human society.

This includes, but not limited to: English; languages; history; religion; philosophy; law; classics; linguistics; literature; cultural studies; media studies; art history; film; economics; education; psychology (cognitive, social, developmental, organisational, community and health); cognitive science; linguistics; archaeology; anthropology; sociology; social, cultural and human geography; social anthropology; architecture, urban design and environmental studies; public health; nursing; public policy; marketing; political science; and business studies.

Life Sciences (LFS).

Research related to understanding the activities that occur in cells and tissues and the interrelationships between organisms and their environment.

This includes, but not limited to: physiology (plant or animal), pathology (animal or plant), pharmacology, molecular biology, genetics, cell biology, microbiology; neurobiology and neuropsychology (including animals as a model species for humans); animal behaviour; population biology genetics; functional genomics and related bioinformatics; biostatistics and modelling; animal, plant and microbial ecology; biogeography; biodiversity; phylogenetics; systematics and evolution; biophysics, chemical biology; and biochemistry.

Physical Sciences, Engineering and Mathematics (PEM)

Research related to the physical world and mathematics.

This includes, but not limited to: physics; physical chemistry; organic chemistry; analytical chemistry; inorganic chemistry; pure and applied mathematics; statistics; logic, theoretical and engineering aspects of computer and information sciences; complexity theory; operations research; nanotechnology; software and hardware engineering; applications and robotics; materials science; engineering (including bioengineering and other cross-disciplinary research activities); geology; geophysics; physical geography; oceanography; hydrology; meteorology; atmospheric science; earth sciences; astronomy; and astrophysics.

Assessment of proposals

Discipline-based panellists are asked to participate in a briefing video conference on one of the two dates indicated in the timetable. Each panel member will receive electronic copies of the applications for their panel. Panel members are asked to read, assess and grade each proposal based on the three selection criteria, taking into account the applicant-solicited referee reports.

When considering your grade, please take into account Vision Mātauranga across the leadership and/or research programme criteria where applicable. Proposals are to be assessed by panel members exclusively on the information provided in the proposal and referee reports.

Panel members also need to identify proposals for which they have a conflict of interest, explaining the nature of the conflict (please refer to Conflicts of interest).

Each panel member is asked to start reading applications at different points through the order of the proposals, to avoid proposals from institutions or researchers first in the alphabet always being read first.

Each panel member will receive an electronic form on which to record their grades and comments.  The spreadsheet should be completed and returned to the Rutherford Discovery Fellowship Secretariat by the due date.

The budget section is included for panel members’ perusal but is not to be graded by the discipline panels.

It should be noted that these discipline-based panellists return their grades to the Rutherford Discovery Fellowships Secretariat and do not convene for a meeting at the end of their assessment.  The collated grades from the panellists will be used to create a ranked list of applications to be considered as the Long List.  This Long List is reviewed by the interview panel.

Assessment in relation to years of research experience

Panel members must consider applicants’ track record in relation to their years of research experience, which may differ from the number of years since PhD conferment. The years of research experience (R) is noted in the application header and on the first page under research area, and excludes periods of maternity/parental leave, medical leave or other relevant leave outlined in section 1e of the CV.

Consideration of referee reports

Applicant-solicited referees are used for the assessment of proposals in conjunction with the selection criteria. Where referees disagree, the panel members must use their own judgement in determining which referee reports to emphasise and what score to assign. These deliberations should be guided by considerations such as: the panel member's own level of expertise on the subject; the comments made by referees to explain their grades; the relative competencies of the referees; and, possible conflicts of interest. The applicants have been informed that referees should not be directly involved in the proposed programme of research or in your chain of line management.

Assessment Criteria

Proposals are assessed on the information provided in the application, the accompanying forms and the applicant’s self-nominated referee reports.

 

For the current funding round the following weightings will be used:

  1. Calibre of the applicant as a researcher          60%
  2. Calibre of the applicant as a research leader   20%
  3. Calibre of the proposed research programme 20%

 

In the case of applicants of the same calibre, preference will be given to applicants who:

  • do not already have tenure or equivalent, or
  • who are living overseas and will use the Fellowship to return to New Zealand to continue their research careers.

Considerations for assessing proposals to each criteria

Panel members may wish to consider the following as a guide for assessing the three criteria.

1. Calibre of the applicant as a researcher

Consider if the applicant’s career is exceptional for a candidate in this discipline, at their career stage.

Exceptional may be determined by consideration of the merit of the applicant’s career to date and how the research compares with other New Zealand or international research in the same field.  If the applicant is at the start of his or her career the calibre must be assessed in relation to the years of research experience.  The curriculum vitae, supplied by the candidate in Section 4, should address the calibre of the applicant as a researcher.  Some expected sources of evidence include: awards/prizes; invitations to editorial boards or keynote addresses at conferences; publication record; patents awarded; and, referee reports.

2. Assessment of the applicant’s leadership quality

Consider the leadership qualities you believe the applicant possesses, or the potential they have.

Note that non-discipline-specific leadership (e.g. leadership in Mātauranga Māori, community leadership, board and council positions/memberships, etc.) should be assessed equally to discipline-specific leadership.

Expected sources of evidence may include but are not limited to: team leadership roles; student numbers and completions; leadership in Mātauranga Māori; community leadership; board memberships; project management responsibilities; quality of stakeholder relationships; external grant funding as a named investigator; presence in relevant research communities; invitations to present keynote or plenary presentations; collaborator networks; knowledge transfer activity; significant contribution to achievement of commercialisation milestones; entrepreneurial activity; indications of peer-esteem; thought leadership (e.g., conceptual development of a research field internationally); and, direct policy facing or public engagement work. If appropriate, applicants may also indicate the future direction they wish to develop their leadership skills.

   3. Assessment of the proposed research programme

Consider the merit of the proposal and the potential of the research.

Merit may be determined by the applicant incorporating originality, insight and rigour.  Please consider the ability of the researcher to carry out the research. Where relevant, applicants should consider the relation of the research to the themes of Vision Mātauranga and how the project will engage with Māori.

Potential of the research may be assessed from the work outlined in Sections 8-10 of the proposal.  The research should significantly contribute to advances in theoretical understanding, develop new methodologies, contribute to new knowledge, or lead to advancement in a field by cross-fertilisation with ideas and results from another field.  Often the design and planning of a programme of research determines its success.  Good design and planning are determined by whether the overall proposal and its specific objectives have a clear focus, and the methods and experimental or sampling design are likely to produce high quality results.  Expected sources of evidence include: evidence of host support; and, referee reports.

Vision Mātauranga

Vision Mātauranga is a policy about innovation, opportunity and the creation of knowledge that highlights the potential contribution of Māori knowledge, resources and people. There are four themes:

 

  • Indigenous Innovation, which involves contributing to economic growth through distinctive research and development;
  • Taiao, which is concerned with achieving environmental sustainability through iwi and hapū relationships with land and sea;
  • Hauora/Oranga, which centres around improving health and social wellbeing; and
  • Mātauranga, which involves exploring indigenous knowledge.

 

A Vision Mātauranga statement must be included for all research that has relevance for Māori. If the applicant ticks N/A, they are also required to provide a rationale for this decision. Panellists need to consider the quality of the Vision Mātauranga statement in their assessment, and take into account Vision Mātauranga across the leadership and/or research programme criteria where applicable when considering their grade. There is a space for panellist comments on Vision Mātauranga on the electronic form.

For more information on Vision Mātauranga, including guidance for applicants, please see Appendix I.

Grades and distribution

There are six scores available; 1 (excellent) to 6 (room for improvement).  Each criterion should be assigned one of the six scores.  Each panel member should use the following target distribution for the proposals that they assess.

 

Score

1 (excellent)

2

3

4

5

6

(room for improvement)

% of proposals

10-20

15-25

20-30

15-25

10-20

0-10

Example (60 proposals)

6-12

9-15

12-18

9-15

6-12

0-6

  • Target distribution.

 

In the example above where 60 proposals are assessed, between 6 and 12 proposals should be assigned a score of 1, between 9 and 15 proposals should be assigned a score of 2, between 12 and 18 proposals should be assigned a score of 3, and so on.

Grading and recommendation to the interview panel

Once the overall scores from the panellists have been received, the Rutherford Discovery Fellowship Secretariat will produce an ordered list of applicants with the highest grades from each of the discipline-based panels.  These top ranking applications will form the Long List for consideration by the interview panel.  The number of applicants from each panel on the Long List will be determined by the number of proposals submitted.

Interview panel (Stage two)

The Chair and members of the interview panel will conduct the interviews.  This is a two-part process:

  • The interview panel assesses the Long List of applicants with the highest ranking grades from the discipline-based panels and will create a shortlist of applicants to be invited for interview. The applicants called to interview will be the highest ranked by the panel and does not need to reflect the number of proposals in a particular discipline.
  • The interview panel will conduct interviews and recommend the successful applicants for the Fellowships.

 

The Chair of the interview panel is responsible for the effective conduct of the assessment process.  This post will be filled by the President of the Society or their nominee.  Each panel member needs to ensure that the funding recommendations made are defensible by ensuring the framework for assessment, including Vision Mātauranga, is followed and identifying, and taking appropriate action, over conflicts of interest.

Each applicant will be asked a series of questions in an allocated 20-minute interview.  Overseas applicants will be interviewed using either teleconferencing or video-conferencing facilities.

The recommendations of the interview panel for successful applicants are ratified by the President of the Society.

Sensitive issues

Privacy

The Society has obligations under the Privacy Act to keep confidential certain information provided by individuals.  Moreover, the records of deliberations by panels are regarded as strictly confidential; as are the contents of applications.

  • Panel members should ensure the safe keeping of all applications and related confidential documents (e.g. applications, referee reports, scoring spreadsheets or summaries).
  • At the conclusion of the grading, panel meetings and the interviews, members should leave documentation with the Society staff and destroy any documentation remaining elsewhere.
  • Panel members should not enter into correspondence or discussion of the contents of the applications with referees, third parties, or the applicants.  Any necessary correspondence shall be addressed by the Secretariat of the Rutherford Discovery Fellowships.
  • The intellectual property of the ideas and hypotheses put forward in the applications should be treated in strict confidence.

Conflicts of interest

The Society takes the issue of conflicts of interest very seriously. A rigorous position is taken in order to maintain the credibility of the allocation process and to ensure that applications are subjected to fair and reasonable appraisal.

The Society wants to ensure that the panel members are active researchers with an excellent background in research.  As these researchers will invariably have connections with some applicants, conflicts of interest will arise. Where these occur for panel members, the following rules will apply. 

  • All conflicts of interest must be declared in writing to the Society.  Society staff will minute all conflicts of interest and actions taken.
  • Where a panel member is a family member or close friend of any applicant(s), that person will not assess the proposal or interview the candidate and take no part in the consideration of that proposal.  They will hear about the outcome of that proposal when official letters are sent to all applicants.
  • If a panel member has an interest in an application, such as collaborating with an applicant or an applicant’s group, or is conflicted with the applicant* then that member shall not assess the proposal or interview the candidate.
  • A panel member cannot be a referee for any applicant in the current funding round.
  • If the interview panel Chair has a conflict of interest then the duties of chairing the interview shall be passed to another panel member.

 

*A panel member is generally deemed to be conflicted if:

  • They work in the same department as the applicant(s). Where the department is large and contact between the panel member and applicant(s) is minimal, the Chair may deem there to be no conflict.
  • They work at the same CRI AND are in the same team as the applicant(s) (the level of conflict will depend on the size of the organisation).
  • They work at the same company as the applicant(s). The level of conflict will depend on the size of the company.
  • They have co-authored publications with the applicant(s) in the last 5 years
  • They have a low level of comfort assessing the application due to their relationship with the applicant(s).

When all conflicts of interest are taken into account, the interview panel Chair may decide that the remaining panellists’ expertise is not sufficient for assessment of a particular proposal.  In this case, an additional opinion from an external independent person may be sought.  Alternatively, a panellist who has previously left the room may be asked to return to answer technical questions only.

Additional Information

Feedback to applicants

In the Proposals On-Line web-based system, applicants are offered the option of receiving feedback in the form of quartiles for the three graded criteria at the conclusion of the funding round.  A general statement about the funding round will also be prepared and given to all applicants.

Applicants will also be notified:

  • if the applicant was successful in making the Long List.
  • if the applicant is considered ineligible to apply for a Rutherford Discovery Fellowship.

Disposal of applicant proposal matter

Referees are asked to return only the completed Referee report form.  Please destroy all proposal material once your report is completed.

Royal Society Staff

It is not the role of Society staff to make funding decisions.  Rather, their role is one of facilitation of and "guardianship" over the assessment process, ensuring that the process is credible and defensible.  To achieve this, staff will:

  • organise all logistical aspects of the process;
  • assist the discipline-based and interview panellists in determining realistic timetables for meetings;
  • provide a framework for assessment;
  • record funding decisions and collate generic feedback for applicants;
  • record any conflicts of interest and identify problem areas;
  • convey funding decisions to applicants and their host organisations - all discussions related to a decision should occur through Society staff; and,
  • negotiate contract details with host institutions.

Timetable

Date

Activity

Thu 5 March 2020

Proposals On-Line web-based application system opens.

Thu 30 Apr 2020

On-Line web portal closes at 5 pm (New Zealand Standard Time).

Thu 28 May 2020

Deadline for receipt of applicant-solicited referee reports by the Secretariat of the Rutherford Discovery Fellowships at 5 pm (New Zealand Standard Time).

Thu 28 May or Tue 2 Jun

Discipline panellists briefing via video conference from 10.00 – 11.00 am.  Panellist to choose one of two dates offered.

Thu 04 Jun 2020

Discipline panellists sent proposals and referee reports

Fri 03 Jul 2020

Last day for discipline-based panellists to submit their recommendations to the Secretariat.

Early Jul 2020

The long-listed proposals are sent to the interview panel.

Tue 11 Aug 2020

Last day for interview panellists to submit their recommendations to the Secretariat.

Late Aug 2020

Interview panel selects a short list of candidates to interview.

Mid Sep 2020

Interviews conducted by the interview panel.  Dates to be confirmed.

Oct 2020 (TBC)

Results announced.

  • Timetable for 2020

Appendix 1 - Vision Mātauranga information for applicants

Vision Mātauranga is a policy about innovation, opportunity and the creation of knowledge that highlights the potential contribution of Māori knowledge, resources and people.

Applicants should identify which, if any, of the four Vision Mātauranga themes below are associated with the proposed research. A Vision Mātauranga statement must be included for all research that has relevance for Māori. If this is not applicable to an applicant’s proposed research, they must tick N/A AND provide a rationale for why this is. 

The four themes are:

  • Indigenous Innovation, which involves contributing to economic growth through distinctive research and development;
  • Taiao, which is concerned with achieving environmental sustainability through iwi and hapū relationships with land and sea;
  • Hauora/Oranga, which centres around improving health and social wellbeing; and
  • Mātauranga, which involves exploring indigenous knowledge.

How do applicants decide whether to include a Vision Mātauranga statement in their proposal?

The five ways of conceptualising Vision Mātauranga in an applicant’s research (see below) may help them decide if this applies to their project. The categories have been adapted from those on the National Science Challenge, Biological Heritage website https://bioheritage.nz/about-us/vision-matauranga/  hosted by Manaaki Whenua Landcare Research. Please note, however, that these categories are fluid. There may well be overlap between them as in categories b and c in terms of the nature and degree of relevance to Māori, and not every point in each category need apply. The original categories were set out by MBIE in information for the Endeavour Fund c. 2015. 

Ways of conceptualising Vison Mātauranga in an applicant’s research

Research with no specific Māori component

  • No mātauranga Māori (Māori knowledge) is used.
  • Māori are not associated with the research process (e.g. not on any research management / advisory / governance panels, it is not inclusive of Māori land or institutions, nor the subject of any component of the research).
  • Work is not likely to be of greater direct relevance to Māori than members of any other group.

Research specifically relevant to Māori

  • There is specific relevance to Māori.
  • Mātauranga Māori may be used in a minor way to guide the work and its relevance to Māori.
  • It includes work that contributes to Māori aspirations and outcomes.

Research involving Māori

  • Mātauranga Māori may be incorporated in the project, but is not central to the project.
  • Research is specifically and directly relevant to Māori and Māori are involved in the design and/or undertaking of the research.
  • The work typically contributes to Māori (e.g., iwi / hapū, organisations) aspirations and outcomes.

Māori-centred research

  • The project is Māori led, and where Mātauranga Māori is used alongside other knowledges (e.g. through frameworks, models, methods, tools, etc.).
  • Kaupapa Māori research is a key focus of the project.
  • Research is typically collaborative or consultative, with direct input from Māori stakeholders.
  • There is alignment with and contribution to Māori (e.g., iwi / hapū, organisations) aspirations.

Kaupapa Māori research

  • Mātauranga Māori is incorporated, used and understood, as a central focus of project and its findings.
  • Research is grounded in te ao Māori and connected to Māori philosophies and principles.
  • Research typically uses kaupapa Māori research methodologies.
  • Te reo Māori may be a central feature to this kaupapa or research activity, and the applicant has medium to high cultural fluency or knowledge of tikanga and reo.
  • The research is generally led by a Māori researcher; non-Indigenous researchers may carry out research under the guidance/mentoring of a Māori researcher.
  • Māori participation (iwi/hapū/marae/individual) is high.
  • The work contributes strongly to Māori (e.g., iwi/hapū, organisations) aspirations and outcomes and is mana enhancing.

Developing a Vision Mātauranga statement

It is important for applicants to keep in mind that there is no single approach or prescription for Vision Mātauranga: one size does not fit all and there are many possible ways of addressing Vision Mātauranga. Vision Mātauranga should not, however, be seen as an add-on, nor should it be treated as separate from the research, methods or people involved in the project. A holistic approach that considers reciprocity and relationships is therefore desirable.

Vision Mātauranga does not begin and end with the applicant’s Vision Mātauranga statement. They should document how they have considered Vision Mātauranga and demonstrate applicable actions and relationships throughout the research. The following questions may be useful for applicants to consider when conceptualising and writing their project:

  • Have you co-created the research topic/issue with an iwi or Māori organisation?
  • What does working in partnership with iwi mean to you as a researcher?
  • To what extent have you discussed the research with Māori stakeholders and agreed on the methodology you will use?
  • Was there full disclosure and informed consent to the proposed research with Māori stakeholders? How has that agreement/informed consent been agreed to?
  • What provisions have you made to ensure there is appropriate technology transfer to Māori stakeholders as the research proceeds and as findings become available towards the end of the project?
  • Are there benefits to Māori? What are they?
  • How will you share the research outcomes with Māori?
  • Is there a Tiriti o Waitangi component or requirement in your research?
  • Is the research mana enhancing?

Vision Mātauranga Resources

Below you will find a non-exhaustive list of published resources that describe, discuss, and talk about how researchers have engaged with Vision Mātauranga and kaupapa Māori research. These range from early conceptions of Vision Mātauranga to more recent frameworks. The resources underscore the diverse ways Vision Mātauranga may be approached across disciplines and methodologies.

 

Enquiries

If you require further information about the Rutherford Discovery Fellowships, please email us at rutherford.discovery@royalsociety.org.nz or phone 04 470 5764.

Additional information on the Rutherford Discovery Fellowships is available on the following website: https://royalsociety.org.nz/what-we-do/funds-and-opportunities/rutherford-discovery-fellowships/