Scientific Peer Review

All complete applications will be peer-reviewed by at least two reviewers with relevant expertise. The reviewers will be primarily selected from the pool of Tufts CTSI Pilot Studies Program Scientific Training and Review Committee members, which includes past pilot award recipients and collaborators from Tufts CTSI institutions. The review process will follow NIH guidelines for peer review based on the criteria listed below.

Relevance – ability to meet NCATS and Tufts CTSI objectives by demonstrating a clear translational path and explicit relevance to improved health.

  1. Has a strong case been made in regard to advancing translational research (as opposed to basic discovery research)?
  2. Does the project address an important question or problem in a Pilot Studies Program priority research area: Comparative Effectiveness Research, Methods Development, and/or One Health?
  3. Will the project yield a broadly applicable, generalizable solution to a translational problem?
  4. Will the proposed work have a broad application across disciplines?

 Significance – quality and merit of the proposed research project.

  1. Does the project address an important problem or critical barrier to progress in translation?
  2. If the aims of the project are achieved, will the scientific knowledge, technical capability, and/or clinical practice be improved?
  3. Will successful completion of the aims change the concepts, methods, technologies, treatments, services, or preventive interventions that drive this field?
  4. If the development of new methods is proposed, do the methods enable an approach to research that is new, otherwise not possible, or less effective with current methods?
  5. Does the project meet federal requirements for scientific rigor and transparency? For guidance on these requirements, applicants are encouraged to view a seminar and related materials available here.
  6. For projects involving human subjects, does the project include individuals in clinical research in a manner appropriate to the scientific question under study so that the knowledge gained is applicable to all those affected by the researched diseases/conditions?

 Innovation – potential for impact through development of novel solutions and processes.

  1. Does the project challenge and seek to shift current research or clinical practice paradigms by using novel theoretical concepts, approaches, methodologies, instrumentation, or interventions?
  2. Does the project involve the development of novel methods or a new application of existing methods?
  3. Does the project take an interdisciplinary approach to answering a question or solving a problem?

 Investigators – qualifications of the research team to carry out the proposed research project.

  1. Do the investigators/collaborators for the project have the appropriate skills and training to complete the project?
  2. If a multi-site project is proposed, is it clear how the research is linked across sites?

 Environment – availability of resources to support the proposed research project.

  1. Will the scientific environment in which the work will be done contribute to the probability of success?
  2. Are institutional support, equipment, and other physical resources available to the investigators adequate for the proposed project?
  3. Will the project benefit from unique features of the scientific environment, participant populations, or collaborative arrangements?

 Approach – scientific rigor of the proposed clinical or methodological design plan to meet the proposed objectives and goals.

  1. Does the project have a high likelihood of being successfully developed during the award’s one-year time frame?
  2. Are the overall strategy, methodology, and analyses well-reasoned and appropriate to accomplish the specific aims of the project?
  3. Does the proposal outline a clear plan for involving community members and other stakeholder groups in the conceptualization, implementation, interpretation, and/or dissemination of study findings? If not, is a compelling rationale for their exclusion provided?
  4. Are potential problems, alternative strategies, benchmarks for success, and a project timeline presented?
  5. Is a statistical analysis and power analysis plan clearly articulated (including a power analysis) and adequately funded in the proposed budget?
  6. Is there a clear plan for full compliance with all federal, state, and institutional policies, rules, and guidelines regarding biomedical ethics?

 Future Plans – clear articulation of next steps.

  1. Can the preliminary data that support the proposed project, including data from the literature, be used to develop hypotheses for future testing?
  2. If the development of new research methods is proposed, are the methodological applications and plans for publication and dissemination clearly outlined?
  3. Does the proposal outline specific steps for publication of results in the peer-reviewed literature if the pilot project is successful and a plan for reporting negative results if it is not successful?
  4. Are plans for subsequent funding clearly defined? Is the future project feasible as a basis for future R01- or P01-type applications? Please note that this is not a requirement of methods development proposals.

Stakeholder Engagement Plan Review

Regardless of the selected translational phase, each stakeholder engagement strategy will be reviewed by at least two members of either the Tufts CTSI Stakeholder Expert Panel or the Tufts CTSI Stakeholder and Community Engagement Task Force. The former comprises community members with diverse professional and cultural backgrounds who will primarily review stakeholder engagement strategies of proposals along the T2-T4 translational continuum. The latter includes Tufts CTSI staff and faculty with expertise in early-stage translational research who will review stakeholder engagement strategies of proposals in the T.5 and T1 phases. Both reviewer groups will evaluate the applicants’ ability to identify relevant stakeholders, engage these stakeholders in the research project, and articulate relevance of project outcomes to the identified stakeholder groups and the public based on the criteria below.

Stakeholders – ability to identify key stakeholder groups and determine the role they play or may play in the proposed research project.

  1. Are key stakeholder groups directly and indirectly affected by the proposed research project or that affect the proposed research project clearly defined and described?
  2. Does the plan provide a compelling rationale for engaging identified stakeholders in the proposed research project? If so, are these core individuals or groups defined using an explicit criterion (e.g., specific condition, focus of study, common field of interest, cultural or ethnic background, prior experience, geographic location)? Is the reasoning for their engagement clearly explained?
  3. If applicable, does the plan describe how the identified stakeholder groups have been involved in the conceptualization of the proposed project and development of research procedures (e.g., defining or refining research question(s), designing research protocol(s), and identifying research participants)?
  4. Is the existing or proposed involvement of the identified stakeholder groups described adequately to assess the role(s) these groups currently play or will play in the project?
  5. Does the plan offer evidence that the identified stakeholder groups will play a meaningful role in the project?
  6. If a letter of support is provided, does it reflect an authentic connection with the project’s principal investigator and/or research team? Does it offer an insight into how those involved have collaborated in the past or will collaborate in the future?

Approach – rigor of the proposed stakeholder engagement plan to meet the proposed objectives and goals.

  1. Does the plan support the overall objectives of the proposed research project?
  2. Does the plan describe how the knowledge, experience, and expertise of the identified stakeholders will be practically applied in the implementation of the proposed research project (e.g., recruitment of participants, data collection and/or analysis, interpretation of findings, verification of conclusions)? Does the proposed engagement strategy foster solid, bidirectional relationships that promote collaboration and an atmosphere of mutual trust and understanding?
  3. Is the proposed communication strategy adequate and tailored to meet the specific needs of each partnership with the identified stakeholder groups?
  4. As proposed, is the plan realistic? Does it have a high likelihood of being successfully implemented during the award’s one-year timeframe? If circumstances were to change, is the plan sufficiently flexible to allow for modifications?

Relevance – ability to demonstrate explicit relevance of the project and its outcomes to the identified stakeholder groups and the public.

  1. Does the plan provide a clear statement of the project’s specific relevance to the identified stakeholder groups? Is the statement supported by compelling and logical reasoning?
  2. Does the project address a pressing and/or overlooked health issue impacting the wellbeing of specific populations (or the public)?
  3. Does the plan describe the value and impact of the proposed research project to the identified stakeholder groups and/or specific populations?
  4. Does the plan explain how the identified stakeholder groups and/or specific populations will benefit from research outcomes? If so, does the plan specify what the applicant will do to ensure that there is appropriate follow-up?
  5. If the aims of the project are achieved, how likely is it that the project’s outcomes and results will be applied (e.g., implemented into clinical practice, used to advance translational science or to inform health policy, applied in the form of new interventions, treatments, or devices)?

Questions?

Please refer to our Frequently Asked Questions page.

Need further assistance? We are here to help. Please contact us at pilots@tuftsctsi.org.