NBME is seeking a Director, Assessment Data Initiatives to focus on health professions education (HPE) assessments for NBME's Competency-based Assessment (CBA) unit.

Assessment in HPE is a rapidly evolving field. NBME's strategically oriented Growth and Innovation team seeks to enhance learners', schools', training programs' and health systems' ability to make data-driven decisions about individuals' and teams' progress toward competence and mastery of the essential skills and behaviors for effective patient care.

Capabilities for data capture, aggregation and analysis are also rapidly evolving. Competency-based education and assessment efforts seek to leverage developmental longitudinal data points support learning and to make informed decisions about progress. This includes the use of descriptive/narrative assessments which increasingly are being analyzed through tools such as natural language processing.

NBME is seeking an innovative individual with program leadership experience and a track record of innovation in HPE, who possesses a focus on the use of data for learner assessment and program evaluation. This individual will work with other NBME staff who possess expertise in medical education leadership and innovation, measurement science/psychometrics, and strategic development and execution - focused on delivering assessment systems solutions for HPE. .


To be successful in this position an individual will possess the following key competencies:

  • Data-Driven Decision Making: Effectively use insights, metrics, and data to guide strategic decisions.
  • Leadership: Effectively liaise with cross functional NBME staff in support of NBME assessment and data initiatives.
  • Communication Skills: Clearly and concisely express oneself in written and oral communications; exhibit effective listening and interpersonal skills; use a variety of communication vehicles.
  • Stakeholder Management: Proactively create and maintain strategic relationships.
  • Collaborative: Create an effective environment to build and maintain trust and respect among NBME collaborators.
  • Cultural Competence and DEI: Respect, understand and appreciate cultural differences and the value of diversity, equity, and inclusion; deliver insights from data projects which analyze and help to mitigate bias in assessment in support of the development of a diverse workforce


For more information and to apply please visit: https://nbme.applicantpro.com/jobs/2188152.html



  • Lead strategic initiatives focused on the use of assessment data in HPE, training, and practice. Such initiatives could include research-focused initiatives, collaborations with external innovators, schools, programs, and/or concept/prototype development.
    • Efforts above will be aimed at assisting learners, schools, and programs to make informed inferences using longitudinal data on performance, and will include formative assessments for learning focused on the development of growth-oriented health professionals. .
    • Target constructs to measure will focus on clinical reasoning, communication skills, systems competencies, and non-cognitive attributes.
  • Serve as a primary point of contact and lead for certain assessment and data-related projects as well as project collaborations with external investigators within NBME's consortia of HPE programs (NBME Assessment Alliance)
  • Collaborate with NBME scientific staff to support research initiative delivery in a timely manner
  • Work with other leaders in NBME's Growth and Innovation team to coordinate cross-cutting implications of strategic initiatives.
  • Liaise with NBME Marketing and Communication staff on market research initiatives as well internal/external communications related to data initiatives.
  • Leverage one's existing professional network in health professions assessment and evaluation to broaden NBME's prospective collaboration network.
  • Provide input into and assist with NBME's emerging data strategy
  • Philanthropic Programs. Track research progress and program impact of NBME's Stemmler Fund and other related initiatives. Communicate with Stemmler funded investigators periodically and as necessary.
  • Supervise CBA program administrative staff to ensure seamless operational aspects of NBME research-related philanthropic programs; partner with NBME program and administrative staff to ensure smooth operation of committee and program meetings.


  • Contribute to overall NBME data vision and data strategy to support formative and longitudinal assessment of skills and behaviors.
  • Identify and assess potential for new data services that leverage educational and workplace data.
  • Evaluate the effectiveness of NBME's assessment data efforts by developing and tracking a set of key metrics. As this is established, produce quarterly reports.
  • Use superior judgement to escalate issues to VP CBA and other program leadership staff
  • Assure efficient planning of activities, meetings (internal/external) and reports across collaborations that support NBME assessment data initiatives,
  • Deliver initiatives on time and within budget
  • Deliver timely reports of progress and budgetary tracking as relevant.
  • Generate accurate and trustworthy written materials to be shared with internal and external stakeholders, such as senior leadership and boards
  • Represent NBME with to external partners, health professions schools/programs and organizations with the highest degree of knowledge, professionalism, trust, and collaborative spirit.


Knowledge, Skills, and Abilities

  • Superior written and oral communication skills, as well as presentation skills
  • Outstanding organizational skills
  • Ability to engage with and collaborate within the Health Professions Education and Assessment and Evaluation Community (e.g., Societies of Directors of Research in Medical Education (SDRME)
  • Anticipation and creative resolution of challenges to ensure the success of initiatives and collaborations
  • Basic meeting planning skills - and ability to work with NBME meeting planning staff
  • High emotional intelligence; Excellent judgment in handling sensitive and confidential material, in particular assessment data.
  • In depth knowledge of HPE, assessment strategies and program evaluation, including metrics for UME and GME program accreditation as well as the structure and current challenges of the UME to GME transition
  • Understanding of grants programs and grants management, including call for applications, selection of collaborators, timely signing of agreements, management of time-bound projects and associated deliverables including progress reports from collaborators, payment to collaborators and grant recipients.
  • Knowledge of strategies for assessment for learning, adaptive learning/expertise, learning analytics.
  • Knowledge and perspective of program-based assessment and evaluation metrics, including some knowledge of clinical practice metrics, which may be available in the electronic health record.
  • Knowledge of quantitative and qualitative research methods; cognitive and non-cognitive assessments
  • Ability to organize and analyze large data sets of quantitative and qualitative data to generate new insights and identify data visualization opportunities for data services.
  • Ability to represent NBME's Competency-based Assessment activities externally at conference presentations



  • A minimum of 10 years of applicable experience
  • Experience with data-driven educational quality improvement efforts for learner assessment and/or program evaluation.
  • Track record of innovation in HPE space, particularly on data-related initiatives
  • Prior experience/roles within regional or national societies in health professions learner assessment and program evaluation; data science network is desirable
  • Prior experience with data reporting and accreditation requirements of health professions programs (e.g., LCME, ACGME AOA-COCA) is desirable.


  • Doctorate degree (i.e., PhD, EdD, ScD,, DO, MD) or a combination of equivalent education and experience.


About NBME:

NBME offers a versatile selection of high-quality assessments and educational services for students, professionals, educators, regulators, and institutions dedicated to the evolving needs of medical education and health care. To ensure our assessments meet the highest standards of quality, stay relevant and align to the current curriculum in medical schools and training programs, we rely on a wide network of collaborators. These include the volunteers who help develop our exam questions, the committees and panels who represent various groups within the medical education community, external researchers, and health profession organizations.

We are committed to meeting the needs of educators and learners globally with assessment products and expert services such as NBME® Subject ExaminationsCustomized Assessment ServicesSelf-Assessments, the International Foundations of Medicine® Program and Item Writing Workshops. Together with the Federation of State Medical Boards, NBME develops and manages the United States Medical Licensing Examination®, which measures the ability to apply knowledge and skills that form the basis of safe and effective patient care. Our Competency-based Assessment unit is focused on new methods as well as the optimization of assessment in the workplace and education.

As a result of leadership in ongoing research, innovative measurement practices and the exploration of forward-thinking assessment modalities and improvements, NBME advances assessment science. Our grant and funding opportunities further support this dedication to medical education and assessment science. We help develop the next generation of assessment professionals through our Summer Psychometric Internship Program. Through the Stemmler FundStrategic Educators Enhancement Fund and Latin America Grants Program, researchers and educators can continue to improve the assessment of health care professionals around the world.

NBME views diversity, equity, and inclusion (DEI) as foundational and enduring to our strategy and vision. We continue to focus on ensuring that our DEI work is impactful and ingrained in everything we do, including with our staff, culture, products and services, the Philadelphia community, and the broader medical education landscape. Our commitment manifests in our hiring and staff development, recruitment for committees, grants programs, design and review of our assessments, and involvement in our local and national communities.

Learn more about NBME at NBME.org.


The NBME offers competitive salaries, excellent benefits, and a rewarding work environment. Excellent Benefits include: Healthcare, Dental, Prescription, and Vision plans; 401(k) w/match, Tuition Reimbursement Plan, Commuter Benefit: Public Transit or Parking options. Remote Friendly Workplace.


COVID-19 Considerations:

Effective January 2022, being fully vaccinated against COVID-19 will be a condition of employment, subject to potential reasonable accommodations for legitimate medical or religious reasons which prevent such vaccination. Applicants who have received a conditional offer of employment will be requested to provide information about their COVID-19 vaccination status.


NBME is an equal opportunity employer as defined by the EEOC.