Applied Scientist III, AGI Responsible AI (RAI)

Written by the T&S Network from Amazon's posting.

Designs and builds responsible AI evaluation for large language models and generative AI, covering safety, fairness, robustness and trustworthiness of frontier models. The work sits in Amazon's AGI Responsible AI team, which turns evaluation results into model launch decisions across pretraining, post-training and product teams.

What you'd do

• Own the creation of evaluation frameworks, benchmarks and metrics for safety, fairness, robustness and harmful content.
• Build scalable automated evaluation pipelines using model-based and human-in-the-loop methods for large language models.
• Work with pretraining, post-training and product teams to convert evaluation findings into model fixes and launch calls.
• Carry out careful experiments and statistical analysis, and publish findings at leading venues.
• Mentor junior scientists and help lift the team's scientific standards.
• Advocate for responsible AI practices at every stage of model development.

What it asks for

• 3+ years of experience building machine learning models for business use.
• PhD, or Master's degree with 6+ years of applied research experience.
• Programming experience in Java, C++, Python or a related language.
• Hands-on experience with neural deep learning and machine learning methods.
• Experience with large-scale distributed systems such as Hadoop or Spark (preferred).

Our best summary of the posting, and it can be wrong. The employer's posting is what counts: read it in full on Amazonnorth_east before you apply. If we got something wrong, report this listing.