Senior Architect, Model Performance and Post-Training

Other Jobs To Apply

No other job posts for this day.

Job Description:

  • Develop meticulous frameworks to measure and analyze coding model performance across code generation, debugging, code completion, repository-level reasoning, and multi-step software engineering tasks
  • Develop and optimize post-training pipelines for coding models, including supervised fine-tuning (SFT), preference optimization, reinforcement learning (RL), and outcome-based or process-based reward modeling
  • Analyze benchmark results, training dynamics, failure modes, and model behaviors to find opportunities for improvement
  • Improve model performance on sophisticated coding tasks through targeted training interventions, reward design, curriculum development, and iterative error analysis
  • Build scalable, reproducible infrastructure for training experiments, evaluation pipelines, and model performance analysis
  • Collaborate with research scientists, infrastructure engineers, and product teams to turn model insights into production-ready capabilities

Requirements:

  • Bachelor's degree (or equivalent experience) in Computer Science or AI
  • 12+ years of meaningful work experience
  • Excellent programming skills in Python
  • Strong understanding of large language models, transformer architectures, and training and evaluation of generative models
  • Foundational understanding of reinforcement learning algorithms and implementation, including MDPs, policies, and reward design
  • Deep understanding of software engineering fundamentals, code generation, automated testing, and program correctness
  • Proven track record to lead model development life cycles, including dataset versioning, experiment tracking, and evaluation pipelines
  • Experience training or post-training state-of-the-art coding models or models designed for software engineering agents
  • Experience with execution-based rewards, verifiable rewards, reward hacking mitigation, and automated code evaluation
  • Familiarity with coding benchmarks and evaluation environments such as SWE-bench

Benefits:

  • Highly competitive salaries
  • Comprehensive benefits package
  • Equity
  • Benefits for you and your family
Share Share
Buy Now

Remote Question and Answer

Virtual Assistant Question and Answer

Content Moderator Question and Answer

Customer Service Question and Answer

Video Game Tester Question and Answer

Apply Now →