Ifm Us

<div>
<div>
<p><span>About the Institute of Foundation Models</span><span> </span></p>
</div>
<div>
<p><span>We are a dedicated research lab for building, understanding, using, and risk-managing foundation models. Our mandate is to advance research, nurture the next generation of AI builders, and drive transformative contributions to a knowledge-driven economy.</span><span> </span></p>
</div>
<div>
<p><span>As part of our team, you’ll have the opportunity to work on the core of cutting-edge foundation model training, alongside world-class researchers, data scientists, and engineers, tackling the most fundamental and impactful challenges in AI development. You will participate in the development of groundbreaking AI solutions that have the potential to reshape entire industries. Strategic and innovative problem-solving skills will be instrumental in establishing MBZUAI as a global hub for high-performance computing in deep learning, driving impactful discoveries that inspire the next generation of AI pioneers.</span><span> </span></p>
</div>
</div>
<h2>The Role</h2><div><span>We are seeking an Admin Operations Coordinator to manage front desk operations and provide administrative and recruiting support to our Operations and Talent Acquisition teams.  This role is evenly split: 50% of your time will be dedicated to front desk management and greeting guests, candidates, and employees at our Sunnyvale office. The remaining 50% will focus on candidate travel management, recruiting coordination, general administrative tasks, and collaborating with our Workplace Operations Coordinator on day-to-day office needs.  The ideal candidate is customer-service-driven, highly organized, and thrives in a fast-paced environment where attention to detail and a welcoming demeanor are essential.</span><span> </span></div>
<h2>Key Responsibilities</h2><div>
<div>
<p><span>1. Front Desk &amp; Guest Experience (50%)</span><span> </span></p>
</div>
<div>
<ul>
<li>
<p><span>Reception &amp; Host:</span><span> Serve as the primary onsite point of contact, managing the front desk during core business hours to greet employees, job candidates, visitors, and vendors. </span><span> </span></p>
</li>

</ul></div>
<div>
<ul>
<li>
<p><span>Visitor Access &amp; Security:</span><span> Manage visitor check-in procedures, issue guest passes, coordinate with internal hosts, and ensure compliance with building security guidelines. </span><span> </span></p>
</li>

</ul></div>
<div>
<ul>
<li>
<p><span>Front-of-House Maintenance:</span><span> Keep the reception area, lobby, and primary candidate/guest spaces immaculate, organized, and welcoming. </span><span> </span></p>
</li>

</ul></div>
<div>
<ul>
<li>
<p><span>Deliveries &amp; Mail:</span><span> Receive, log, and distribute all incoming mail, packages, and deliveries; assist with outgoing shipments.</span><span> </span></p>
</li>

</ul></div>
<div>
<p><span> </span></p>
</div>
<div>
<p><span>2. Candidate Travel, Recruiting &amp; Admin Operations Support (50%)</span><span> </span></p>
</div>
<div>
<ul>
<li>
<p><span>Candidate Travel Coordination:</span><span> Book and coordinate candidate travel arrangements (flights, hotels, rental cars/ground transportation, and expense/reimbursement guidance) for onsite interview loops. </span><span> </span></p>
</li>

</ul></div>
<div>
<ul>
<li>
<p><span>Recruiting Logistics:</span><span> Schedule candidate interviews, manage complex interviewing calendars, coordinate candidate communications, and host onsite interview loops. </span><span> </span></p>
</li>

</ul></div>
<div>
<ul>
<li>
<p><span>Candidate &amp; Guest Onsite Experience:</span><span><span> Coordinate </span><span>candidate</span><span> arrival </span><span>logistics</span><span>, escort candidates to interview rooms, and ensure an exceptional, high-touch interview experience. </span></span><span> </span></p>
</li>

</ul></div>
<div>
<ul>
<li>
<p><span>Office &amp; Vendor Operations:</span><span><span> Partner with the Workplace Operations Coordinator to order office supplies, coordinate catering/lunches, prep meeting spaces, and </span><span>assist</span><span> with vendor service visits. </span></span><span> </span></p>
</li>

</ul></div>
<div>
<ul>
<li>
<p><span>Process Improvement:</span><span> Identify opportunities to continuously improve travel booking workflows, recruiting coordination, and front office operational efficiency.</span><span> </span></p>
</li>

</ul></div>
<div>
<ul>
<li>
<p><span>Cross-Functional Support:</span><span> Provide administrative and operational support across Operations, Human Resources, Recruiting, Finance, IT, and Workplace Services as business needs arise. Assist with special projects and perform other duties as reasonably assigned. </span><span> </span></p>
</li>

</ul></div>
</div>
<h2>Requirements</h2><div>
<div>
<ul>
<li>
<p><span>Experience:</span><span> 2–4 years of experience in front desk reception, office administration, recruiting coordination, or customer service. </span><span> </span></p>
</li>

</ul></div>
<div>
<ul>
<li>
<p><span>Calendar &amp; Travel Coordination:</span><span><span> </span><span>Demonstrated</span><span> experience coordinating complex interview calendars and managing corporate travel bookings for candidates or team members. </span></span><span> </span></p>
</li>

</ul></div>
<div>
<ul>
<li>
<p><span>Customer Service Mindset:</span><span> Outstanding interpersonal and communication skills with a polished, welcoming, and professional demeanor. </span><span> </span></p>
</li>

</ul></div>
<div>
<ul>
<li>
<p><span>Software Skills:</span><span><span> </span><span>Proficiency</span><span> with Microsoft Office / Microsoft 365. </span></span><span> </span></p>
</li>

</ul></div>
<div>
<ul>
<li>
<p><span>ATS Experience:</span><span><span> Hands-on experience with an applicant tracking system (Lever, Greenhouse, Ashby, or similar) preferred.</span><span> </span></span><span> </span></p>
</li>

</ul></div>
<div>
<ul>
<li>
<p><span>Multi-tasking:</span><span><span> Ability to balance front-desk duties with administrative and travel </span><span>logistics</span><span> in a fast-paced environment. </span></span><span> </span></p>
</li>

</ul></div>
<div>
<ul>
<li>
<p><span>Education:</span><span> Bachelor's degree preferred.</span><span> </span></p>
</li>

</ul></div>
</div>

Admin Operations Coordinator

<div>
<div><strong>About the Institute of Foundation Models</strong></div>
<div> </div>
<div>The Institute of Foundation Models is dedicated to advancing the science and engineering of large-scale AI systems. Our researchers and engineers develop cutting-edge foundation models while pushing the limits of high-performance computing and efficient AI inference. By combining deep expertise in machine learning, systems engineering, and hardware optimization, we build scalable AI solutions that drive scientific discovery and real-world impact.</div>
<div>As part of the team, interns work alongside world-class researchers and performance engineers to optimize the execution of large-scale foundation models on next-generation NVIDIA GPU architectures. This internship provides hands-on experience in low-level GPU performance analysis, kernel optimization, and hardware-aware inference acceleration.</div>
</div>
<h2>Key Responsibilities</h2><div>
<div>This intensive internship offers a unique opportunity to contribute to the development of a simulator and profiling framework for foundation model inference on NVidia GPUs.</div>
<div>Responsibilities include:</div>

<li>
<div>Develop analytical performance models for GPU kernels and inference workloads.</div>
</li>
<li>
<div>Build and validate a simulator to estimate theoretical hardware performance limits.</div>
</li>
<li>
<div>Compare measured kernel performance against architectural peak throughput.</div>
</li>
<li>
<div>Identify performance bottlenecks in compute, memory, communication, and scheduling.</div>
</li>
<li>
<div>Analyze GPU execution using NVIDIA Nsight Systems and Nsight Compute.</div>
</li>
<li>
<div>Investigate PTX and SASS code generation to understand low-level execution behavior.</div>
</li>
<li>
<div>Collaborate with researchers and engineers to optimize inference kernels for transformer-based models.</div>
</li>
<li>
<div>Evaluate utilization of Tensor Cores, memory bandwidth, caches, and instruction pipelines.</div>
</li>
<li>
<div>Design profiling methodologies for Hopper and Blackwell architectures.</div>
</li>
<li>
<div>Document findings and provide actionable recommendations for performance improvements.</div>
</li>

</div>
<h2>Academic Qualifications</h2><div>Currently pursuing a degree in Computer Science, Computer Engineering, Electrical Engineering, Artificial Intelligence, High-Performance Computing, or a related quantitative discipline.</div>
<h2>Preferred Qualifications</h2><div>

<li>
<div>Experience with CUDA programming and GPU kernel development.</div>
</li>
<li>
<div>Understanding of NVIDIA GPU architecture and memory hierarchy.</div>
</li>
<li>
<div>Familiarity with performance profiling tools such as Nsight Systems and Nsight Compute.</div>
</li>
<li>
<div>Knowledge of PTX, SASS, and low-level GPU execution.</div>
</li>
<li>
<div>Experience optimizing CUDA kernels for throughput and latency.</div>
</li>
<li>
<div>Understanding of roofline analysis, performance modeling, and hardware utilization metrics.</div>
</li>
<li>
<div>Experience with deep learning frameworks such as PyTorch or TensorFlow.</div>
</li>
<li>
<div>Strong programming skills in C++, CUDA, and Python.</div>
</li>

</div>
<h2>Desired Skills</h2><div>

<li>
<div>Performance engineering mindset.</div>
</li>
<li>
<div>Strong analytical and debugging abilities.</div>
</li>
<li>
<div>Interest in AI systems, inference optimization, and hardware-software co-design.</div>
</li>
<li>
<div>Ability to work independently on research and engineering challenges.</div>
</li>
<li>
<div>Excellent written and verbal communication skills.</div>
</li>

</div>

Inference Optimization Intern – Performance Modeling

<div>
<div>
<p><strong><span>About the Institute of Foundation Models</span></strong></p>
</div>
<p>The Institute of Foundation Models is a dedicated research lab focused on building, understanding, using, and risk-managing foundation models. Our mission is to advance AI research, support the next generation of AI builders, and develop impactful systems that improve how frontier models are trained, evaluated, deployed, and governed.</p>
<p>As part of our team, you will work closely with researchers, machine learning engineers, data scientists, software engineers, and product teams on some of the most important challenges in AI development. You will contribute to systems that help measure model quality, identify failure modes, and improve the reliability, safety, and readiness of model releases.</p>
</div>
<h2>The Role</h2><div>
<p>We are looking for an <strong>Eval360 - Error Analysis Engineer</strong> to help build, improve, and operate Eval360, an evaluation service that serves as a quality gate for AI models. This person will focus specifically on <strong>error analysis</strong>: understanding where models fail, why they fail, how those failures should be categorized, and how evaluation systems can better detect, measure, and prevent these issues before models are released.</p>
<p>You will collaborate with researchers, machine learning engineers, product managers, data scientists, and platform teams to develop AI evaluation applications and internal tools based on next-generation AI research. You will be part of a cross-functional team responsible for the full software development lifecycle, from requirements gathering and system design to implementation, deployment, monitoring, debugging, documentation, and continuous improvement.</p>
<p>The ideal candidate is comfortable working across the stack, including front-end interfaces for reviewing errors, back-end evaluation pipelines, data analysis workflows, model evaluation infrastructure, databases, dashboards, and APIs. This person should have strong software engineering skills, excellent analytical judgment, and the ability to turn ambiguous model failures into structured insights that improve evaluation quality.</p>
</div>
<h2>Key Responsibilities</h2><div>
<p><span>•</span>  <span>Collaborate with researchers, machine learning engineers, data scientists, product managers, and internal stakeholders to implement innovative software solutions for Eval360 and related model evaluation workflows.</span></p>
<p><span>•</span>  <span>Build and improve Eval360 as an evaluation service that acts as a quality gate for model development, model comparison, and model release decisions.</span></p>
<p><span>•</span>  <span>Perform deep error analysis on model outputs, including identifying failure patterns, categorizing issues, tracing root causes, and proposing improvements to evaluation methodology.</span></p>
<p><span>•</span>  <span>Develop tools, workflows, and dashboards that make it easier for researchers and engineers to inspect model failures, compare model behavior, and understand quality regressions.</span></p>
<p><span>•</span>  <span>Design and implement client-side and server-side architecture for evaluation review systems, error analysis interfaces, reporting tools, and internal evaluation applications.</span></p>
<p><span>•</span>  <span>Develop responsive, usable interfaces that support error triage, annotation review, evaluation debugging, and model quality investigation.</span></p>
<p><span>•</span>  <span>Build and maintain back-end services, APIs, data pipelines, and integrations that support evaluation execution, results storage, analysis, and reporting.</span></p>
<p><span>•</span>  <span>Test software to ensure responsiveness, correctness, reliability, and efficiency across evaluation workflows.</span></p>
<p><span>•</span>  <span>Troubleshoot, debug, and upgrade evaluation systems, including identifying issues in data processing, evaluation metrics, model output handling, job orchestration, and user-facing analysis tools.</span></p>
<p><span>•</span>  <span>Create and maintain security, access control, and data protection settings for evaluation data, model outputs, annotations, and internal tooling.</span></p>
<p><span>•</span>  <span>Write clear technical documentation for Eval360 systems, error taxonomies, evaluation workflows, debugging procedures, and user-facing tools.</span></p>
<p><span>•</span>  <span>Work with researchers, data scientists, analysts, and machine learning engineers to improve evaluation quality, model diagnostics, and failure-mode visibility.</span></p>
<p><span>•</span>  <span>Keep track of new development tools, evaluation frameworks, model analysis methods, data quality techniques, and architectures relevant to AI evaluation systems.</span></p>
<p><span>•</span>  <span>Contribute to the design of error taxonomies, evaluation rubrics, quality thresholds, regression detection methods, and model readiness criteria.</span></p>
<p><span>•</span>  <span>Help ensure Eval360 produces reliable, interpretable, and actionable signals for model quality gates.</span></p>
<p><span>•</span>  <span>Contribute to research publications, technical reports, internal knowledge sharing, and external presentations where appropriate.</span></p>
<p><span>•</span>  <span>Contribute to intellectual property and thought leadership in AI evaluation, error analysis, model quality measurement, and evaluation infrastructure.</span></p>
<p><span>•</span>  <span>Perform all other duties as reasonably directed by the line manager that are aligned with these functional objectives.</span></p>
</div>
<h2>Academic Qualifications</h2><div>
<p><span>•</span>  <span>Bachelor's degree in Computer Science, Machine Learning, Data Science, Software Engineering, Statistics, or a related technical field required.</span></p>
<p><span>•</span>  <span>Master's or Ph.D. in Computer Science, Machine Learning, Artificial Intelligence, Data Science, or a related field preferred.</span></p>
</div>
<h2>Professional Experience</h2><div>
<p><span>•</span>  <span>Proven experience as a Software Engineer, Full Stack Developer, Machine Learning Evaluation Engineer, Data Scientist, AI Engineer, or similar role.</span></p>
<p><span>•</span>  <span>Experience building software systems for AI, machine learning, data analysis, evaluation, annotation, experimentation, or model monitoring.</span></p>
<p><span>•</span>  <span>Experience working with AI algorithms and the ability to develop systems that accommodate AI-related requirements.</span></p>
<p><span>•</span>  <span>Experience performing error analysis, model evaluation, data quality analysis, or failure-mode investigation for machine learning or language model systems.</span></p>
<p><span>•</span>  <span>Experience developing internal applications, dashboards, review tools, or web-based workflows for technical users.</span></p>
<p><span>•</span>  <span>Familiarity with common software stacks, including front-end frameworks, back-end services, databases, APIs, and cloud or internal infrastructure.</span></p>
<p><span>•</span>  <span>Familiarity with GitHub, Git, CI/CD workflows, and collaborative software development practices.</span></p>
<p><span>•</span>  <span>Knowledge of front-end languages and libraries such as HTML, CSS, JavaScript, TypeScript, React, Angular, or similar technologies.</span></p>
<p><span>•</span>  <span>Knowledge of back-end languages and frameworks such as Python, Java, C#, Node.js, FastAPI, Flask, Django, or similar technologies.</span></p>
<p><span>•</span>  <span>Familiarity with databases such as MySQL, PostgreSQL, MongoDB, or other structured and unstructured data stores.</span></p>
<p><span>•</span>  <span>Familiarity with evaluation frameworks, experiment tracking systems, data pipelines, or machine learning infrastructure is strongly preferred.</span></p>
<p><span>•</span>  <span>Ability to analyze complex model outputs and translate qualitative failures into structured, measurable categories.</span></p>
<p><span>•</span>  <span>Strong problem-solving and troubleshooting skills, especially for ambiguous technical issues involving models, data, metrics, and software systems.</span></p>
<p><span>•</span>  <span>Effective communication and collaboration skills, with the ability to work across research, engineering, data, and product teams.</span></p>
<p><span>•</span>  <span>Strong attention to detail and a high bar for evaluation quality, reliability, and interpretability.</span></p>
</div>
<h2>Preferred Qualifications</h2><div>
<p><span>•</span>  <span>Experience with large language models, foundation models, multimodal models, or model evaluation systems.</span></p>
<p><span>•</span>  <span>Experience designing or using error taxonomies, evaluation rubrics, benchmark datasets, human evaluation workflows, or automated grading systems.</span></p>
<p><span>•</span>  <span>Experience with Python-based data analysis tools such as pandas, NumPy, Jupyter, or similar.</span></p>
<p><span>•</span>  <span>Experience with visualization or dashboarding tools for model quality analysis.</span></p>
<p><span>•</span>  <span>Experience with distributed systems, job queues, workflow orchestration, or large-scale data processing.</span></p>
<p><span>•</span>  <span>Experience working in a research environment or with fast-moving AI product and model teams.</span></p>
</div>

Eval360 - Error Analysis Engineer

<div><span>About the Institute of Foundation Models</span></div>
<div>We are a dedicated research lab for building, understanding, using, and risk-managing foundation models. Our mandate is to advance research, nurture the next generation of AI builders, and drive transformative contributions to a knowledge-driven economy.</div>
<div> </div>
<div><strong>As part of our team, you’ll have the opportunity to work on the core of cutting-edge foundation model training, alongside world-class researchers, data scientists, and engineers, tackling the most fundamental and impactful challenges in AI development. </strong>You will participate in the development of groundbreaking AI solutions that have the potential to reshape entire industries. Strategic and innovative problem-solving skills will be instrumental in establishing MBZUAI as a global hub for high-performance computing in deep learning, driving impactful discoveries that inspire the next generation of AI pioneers.</div>
<div> </div>
<div><strong>Key Responsibilities </strong></div>
<div>This <strong>intensive, short-term internship</strong> offers a unique opportunity for aspiring AI engineers to;</div>
<div>• Collect, prepare and processing training and validation datasets.</div>
<div>• Assist in designing and tweaking model architectures.</div>
<div>• Implement testing frameworks to assess model performance.</div>
<div>• Analyze model outputs for accuracy and bias.</div>
<div>• Participate in the review and refinement of AI algorithms.</div>
<div>• Document findings and procedures for team collaboration.</div>
<div> </div>
<div><strong>Academic Qualifications</strong></div>
<div>Currently pursuing or recently completed a Master's, or Ph.D. in Computer Science, Data Science, Artificial Intelligence, Applied Mathematics, Statistics, or a related quantitative field.</div>

AI Research Internship - WM

<div>
<div>
<p><strong><span>About the Institute of Foundation Models</span><span> </span></strong></p>
</div>
<div>
<p><span>The Institute of Foundation Models (IFM) is a dedicated research lab for building, understanding, using, and risk-managing foundation models. Our mandate is to advance research, nurture the next generation of AI builders, and drive transformative contributions to a knowledge-driven economy.</span><span> </span></p>
</div>
<div>
<p><span>As part of our team, you'll work at the core of cutting-edge foundation model training, alongside world-class researchers, data scientists, and engineers, tackling the most fundamental and impactful challenges in AI development. You'll help build groundbreaking AI systems with the potential to reshape entire industries, and contribute to establishing MBZUAI as a global hub for high-performance computing and deep learning.</span><span> </span></p>
</div>
<div>
<p><strong><span>About the role</span><span> </span></strong></p>
</div>
<div>
<p><span>The Agents team trains advanced agentic language models that use reasoning and tool use to complete real tasks on a computer. This is a specialist role at the center of the loop that drives those models: </span><span>the data we train on and the benchmarks we measure against.</span><span> </span></p>
</div>
<div>
<p><span>You'll own the agentic data pipeline end-to-end — sourcing and generating high-quality trajectories, tool-use data, and RL environments — and the evaluation suite that tells us, rigorously and reproducibly, what our agents can actually do. These two halves are inseparable: benchmarks expose where models fail, and targeted data closes the gap. The agents are only as good as the data they learn from and the evals that keep us honest, and this role owns both.</span><span> </span></p>
</div>
<div>
<p><span>This is a research scientist position for someone who wants depth in data and measurement rather than breadth across the whole stack. You should be the kind of person who reads through datasets line by line, distrusts a metric until it's been validated, and gets satisfaction from making an eval suite that nobody questions.</span><span> </span></p>
</div>
</div>
<h2>Key responsibilities</h2><div>
<div>
<div>
<p><strong><span>Benchmarking &amp; evaluation</span><span> </span></strong></p>
</div>
<div>
<ul>
<li>
<p><span>Design and run evaluations of agentic capabilities — multi-step reasoning, tool use, long-horizon planning, computer use, and safety properties — turning ambiguous notions of "intelligence" into defensible, reproducible metrics.</span><span> </span></p>
</li>

</ul></div>
<div>
<ul>
<li>
<p><span>Build and harden evaluation harnesses so benchmarks run reliably at scale against training checkpoints, with clear signal on regressions and model health.</span><span> </span></p>
</li>

</ul></div>
<div>
<ul>
<li>
<p><span>Run experiments characterizing how prompting, sampling, scaffolding, and environment design affect agentic performance on internal and public benchmarks.</span><span> </span></p>
</li>

</ul></div>
<div>
<ul>
<li>
<p><span>Diagnose anomalous eval results mid-training-run — determine whether the cause is the model, the data, the harness, or the infrastructure — and communicate the answer clearly.</span><span> </span></p>
</li>

</ul></div>
<div>
<p><strong><span>Agentic data</span><span> </span></strong></p>
</div>
<div>
<ul>
<li>
<p><span>Source, generate, and curate high-quality agentic training data: trajectories, tool-use traces, and task datasets for new capabilities.</span><span> </span></p>
</li>

</ul></div>
<div>
<ul>
<li>
<p><span>Design and scale RL environments and reward signals, and measure their impact on model performance.</span><span> </span></p>
</li>

</ul></div>
<div>
<ul>
<li>
<p><span>Manage technical relationships with external data vendors and domain experts, evaluating data quality and iterating quickly on feedback.</span><span> </span></p>
</li>

</ul></div>
</div>
<div>
<div>
<ul>
<li>
<p><span>Develop QA frameworks that catch reward hacking, label noise, and contamination, keeping data and benchmark quality high.</span><span> </span></p>
</li>

</ul></div>
<div>
<p><strong><span>Across both</span><span> </span></strong></p>
</div>
<div>
<ul>
<li>
<p><span>Contribute to technical reports, research publications, and open-source benchmarks and tooling.</span><span> </span></p>
</li>

</ul></div>
<div>
<ul>
<li>
<p><span>Partner with research and product teams to translate capability goals into measurable data and evaluation artifacts.</span><span> </span></p>
</li>

</ul></div>
</div>
</div>
<h2>Qualifications</h2><div>
<div>
<p><strong><span>Academic qualifications</span><span> </span></strong></p>
</div>
<div>
<ul>
<li>
<p><span>BS, MS, or PhD (or equivalent experience) in Computer Science, Machine Learning, or a related field.</span><span> </span></p>
</li>

</ul></div>
<div>
<p><strong><span>Minimum qualifications</span><span> </span></strong></p>
</div>
<div>
<ul>
<li>
<p><span>2+ years of experience with a clear emphasis on </span><span>evaluations and/or training-data curation</span><span> for ML systems (related areas: LLM training/fine-tuning, RL, or distributed ML systems).</span><span> </span></p>
</li>

</ul></div>
<div>
<ul>
<li>
<p><span>Strong Python and PyTorch development experience.</span><span> </span></p>
</li>

</ul></div>
<div>
<ul>
<li>
<p><span>Demonstrated experience designing and deep-diving into evaluations, </span><span>or</span><span> curating and generating training datasets — ideally both.</span><span> </span></p>
</li>

</ul></div>
<div>
<ul>
<li>
<p><span>Hands-on experience using LLM agents in your personal or professional work.</span><span> </span></p>
</li>

</ul></div>
<div>
<ul>
<li>
<p><span>A habit of reading through raw data and trajectories to understand them and spot issues, and an instinct to distrust a metric until it's validated.</span><span> </span></p>
</li>

</ul></div>
<div>
<p><strong><span>Preferred qualifications</span><span> </span></strong></p>
</div>
<div>
<ul>
<li>
<p><span>Experience with reinforcement learning, reward design, or RL environment construction for LLMs.</span><span> </span></p>
</li>

</ul></div>
<div>
<ul>
<li>
<p><span>Background in statistics and experimental design — a feel for signal-to-noise, statistical power, and contamination in evaluations.</span><span> </span></p>
</li>

</ul></div>
<div>
<ul>
<li>
<p><span>Experience with large-scale dataset sourcing, curation, and processing, including working with external vendors or domain experts.</span><span> </span></p>
</li>

</ul></div>
<div>
<ul>
<li>
<p><span>Strong knowledge of the literature on agent evaluation, RL, LLM reasoning, and tool use.</span><span> </span></p>
</li>

</ul></div>
<div>
<ul>
<li>
<p><span>Experience building or operating data pipelines and evaluation infrastructure reliable at scale (e.g., PyTorch, Ray).</span><span> </span></p>
</li>

</ul></div>
<div>
<ul>
<li>
<p><span>Experience evaluating or generating data for software-engineering or computer-use agents.</span><span> </span></p>
</li>

</ul></div>
<div>
<ul>
<li>
<p><span>Contributions to published research, public benchmarks, and/or open-source ML software.</span><span> </span></p>
</li>

</ul></div>
</div>
<h2>Representative projects</h2><div>
<div>
<ul>
<li>
<p><span>Stand up a new agentic benchmark from scratch — define the task, build the dataset and scoring, validate against known signals, and ship a view that makes the result legible to researchers and leadership.</span><span> </span></p>
</li>

</ul></div>
<div>
<ul>
<li>
<p><span>Build an RL environment for a new high-value capability: design the reward, generate and QA the trajectory data, and measure the lift on model performance.</span><span> </span></p>
</li>

</ul></div>
<div>
<ul>
<li>
<p><span>Diagnose a mid-training regression: an eval suite returns anomalous numbers and you determine whether it's the model, the harness, the data, or the infrastructure.</span><span> </span></p>
</li>

</ul></div>
<div>
<ul>
<li>
<p><span>Partner with an external data vendor or domain expert to source high-quality trajectories, then build the QA framework that keeps reward hacking and contamination out.</span><span> </span></p>
</li>

</ul></div>
<div>
<ul>
<li>
<p><span>Take a flaky distributed eval pipeline and make it reliable — better retries, better observability, faster feedback to researchers.</span><span> </span></p>
</li>

</ul></div>
</div>

Ifm Us

Open jobs at Ifm Us

Admin Operations Coordinator

Inference Optimization Intern – Performance Modeling

Eval360 - Error Analysis Engineer

AI Research Internship - WM

Research Scientist, Agentic Data & Benchmarking

Working for Ifm Us?

Featured companies