FAR.AI

<h2>About Us</h2><a target="_blank" rel="noopener noreferrer ugc" href="http://FAR.AI">FAR.AI</a> is a non-profit AI research institute working to ensure advanced AI is safe and beneficial for everyone. Our mission is to facilitate breakthrough AI safety research, advance global understanding of AI risks and solutions, and foster a coordinated global response. Since our founding in July 2022, we've grown to <a target="_blank" rel="noopener noreferrer ugc" href="https://www.far.ai/about/team">50+ staff</a>, published <a target="_blank" rel="noopener noreferrer ugc" href="https://scholar.google.com/citations?user=FVJ24k8AAAAJ">40+ academic papers</a>, and convened leading <a target="_blank" rel="noopener noreferrer ugc" href="https://far.ai/events/">AI safety events</a>. Our work is recognized globally, with publications at premier venues such as NeurIPS, ICML including a <a target="_blank" rel="noopener noreferrer ugc" href="https://icml.cc/virtual/2026/oral/71065">Best Paper Honorable Mention in 2026</a>, and ICLR, and features in the <a target="_blank" rel="noopener noreferrer ugc" href="https://www.ft.com/content/175e5314-a7f7-4741-a786-273219f433a1">Financial Times</a>, <a target="_blank" rel="noopener noreferrer ugc" href="https://www.nature.com/articles/d41586-024-02218-7">Nature News</a>, <a target="_blank" rel="noopener noreferrer ugc" href="https://www.wired.com/story/jailbreaking-ai-models-google-anthropic-openai-spacexai/">Wired Magazine</a> and <a target="_blank" rel="noopener noreferrer ugc" href="https://www.technologyreview.com/2020/02/28/905615/reinforcement-learning-adversarial-attack-gaming-ai-deepmind-alphazero-selfdriving-cars/">MIT Technology Review</a>. We conduct pre-deployment testing on behalf of frontier developers such as OpenAI, independent evaluations for governments <a target="_blank" rel="noopener noreferrer ugc" href="https://www.far.ai/news/far-ai-selected-to-lead-eu-ai-act-cbrn-risk-consortium">including the EU AI Office</a>, and publish the <a target="_blank" rel="noopener noreferrer ugc" href="https://leaderboard.far.ai/">AI Security Leaderboard</a> based on our red-teaming expertise. We help steer and grow the AI safety field through <a target="_blank" rel="noopener noreferrer ugc" href="https://arxiv.org/abs/2405.06624">developing</a> <a target="_blank" rel="noopener noreferrer ugc" href="https://arxiv.org/abs/2506.20702">research</a> <a target="_blank" rel="noopener noreferrer ugc" href="https://www.researchgate.net/publication/396910034_Open_Technical_Problems_in_Open-Weight_AI_Model_Risk_Management">roadmaps</a> with renowned researchers such as Yoshua Bengio; running <a target="_blank" rel="noopener noreferrer ugc" href="https://www.far.ai/programs/far-labs">FAR.Labs</a>, an AI safety-focused co-working space in Berkeley housing 40+ members; and supporting the community through <a target="_blank" rel="noopener noreferrer ugc" href="https://www.far.ai/programs/grantmaking">targeted grants</a> to technical researchers.<a target="_blank" rel="noopener noreferrer ugc" href="http://FAR.AI">FAR.AI</a> is hiring a Research Lead to develop and lead a research agenda that reduces catastrophic risks from advanced AI. You'll build and lead a team executing this agenda — setting research direction, mentoring Members of Technical Staff to scale your vision, and remaining hands-on enough to write code and run experiments yourself. What counts is whether AI labs and governments actually change how they act; publications are useful but aren't the measure. Beyond your team, you can shape <a target="_blank" rel="noopener noreferrer ugc" href="http://FAR.AI">FAR.AI</a>'s broader work by directing millions of dollars in grants to external researchers extending your agenda, convening the people who can act on it, and influencing our independent testing and advising of AI companies and governments. This role suits you if you want high autonomy in an impact-driven environment, pursuing empirically grounded, scalable ML safety work.<h2>About FAR.Research</h2>We explore promising research directions in AI safety and scale up only those showing a high potential for impact. Once the core research problems are solved, we work to scale them to a minimum viable prototype, demonstrating their validity to AI companies and governments to drive adoption.Our recent and ongoing research includes:Adversarial Robustness: working to rigorously solve security problems through building a science of security and robustness for AI, from <a target="_blank" rel="noopener noreferrer ugc" href="https://far.ai/post/2023-07-superhuman-go-ais/">demonstrating superhuman systems can be vulnerable</a>, to <a target="_blank" rel="noopener noreferrer ugc" href="https://www.far.ai/news/does-robustness-improve-with-scale">scaling laws for robustness</a> and <a target="_blank" rel="noopener noreferrer ugc" href="https://arxiv.org/abs/2506.24068">jailbreaking constitutional classifiers</a>.Mechanistic Interpretability: <a target="_blank" rel="noopener noreferrer ugc" href="https://arxiv.org/abs/2502.12892">finding</a> <a target="_blank" rel="noopener noreferrer ugc" href="https://arxiv.org/abs/2508.16560">issues</a> <a target="_blank" rel="noopener noreferrer ugc" href="https://arxiv.org/abs/2505.11756">with</a> Sparse Autoencoders, probing deception using <a target="_blank" rel="noopener noreferrer ugc" href="https://arxiv.org/abs/2504.04072">AmongUs</a>, understanding <a target="_blank" rel="noopener noreferrer ugc" href="https://far.ai/post/2024-07-learned-planners/">learned planning</a> in SokoBan, and interpretable data attribution.Red-teaming: conducting pre- and post-release adversarial evaluations of frontier models (e.g. <a target="_blank" rel="noopener noreferrer ugc" href="https://x.com/ARGleave/status/1926138376509440433">Claude 4 Opus</a>, <a target="_blank" rel="noopener noreferrer ugc" href="https://cdn.openai.com/pdf/839e66fc-602c-48bf-81d3-b21eacc3459d/chatgpt_agent_system_card.pdf">ChatGPT Agent</a>, <a target="_blank" rel="noopener noreferrer ugc" href="https://cdn.openai.com/gpt-5-system-card.pdf">GPT-5</a>); developing <a target="_blank" rel="noopener noreferrer ugc" href="https://www.far.ai/news/defense-in-depth">novel attacks</a> to support this work.Evals: developing evaluations for new threat models, e.g. <a target="_blank" rel="noopener noreferrer ugc" href="https://arxiv.org/abs/2506.02873">persuasion</a> and <a target="_blank" rel="noopener noreferrer ugc" href="https://arxiv.org/abs/2507.11630">tampering risks</a>.Mitigating AI deception: studying when <a target="_blank" rel="noopener noreferrer ugc" href="https://www.far.ai/news/avoiding-ai-deception">lie detectors induce honesty or evasion</a>, and developing approaches to deception and sandbagging.We are particularly looking to add Research Leads in the following pod shapes:<ul><li>Applied Interpretability — using interpretability to tackle concrete safety problems (better probes, backdoor detection, deception monitoring), aiming for fast feedback loops, often in collaboration with our other pods. A new pod, greenfield.</li><li>Scalable Oversight / Alignment — methods that keep oversight robust as models become more capable than their supervisors: recursive reward modeling, debate, weak-to-strong generalization, process-based supervision.</li><li>Adversarial Robustness —extending our independent-testing work into deployed-system protection: better safety guardrails, pre-training safety interventions (initially CBRN misuse, especially for open-weight models), backdoor detection and mitigation, realistic cybersecurity evaluations, and loss-of-control deception evaluations.</li><li>Auditing / Evals — safety and alignment auditing: evaluation awareness (construct validity, safety-relevance, hyper-realistic evals), CoT monitorability and faithfulness training, black-box monitoring as a complement to our existing white-box work.</li><li>Persuasion / Epistemic Risks — science of epistemic risks and intervention points, persuasion's role in loss of control risks, evaluations and independent testing, connections to broader harmful manipulation, solutions and epistemic uplift. Building on our existing work and shaping your own agenda in the area.</li><li>Bring Your Own Agenda — an open track for senior researchers with a strong vision outside the pods above.</li></ul><h2>About the Role</h2>Research Leads define and own a research workstream end-to-end. Day-to-day, that means:<ul><li>Articulate a research agenda with a clear theory of change for mitigating catastrophic risks from human-level or superhuman AI systems, and/or vastly increasing the upside of such systems.</li><li>Grow and lead a team of technical staff in pursuit of this agenda, either directly or in partnership with an engineering co-lead.</li><li>Lead novel research projects where there may be unclear markers of progress or success.</li><li>Share your research findings through written content (e.g. academic publications, blog posts) and presentations (e.g. ML conferences, policymaker briefings) to drive adoption and change.</li><li>Mentor and coach junior team members in research skills and ML engineering.</li><li>Contribute to the <a target="_blank" rel="noopener noreferrer ugc" href="http://FAR.AI">FAR.AI</a> intellectual environment, for example by giving feedback on early-stage proposals.</li><li>Build a research field around your agenda through <a target="_blank" rel="noopener noreferrer ugc" href="http://FAR.AI">FAR.AI</a>'s grantmaking and events, and connect it to real-world deployments through our independent testing and government advising.</li></ul>This role would be a great fit if you:<ul><li>Want to work on the most impactful research directions, alongside mission-driven colleagues who'll push them forward with you.</li><li>Wish to pursue empirically grounded, scalable research directions that lean, technically strong teams can drive forward.</li><li>Value the ability to speak freely. We don't censor our researchers — we just ask that you protect confidential information and make clear when you're speaking personally or on behalf of the organization.</li><li>Want to advise and collaborate with governments, leading AI companies, and academics. We're a small organization that punches above its weight by working closely with these partners — through red-teaming, technical standards work, and research collaborations.</li></ul>This role would be a poor fit if you:<ul><li>Prefer solo IC research to leading a team toward a shared agenda. Some people can do great research that way, but in this role we're looking for someone whose research direction is strong enough that other excellent researchers want to build it with them.</li><li>Prioritize novelty and intellectual elegance over impact. We care about both — a mathematically elegant solution to AI safety would be wonderful — but when we have to choose, we choose what makes AI safer in practice.</li><li>Can only work with the largest compute clusters available at industry labs or need to be compensated with equity in a rapidly growing startup. We offer competitive salaries and sizable compute budgets on a cluster that we manage, but if you value these things over having a positive impact on the future, then you may be more suited to a for-profit lab.</li></ul><h2>About You</h2>To be a strong candidate for the Research Lead role, you likely:<ul><li>Have a strong existing research track record in AI or another highly technical subject (e.g. CS, math, physics).</li><li>Have a clear view of which safety research directions are likely to matter most over the next few years, and why.</li><li>Have either (a) a clear research agenda you'd pursue at <a target="_blank" rel="noopener noreferrer ugc" href="http://FAR.AI">FAR.AI</a>, with a theory of change explaining why it's valuable, or (b) a strong track record and a research space you'd sharpen into an agenda over your first months. We assess both paths against the same bar — depth of articulation at application is itself a signal about expected runway.</li><li>Have led a team, mentored graduate students, or supported early-career researchers through fellowship programs. Informal leadership in flatter organizations counts, as we’re more interested in experience than job titles.</li><li>Can effectively communicate novel methods and solutions to both technical and non-technical audiences.</li></ul><ul><li>Are not a new entrant to AI safety. We don't require a PhD or specific years of experience, but you should have engaged substantively with the field — through prior research, employment, or sustained independent contribution.</li></ul>It is preferable if you:<ul><li>Have an established publication record in AI safety.</li><li>Are comfortable writing grant proposals and navigating collaborations with other organizations or external research groups.</li></ul>If you are missing key leadership experience or are earlier in your career, we encourage you to consider the open <a target="_blank" rel="noopener noreferrer ugc" href="https://far.ai/careers/research-scientist-39dfd?ashby_jid=1bda4204-bfef-4a47-b72b-3562ec0bb3f9">Research Scientist</a> pathway and invite you to contribute to one of our existing agendas.We're also open to more senior versions of this role; simply apply or reach out to <a target="_blank" rel="noopener noreferrer nofollow" href="mailto:talent@far.ai">talent@far.ai</a>.<h2>Benefits* </h2><ul><li>Health Insurance - 94% of Insurance premium paid by Organization commencing within 1 month after your start date</li><li>Retirement - 401(k) plan with up to 2% match</li><li>PTO - 25 days Paid Time Off per year, accrued weekly and up to 10 days of paid sick leave per year</li><li>Paid Leave - Paid Bereavement, Family, Medical and Pregnancy Disability Leave</li><li>WFH Stipend &amp; Equipment - Work computer and stipend provided for eligible employees</li><li>Catered Meals (Berkeley Office Only) - Catered lunches and dinners on workdays at our office</li></ul>*(Available only to full-time employees located in the US)<h2>Logistics</h2>If based in the USA or Singapore, you will be an employee of <a target="_blank" rel="noopener noreferrer ugc" href="http://FAR.AI">FAR.AI</a> (501(c)(3) research non-profit / non-profit CLG). Outside the USA or Singapore, you will be employed via an EOR organisation on behalf of <a target="_blank" rel="noopener noreferrer ugc" href="http://FAR.AI">FAR.AI</a> or as a contractor.<ul><li>Location: Both remote and in-person (Berkeley, CA or Singapore) are possible. We sponsor visas for in-person employees, and can hire remotely in most countries.</li><li>Hours: Full-time (40 hours/week).</li><li>Application materials: Expect ~1–2 hours of preparation; most carries forward from prior job searches. We ask for a CV, a short research direction statement (the form supports both fully-formed agendas and developing ones), 2–3 selected works with a brief note on your personal contribution, and a short note on why <a target="_blank" rel="noopener noreferrer ugc" href="http://FAR.AI">FAR.AI</a> is a good home for your direction. If you advance to portfolio review, we'll ask for a full research direction statement (1–2 pages, with a theory of change to real-world implementation; ~1.5–2 hours, due within about a week).</li><li>Process: From application: a portfolio review (async), a 60-minute bilateral fit call, a research deep-day (~3.5 hours live, including an open talk to FAR research staff and two interview sessions), a 5-day paid work trial, structured reference calls, and a final decision panel. Typical elapsed time: 4–6 weeks. Total candidate time end-to-end is ~50 hours, with the paid work trial being the bulk. If a 5-day block isn't feasible for you, reach out — we can discuss alternatives.</li></ul>If you have any questions about the role, feel free to contact us at <a target="_blank" rel="noopener noreferrer nofollow" href="mailto:talent@far.ai">talent@far.ai</a>. Otherwise, if you don't have questions, the best way to ensure a proper review of your skills and qualifications is by applying directly via the application form. Please don't email us to share your resume (it won't have any impact on our decision). Thank you!

Research Lead

<h2>About Us</h2><a target="_blank" rel="noopener noreferrer ugc" href="http://FAR.AI">FAR.AI</a> is a non-profit AI research institute working to ensure advanced AI is safe and beneficial for everyone. Our mission is to facilitate breakthrough AI safety research, advance global understanding of AI risks and solutions, and foster a coordinated global response. Since our founding in July 2022, we've grown to <a target="_blank" rel="noopener noreferrer ugc" href="https://www.far.ai/about/team">50+ staff</a>, published <a target="_blank" rel="noopener noreferrer ugc" href="https://scholar.google.com/citations?user=FVJ24k8AAAAJ">40+ academic papers</a>, and convened leading <a target="_blank" rel="noopener noreferrer ugc" href="https://far.ai/events/">AI safety events</a>. Our work is recognized globally, with publications at premier venues such as NeurIPS, ICML including a <a target="_blank" rel="noopener noreferrer ugc" href="https://icml.cc/virtual/2026/oral/71065">Best Paper Honorable Mention in 2026</a>, and ICLR, and features in the <a target="_blank" rel="noopener noreferrer ugc" href="https://www.ft.com/content/175e5314-a7f7-4741-a786-273219f433a1">Financial Times</a>, <a target="_blank" rel="noopener noreferrer ugc" href="https://www.nature.com/articles/d41586-024-02218-7">Nature News</a>, <a target="_blank" rel="noopener noreferrer ugc" href="https://www.wired.com/story/jailbreaking-ai-models-google-anthropic-openai-spacexai/">Wired Magazine</a> and <a target="_blank" rel="noopener noreferrer ugc" href="https://www.technologyreview.com/2020/02/28/905615/reinforcement-learning-adversarial-attack-gaming-ai-deepmind-alphazero-selfdriving-cars/">MIT Technology Review</a>. We conduct pre-deployment testing on behalf of frontier developers such as OpenAI, independent evaluations for governments <a target="_blank" rel="noopener noreferrer ugc" href="https://www.far.ai/news/far-ai-selected-to-lead-eu-ai-act-cbrn-risk-consortium">including the EU AI Office</a>, and publish the <a target="_blank" rel="noopener noreferrer ugc" href="https://leaderboard.far.ai/">AI Security Leaderboard</a> based on our red-teaming expertise. We help steer and grow the AI safety field through <a target="_blank" rel="noopener noreferrer ugc" href="https://arxiv.org/abs/2405.06624">developing</a> <a target="_blank" rel="noopener noreferrer ugc" href="https://arxiv.org/abs/2506.20702">research</a> <a target="_blank" rel="noopener noreferrer ugc" href="https://www.researchgate.net/publication/396910034_Open_Technical_Problems_in_Open-Weight_AI_Model_Risk_Management">roadmaps</a> with renowned researchers such as Yoshua Bengio; running <a target="_blank" rel="noopener noreferrer ugc" href="https://www.far.ai/programs/far-labs">FAR.Labs</a>, an AI safety-focused co-working space in Berkeley housing 40+ members; and supporting the community through <a target="_blank" rel="noopener noreferrer ugc" href="https://www.far.ai/programs/grantmaking">targeted grants</a> to technical researchers.<h2>About the Applied White-Box Methods team</h2>The Applied White-Box Methods team develops, evaluates, and demonstrates methods that leverage model internals to improve the safety of AI systems. We work on diverse AI safety applications of white-box methods, from white-box control to evaluation awareness to shaping training dynamics to improve alignment.Black-box methods, such as chain-of-thought monitoring, work well for now, but we are quickly entering a world where black-box interventions and monitoring are insufficient. Interpretability research is still often early-stage, curiosity-driven work without realistic evaluations on applications that matter. The team bridges the gap between exploration and deployment by stress-testing white-box methods on real-world (e.g. long-context agentic coding) tasks and using this feedback loop to enable the deployment of better white-box methods at frontier scale.Our scope includes any method that uses model internals to understand, predict, or intervene on model behavior, not only what is conventionally called interpretability. Methods of current interest include natural language autoencoders and other activation explainers, activation oracles, steering, patching, and other activation-level interventions, influence functions and data attribution, and singular learning theory. We evaluate our methods against real baselines – strong black-box methods and activation probes – to be able to make an honest case that the methods are worth implementing, or conclude that simpler methods work better for now.AI Research Automation. AI will soon automate most of the hill-climbing in the research process. We anticipate this by focusing our effort and judgement on defining realistic evaluations with Goodhart-resistant metrics. With the evaluation framework properly built, we can pour vast amounts of AI labor into method development and iteration without overfitting. We believe this is the way to scale white-box research into the age of RSI.Realistic, large-scale models. Unrealistic models lead to unconfident conclusions about which methods do and do not work. We leverage FAR's shared compute and infrastructure to work on organisms that come out of pipelines a frontier lab could plausibly have run: for example, reward seekers trained by RL in broken environments, with other contaminated data mixed in to induce other misalignments.Realistic, large-scale evaluations. We will primarily study long-context agentic coding as this is where most of the risk currently lies. In addition to the classic AI control sabotage settings, we will also study reward hacking, sandbagging, and research tampering, which are some of the key failure modes that matter during RSI.Practical monitors and interventions. Deployment of new methods has real costs for AI developers. Part of our focus on real-world applications is ensuring that methods are simple and efficient enough to deploy at frontier scale.<h2>About the Role</h2>As a Research Scientist on the team, you will take ownership of and accelerate the team's research agenda, publish findings broadly, and engage with the AI alignment community. You are encouraged to propose new directions within the team's agenda. You are welcome and encouraged to attend relevant conferences and other community events, and leverage <a target="_blank" rel="noopener noreferrer ugc" href="http://FAR.AI">FAR.AI</a>’s existing comprehensive infrastructure for events convening and government relations as you see fit. Beyond <a target="_blank" rel="noopener noreferrer ugc" href="http://FAR.AI">FAR.AI</a>, you can work with national AI safety institutes, frontier model developers, and top academics.<h2>About You</h2>Two backgrounds are particularly well suited to this role: researchers with some experience in more fundamental interpretability who want to evaluate and refine those methods in realistic settings, including agentic coding, long contexts, and realistic threat models; and researchers from evaluations, AI control, red-teaming, or reinforcement learning who have begun working with model internals and want to deepen that work. In both cases, prior hands-on experience with white-box methods and a demonstrated interest in their practical application are preferred. (Note that this role is not well suited to researchers whose primary interest is interpretability as a foundational science.)If you are new to AI Safety research, that is also ok! But please be prepared to explain how your previous research experience (e.g. in applied ML) could be leveraged for our work, and how you are engaging with the field of technical AI safety research today.We are hiring across a range of levels of experience. You may already have some of the following, though not necessarily all.<ul><li>Hands-on experience applying at least one white-box method to a real model (activation explainers, SAEs, steering, attribution, influence functions, probes, or similar), and an informed view of its limitations.</li><li>A track record in AI safety: a paper, a fellowship project, or substantive public writing. Alumni of programs such as MATS, Astra, Anthropic Fellows, SPAR, Pivotal, LASR or similar programs are especially encouraged to apply.</li><li>Experience with evaluations, AI control, red-teaming, reinforcement learning, or post-training of LLMs. Experience running large training runs is a plus.</li><li>The ability to communicate novel methods and results clearly to technical and non-technical audiences.</li><li>A PhD or several years of research experience in computer science, machine learning, physics, statistics, or a related field.</li><li>Previous experience in applied ML for other fields: e.g. biology, chemistry, materials science, robotics, etc.</li></ul><h2>Benefits*</h2><ul><li>Health Insurance - 94% of Insurance premium paid by Organization commencing within 1 month after your start date</li><li>Retirement - 401(k) plan with up to 2% match</li><li>PTO - 25 days Paid Time Off per year, accrued weekly and up to 10 days of paid sick leave per year</li><li>Paid Leave - Paid Bereavement, Family, Medical and Pregnancy Disability Leave</li><li>WFH Stipend &amp; Equipment - Work computer and stipend provided for eligible employees</li><li>Catered Meals (Berkeley Office Only) - Catered lunches and dinners on workdays at our office</li></ul>*(Available only to full-time employees located in the US)<h2>Logistics</h2><ul><li>If based in the USA, you will be an employee of <a target="_blank" rel="noopener noreferrer ugc" href="http://FAR.AI">FAR.AI</a>, a 501(c)(3) research non-profit. Outside the USA, you will be an employee of an EoR organization on behalf of <a target="_blank" rel="noopener noreferrer ugc" href="http://FAR.AI">FAR.AI</a>. </li><li>Location: Berkeley, CA. This role is in-person; we strongly prefer candidates who are in the Bay Area or willing to relocate, and we sponsor visas for in-person employees. We will consider remote arrangements for exceptional candidates.</li><li>Hours: full-time (40 hours/week).</li><li>Compensation: $150,000–$250,000/year depending on experience and location, plus work-related travel and equipment expenses. Catered lunch and dinner at our Berkeley office.</li><li>Application process: interviews with members of our technical staff, followed by a paid work trial of up to one week. If you are not available for a work trial we may be able to find alternative ways of assessing fit.</li></ul>If you have any questions about the role, feel free to contact us at <a target="_blank" rel="noopener noreferrer nofollow" href="mailto:talent@far.ai">talent@far.ai</a>. Otherwise, if you don't have questions, the best way to ensure a proper review of your skills and qualifications is by applying directly via the application form. Please don't email us to share your resume (it won't have any impact on our decision). Thank you!

FAR.AI

Open jobs at FAR.AI

Research Lead

Research Scientist, Applied White-Box Methods

Working for FAR.AI?

Featured companies