Nexxa.AI

<h2>About Nexxa</h2>Nexxa is building the best AI systems for heavy industries, enabling machines, systems and operations to think, decide and act autonomously across manufacturing, large-scale infrastructure, logistics and legacy environments.Our mission is to translate deep technical breakthroughs into operational reality, solving some of the hardest systems-level problems in industry.<div> </div><h2>Role Overview</h2>We’re looking for a Senior / Staff SDET to own quality for Nexxa’s AI agent systems, combining deep test automation with hands-on manual and exploratory testing. Our agents plan, call tools, and take multi-step actions autonomously in industrial environments. You’ll write production-grade code to build evaluation harnesses, simulation environments, and CI/CD quality gates, and you’ll also get hands-on with the agents yourself: probing behavior, reviewing traces, and validating releases where automation alone can’t judge whether the agent did the right thing.You’ll work closely with ML engineers, backend engineers, and Forward Deployed Engineers to define what “good” looks like for an agent operating in high-stakes industrial settings, then build and run the systems and processes that measure it continuously.<div> </div><h2>Key Responsibilities</h2>Automation and Infrastructure<ul><li>Design, build, and maintain test frameworks, evaluation harnesses, and regression suites for LLM-based agents, covering reasoning quality, tool-call correctness, task completion, and multi-turn coherence.</li><li>Build automated pipelines that run evals and test suites on every model, prompt, or tool-integration change, and integrate them into CI/CD as enforceable quality gates.</li><li>Develop simulators, mocks, and sandbox environments for industrial systems and tool APIs so agents can be tested safely, repeatably, and at scale.</li><li>Implement API, contract, integration, and end-to-end tests for the services, tools, and orchestration layers that agents depend on.</li><li>Develop performance, load, and reliability tests for agent workflows, including failure injection and resilience testing.</li></ul>Manual and Exploratory Testing<ul><li>Perform hands-on exploratory testing of agent behavior, including new capabilities, ambiguous or underspecified requests, long multi-turn sessions, and unusual operational scenarios that scripted tests don’t anticipate.</li><li>Manually review agent traces and outputs to judge reasoning quality, appropriateness of actions, and safety, and use those findings to improve automated checks.</li><li>Execute manual regression and release verification passes in staging and customer-like environments before new agent capabilities ship.</li><li>Conduct manual red-teaming sessions (prompt injection, jailbreaks, tool misuse, unsafe actions) in partnership with security teams, then convert successful attacks into automated test cases.</li><li>Support user acceptance testing and pilot rollouts with Forward Deployed Engineers, capturing real-world failures and feeding them back into test suites.</li><li>Write clear test plans, test charters, and bug reports, and maintain manual test documentation where automation isn’t yet practical or cost-effective.</li></ul>Datasets, Metrics, and Quality Strategy<ul><li>Create and manage golden datasets and labeled test sets, including edge cases, ambiguous inputs, and adversarial prompts specific to industrial and operational contexts, with tooling for versioning and drift detection.</li><li>Design rubrics and run human-in-the-loop evaluations where automated scoring isn’t reliable enough.</li><li>Define and instrument quality metrics beyond simple accuracy (groundedness, hallucination rate, task success rate, latency/cost tradeoffs, and safety violations) and build dashboards to track them over time.</li><li>Test agent behavior across the full action loop (planning, tool selection, tool execution, error recovery, and final output), not just the final response.</li><li>Investigate and triage failures where the root cause could be the model, the prompt, the tool/API, or the orchestration logic, and turn them into reproducible, actionable bug reports.</li><li>Establish quality bars, release criteria, and sign-off processes for new agent capabilities, blending automated gates with manual verification.</li><li>Decide what to automate and what to keep manual, and continually shift repeatable manual work into automation.</li><li>Mentor engineers on testing strategies and test-driven practices for probabilistic, LLM-driven systems.</li><li>Advocate for testability, observability, and tracing in agent architecture from day one.</li></ul><h2>Qualifications</h2><ul><li>5+ years as an SDET, QA engineer, or software engineer with a strong testing focus, with demonstrated ownership of test strategy and automation for complex distributed systems.</li><li>Proven experience in both manual and automated testing, including exploratory testing, test planning, and writing high-quality bug reports.</li><li>Strong programming skills (Python preferred) with a software engineering mindset: clean, maintainable, well-reviewed code for frameworks, data pipelines, and tooling.</li><li>Experience building test frameworks and infrastructure from the ground up, not just writing test cases within an existing framework.</li><li>Hands-on experience with CI/CD systems (e.g., GitHub Actions, GitLab CI, Jenkins) and containerized test environments (Docker, Kubernetes).</li><li>Experience testing APIs and microservices, including contract, integration, and end-to-end testing.</li><li>Hands-on experience testing LLM-based products, chatbots, or AI agents, and an understanding of why traditional deterministic assertions break down for generative systems.</li><li>Practical experience with eval frameworks or tooling (e.g., promptfoo, DeepEval, RAGAS, LangSmith) or a track record of building your own.</li><li>Understanding of how LLM agents work: prompting, tool/function calling, context management, RAG, memory, and orchestration frameworks.</li><li>Experience designing test data and labeled datasets, including sourcing, sampling, and managing dataset drift over time.</li><li>Familiarity with LLM-specific failure modes: hallucination, prompt injection, context poisoning, tool misuse, goal drift, and non-determinism.</li><li>Comfort operating in ambiguity, defining what “correct” means for a task when there’s no single right answer.</li><li>Strong written communication skills for turning fuzzy quality signals into clear, actionable findings for engineering and product stakeholders.</li></ul><h2>Preferred</h2><ul><li>Experience with human-in-the-loop evaluation workflows (labeling pipelines, inter-rater reliability, rubric design).</li><li>Experience with performance and load testing tools (e.g., k6, Locust) and chaos or fault-injection testing.</li><li>Background in ML/data science sufficient to read model evals and statistical significance.</li><li>Experience red-teaming or doing adversarial/security testing on ML systems.</li><li>Familiarity with observability and tracing tools for LLM applications (e.g., LangSmith, Arize, Langfuse, Weights &amp; Biases) and general observability stacks (OpenTelemetry, Grafana, Datadog).</li><li>Experience testing systems in industrial, IoT, or operational technology (OT) environments, including industrial protocols (OPC UA, MQTT, Modbus) or simulation and hardware-in-the-loop setups.</li><li>Experience supporting UAT, pilots, or customer-facing validation of complex products.</li><li>Experience setting up test and eval infrastructure from scratch at a startup or fast-moving team.</li></ul><h2>What We’re Looking For</h2><ul><li>An engineer who is equally comfortable writing a test framework and sitting down to break an agent by hand.</li><li>Someone who can build rigorous evaluation systems for problems that don’t have a single right answer, and who knows when human judgment is the right tool.</li><li>A systems thinker who enjoys turning ambiguous agent behavior into measurable, trustworthy signals.</li><li>A strong collaborator who partners well with ML engineers, backend engineers, and Forward Deployed teams.</li></ul><h2>Why Join <a target="_blank" rel="noopener noreferrer ugc" href="http://Nexxa.AI">Nexxa.AI</a>?</h2><ul><li>Innovative Environment: Play a critical role in transforming heavy industries through groundbreaking AI and automation technologies.</li><li>Collaborative Culture: Be part of a team that values innovation, discipline, and continuous improvement.</li><li>Professional Growth: Benefit from significant opportunities for career development and advancement.</li><li>Competitive Compensation: Enjoy a comprehensive salary and equity package reflective of your expertise and contributions.</li></ul>If you’re passionate about AI quality and eager to help define what trustworthy autonomous systems look like in heavy industry, we’d love to connect.

Senior Software Development Engineer in Test (SDET), QA - AI Systems

<h2>About Nexxa</h2>Nexxa is building the best AI systems for heavy industries, enabling machines, systems and operations to think, decide and act autonomously across manufacturing, large-scale infrastructure, logistics and legacy environments.Our mission is to translate deep technical breakthroughs into operational reality, solving some of the hardest systems-level problems in industry.<h2>Role Overview</h2>We’re looking for a Senior / Staff SDET to own quality for Nexxa’s AI agent systems, combining deep test automation with hands-on manual and exploratory testing. Our agents plan, call tools, and take multi-step actions autonomously in industrial environments. You’ll write production-grade code to build evaluation harnesses, simulation environments, and CI/CD quality gates, and you’ll also get hands-on with the agents yourself: probing behavior, reviewing traces, and validating releases where automation alone can’t judge whether the agent did the right thing.You’ll work closely with ML engineers, backend engineers, and Forward Deployed Engineers to define what “good” looks like for an agent operating in high-stakes industrial settings, then build and run the systems and processes that measure it continuously.<h2>Key Responsibilities</h2>Automation and Infrastructure<ul><li>Design, build, and maintain test frameworks, evaluation harnesses, and regression suites for LLM-based agents, covering reasoning quality, tool-call correctness, task completion, and multi-turn coherence.</li><li>Build automated pipelines that run evals and test suites on every model, prompt, or tool-integration change, and integrate them into CI/CD as enforceable quality gates.</li><li>Develop simulators, mocks, and sandbox environments for industrial systems and tool APIs so agents can be tested safely, repeatably, and at scale.</li><li>Implement API, contract, integration, and end-to-end tests for the services, tools, and orchestration layers that agents depend on.</li><li>Develop performance, load, and reliability tests for agent workflows, including failure injection and resilience testing.</li></ul>Manual and Exploratory Testing<ul><li>Perform hands-on exploratory testing of agent behavior, including new capabilities, ambiguous or underspecified requests, long multi-turn sessions, and unusual operational scenarios that scripted tests don’t anticipate.</li><li>Manually review agent traces and outputs to judge reasoning quality, appropriateness of actions, and safety, and use those findings to improve automated checks.</li><li>Execute manual regression and release verification passes in staging and customer-like environments before new agent capabilities ship.</li><li>Conduct manual red-teaming sessions (prompt injection, jailbreaks, tool misuse, unsafe actions) in partnership with security teams, then convert successful attacks into automated test cases.</li><li>Support user acceptance testing and pilot rollouts with Forward Deployed Engineers, capturing real-world failures and feeding them back into test suites.</li><li>Write clear test plans, test charters, and bug reports, and maintain manual test documentation where automation isn’t yet practical or cost-effective.</li></ul>Datasets, Metrics, and Quality Strategy<ul><li>Create and manage golden datasets and labeled test sets, including edge cases, ambiguous inputs, and adversarial prompts specific to industrial and operational contexts, with tooling for versioning and drift detection.</li><li>Design rubrics and run human-in-the-loop evaluations where automated scoring isn’t reliable enough.</li><li>Define and instrument quality metrics beyond simple accuracy (groundedness, hallucination rate, task success rate, latency/cost tradeoffs, and safety violations) and build dashboards to track them over time.</li><li>Test agent behavior across the full action loop (planning, tool selection, tool execution, error recovery, and final output), not just the final response.</li><li>Investigate and triage failures where the root cause could be the model, the prompt, the tool/API, or the orchestration logic, and turn them into reproducible, actionable bug reports.</li><li>Establish quality bars, release criteria, and sign-off processes for new agent capabilities, blending automated gates with manual verification.</li><li>Decide what to automate and what to keep manual, and continually shift repeatable manual work into automation.</li><li>Mentor engineers on testing strategies and test-driven practices for probabilistic, LLM-driven systems.</li><li>Advocate for testability, observability, and tracing in agent architecture from day one.</li></ul><h2>Qualifications</h2><ul><li>5+ years as an SDET, QA engineer, or software engineer with a strong testing focus, with demonstrated ownership of test strategy and automation for complex distributed systems.</li><li>Proven experience in both manual and automated testing, including exploratory testing, test planning, and writing high-quality bug reports.</li><li>Strong programming skills (Python preferred) with a software engineering mindset: clean, maintainable, well-reviewed code for frameworks, data pipelines, and tooling.</li><li>Experience building test frameworks and infrastructure from the ground up, not just writing test cases within an existing framework.</li><li>Hands-on experience with CI/CD systems (e.g., GitHub Actions, GitLab CI, Jenkins) and containerized test environments (Docker, Kubernetes).</li><li>Experience testing APIs and microservices, including contract, integration, and end-to-end testing.</li><li>Hands-on experience testing LLM-based products, chatbots, or AI agents, and an understanding of why traditional deterministic assertions break down for generative systems.</li><li>Practical experience with eval frameworks or tooling (e.g., promptfoo, DeepEval, RAGAS, LangSmith) or a track record of building your own.</li><li>Understanding of how LLM agents work: prompting, tool/function calling, context management, RAG, memory, and orchestration frameworks.</li><li>Experience designing test data and labeled datasets, including sourcing, sampling, and managing dataset drift over time.</li><li>Familiarity with LLM-specific failure modes: hallucination, prompt injection, context poisoning, tool misuse, goal drift, and non-determinism.</li><li>Comfort operating in ambiguity, defining what “correct” means for a task when there’s no single right answer.</li><li>Strong written communication skills for turning fuzzy quality signals into clear, actionable findings for engineering and product stakeholders.</li></ul><h2>Preferred</h2><ul><li>Experience with human-in-the-loop evaluation workflows (labeling pipelines, inter-rater reliability, rubric design).</li><li>Experience with performance and load testing tools (e.g., k6, Locust) and chaos or fault-injection testing.</li><li>Background in ML/data science sufficient to read model evals and statistical significance.</li><li>Experience red-teaming or doing adversarial/security testing on ML systems.</li><li>Familiarity with observability and tracing tools for LLM applications (e.g., LangSmith, Arize, Langfuse, Weights &amp; Biases) and general observability stacks (OpenTelemetry, Grafana, Datadog).</li><li>Experience testing systems in industrial, IoT, or operational technology (OT) environments, including industrial protocols (OPC UA, MQTT, Modbus) or simulation and hardware-in-the-loop setups.</li><li>Experience supporting UAT, pilots, or customer-facing validation of complex products.</li><li>Experience setting up test and eval infrastructure from scratch at a startup or fast-moving team.</li></ul><h2>What We’re Looking For</h2><ul><li>An engineer who is equally comfortable writing a test framework and sitting down to break an agent by hand.</li><li>Someone who can build rigorous evaluation systems for problems that don’t have a single right answer, and who knows when human judgment is the right tool.</li><li>A systems thinker who enjoys turning ambiguous agent behavior into measurable, trustworthy signals.</li><li>A strong collaborator who partners well with ML engineers, backend engineers, and Forward Deployed teams.</li></ul><h2>Why Join <a target="_blank" rel="noopener noreferrer ugc" href="http://Nexxa.AI">Nexxa.AI</a>?</h2><ul><li>Innovative Environment: Play a critical role in transforming heavy industries through groundbreaking AI and automation technologies.</li><li>Collaborative Culture: Be part of a team that values innovation, discipline, and continuous improvement.</li><li>Professional Growth: Benefit from significant opportunities for career development and advancement.</li><li>Competitive Compensation: Enjoy a comprehensive salary and equity package reflective of your expertise and contributions.</li></ul>If you’re passionate about AI quality and eager to help define what trustworthy autonomous systems look like in heavy industry, we’d love to connect.

<a target="_blank" rel="noopener noreferrer ugc" href="http://Nexxa.ai">Nexxa</a> is building the best AI systems for heavy industries — enabling machines, systems and operations to think, decide and act autonomously across manufacturing, large-scale infrastructure, logistics and legacy environments. Our mission is to translate deep technical breakthroughs into operational reality, solving some of the hardest systems-level problems in industry. <h2>Role Overview</h2>We are looking for an AI Vision Engineer to help design, build, and deploy next-generation computer vision systems across a diverse set of real-world industrial applications.This role is ideal for someone with a strong foundation in Computer Vision and Machine Learning who is excited about working across the full vision stack, including classical and deep-learning-based CV, vision-language models (VLMs), multimodal reasoning, and real-time inference at the edge and in the cloud.You will work closely with our AI and engineering teams to develop production-ready vision solutions, improve model performance, and help shape the next generation of intelligent visual systems.<h2>What You'll Do</h2><ul><li>Design, train, evaluate, and deploy computer vision models for real-world industrial applications.</li><li>Build and optimize CV pipelines for tasks such as object detection, segmentation, classification, OCR, tracking, and visual understanding.</li><li>Develop and fine-tune vision-language models (VLMs) for multimodal reasoning, visual question answering, and document understanding.</li><li>Design and optimize real-time inference pipelines for deployment on edge devices and in the cloud.</li><li>Build scalable data pipelines for image and video collection, annotation, augmentation, training, and evaluation.</li><li>Fine-tune and evaluate open-source vision and multimodal foundation models using modern training and inference frameworks.</li><li>Develop robust evaluation frameworks and benchmarks to measure model accuracy, robustness, latency, and business impact.</li><li>Optimize models for production constraints, including quantization, pruning, and hardware-accelerated inference.</li><li>Collaborate with product, engineering, and research teams to translate business requirements into technical vision solutions.</li><li>Contribute to architecture decisions, technical design reviews, and AI/CV best practices.</li><li>Stay current with the latest advancements in computer vision, multimodal AI, and autonomous systems.</li></ul><h2>Required Qualifications</h2><h3>Education</h3>Bachelor's degree in Computer Science, Computer Engineering, Electrical Engineering, Mathematics, Statistics, Artificial Intelligence, or a related technical field, or equivalent practical experience.<h3>Experience &amp; Skills</h3><ul><li>3+ years of industry experience in Computer Vision, Machine Learning, Applied AI, or related fields.</li><li>Demonstrated experience independently owning and delivering computer vision projects from concept to production.</li><li>Strong programming skills in Python.</li><li>Hands-on experience with PyTorch and modern deep learning workflows.</li><li>Experience developing and deploying computer vision models (detection, segmentation, classification, OCR) in production environments.</li><li>Experience working with image and video processing libraries such as OpenCV.</li><li>Experience with common CV/detection frameworks (e.g., YOLO, Detectron2, MMDetection, or similar).</li><li>Experience working with VLMs, multimodal models, or Generative AI applications.</li><li>Strong understanding of machine learning fundamentals, model evaluation, experimentation, and model deployment.</li><li>Experience working with Hugging Face Transformers and open-source AI ecosystems.</li><li>Familiarity with data annotation workflows, dataset curation, hyperparameter optimization, and inference optimization.</li><li>Experience building production-grade software and AI systems.</li><li>Strong analytical and problem-solving skills.</li><li>Excellent communication and collaboration skills.</li></ul><h2>Preferred Qualifications</h2><ul><li>Master's degree in Computer Science, Artificial Intelligence, Machine Learning, Computer Vision, or a related field.</li><li>Experience with OCR, document understanding, or visual reasoning systems.</li><li>Experience with Vision-Language Models (VLMs) and multimodal AI applications.</li><li>Familiarity with 3D vision, SLAM, or sensor fusion (e.g., camera + LiDAR) for industrial or robotics applications.</li><li>Experience with real-time inference optimization (TensorRT, ONNX Runtime, quantization, pruning).</li><li>Experience deploying models on edge hardware (e.g., NVIDIA Jetson, embedded systems).</li><li>Experience with LangChain, LangGraph, or agentic AI frameworks combining vision and language models.</li><li>Experience with vector databases and visual/semantic retrieval systems.</li><li>Experience deploying AI systems on AWS, GCP, or other cloud platforms.</li><li>Experience with Docker, Kubernetes, and MLOps workflows.</li><li>Experience with PostgreSQL and large-scale data systems.</li><li>Experience with model serving, distributed training, and inference optimization at scale.</li><li>Contributions to open-source projects, technical blogs, research publications, Kaggle competitions, or other demonstrable CV/AI work.</li><li>Experience working in startup or high-growth environments.</li></ul><h2>What We're Looking For</h2><ul><li>A builder who enjoys taking vision systems from prototype to production.</li><li>Someone comfortable working across classical Computer Vision, deep learning, and multimodal Generative AI.</li><li>An engineer who is curious, adaptable, and eager to learn emerging vision and AI technologies.</li><li>A strong collaborator who can contribute across research, engineering, and product discussions.</li><li>Someone excited about solving challenging real-world problems using computer vision.</li><li>An individual who takes ownership, moves quickly, and thrives in an environment with significant autonomy.</li></ul>Why Join<a target="_blank" rel="noopener noreferrer ugc" href="http://nexxa.ai/"> </a><a target="_blank" rel="noopener noreferrer ugc" href="http://Nexxa.AI">Nexxa.AI</a>?<ul><li>Innovative Environment: Play a critical role in transforming heavy industries through groundbreaking AI and automation technologies.</li><li>Collaborative Culture: Be part of a team that values innovation, discipline, and continuous improvement.</li><li>Professional Growth: Benefit from significant opportunities for career development and advancement.</li><li>Competitive Compensation: Enjoy a comprehensive salary and equity package reflective of your expertise and contributions.</li></ul>If you're passionate about development and eager to shape the infrastructure powering advanced AI solutions, we'd love to connect.

Nexxa.AI

Open jobs at Nexxa.AI

Senior Software Development Engineer in Test (SDET), QA - AI Systems

Senior Software Development Engineer in Test (SDET), QA - AI Systems

AI Vision Engineer

AI Vision Engineer

Working for Nexxa.AI?

Featured companies