Egen

<div>
<div><strong>About Egen: </strong></div>
<div> </div>
<div>Egen is a fast-growing and entrepreneurial company with a data-first mindset. We bring together the best engineering talent working with the most advanced technology platforms, including Google Cloud and Salesforce, to help clients drive action and impact through data and insights. We are committed to being a place where the best people choose to work so they can apply their engineering and technology expertise to envision what is next for how data and platforms can change the world for the better. We are dedicated to learning, thrive on solving tough problems, and continually innovate to achieve fast, effective results. If this describes you, we want you on our team.</div>
<div> </div>
<div><em>Want to learn more about life at Egen? Check out these resources in addition to the job description.</em></div>
<div> </div>
<div><a rel="noopener noreferrer ugc" href="https://egen.ai/company/" target="_blank">Meet Egen</a></div>
<div><a rel="noopener noreferrer ugc" href="https://egen.ai/people/#careers" target="_blank">Life at Egen</a></div>
<div><a rel="noopener noreferrer ugc" href="https://egen.ai/people/" target="_blank">Culture and Values at Egen</a></div>
<div><a rel="noopener noreferrer ugc" href="https://egen.ai/people/#career-development" target="_blank">Career Development at Egen</a></div>
<div><a rel="noopener noreferrer ugc" href="https://egen.ai/people/#benefits" target="_blank"><u>Benefits at Egen</u></a></div>
</div>
<h2>About the job:</h2><div>

<li><strong>Infrastructure-as-Code (IaC) Delivery:</strong> Design, author, and maintain reusable Terraform modules and deployment pipelines for the Azure Enterprise Data environment, including core networking, security controls, resource groups, and service integrations</li>
<li><strong>Three-Plane Architecture Setup:</strong> Implement and manage the three-plane architecture (Control Plane, Data Plane, and Target Stores) across designated Azure subscriptions and resource groups supporting high-assurance (stage/prod) and low-assurance (dev/test) environments</li>
<li><strong>Hybrid Multi-Cloud Networking &amp; Security:</strong> Configure secure private networking and connectivity between Azure and the Google Cloud Advanced Data Lake (ADL)—reaching BigQuery, Cloud Storage, FHIR Store, and DICOM Store via private endpoints without public internet exposure</li>
<li><strong>Platform Services &amp; Storage Provisioning:</strong> Deploy and configure Azure services including Azure Kubernetes Service (AKS), Azure Database for PostgreSQL flexible server (with <code>pgvector</code>), Blob Storage, Key Vault, Azure Container Registry (ACR), and Azure Artifacts</li>
<li><strong>Identity &amp; Access Management (IAM):</strong> Integrate Entra ID sign-in enforcing role-based access control (RBAC), least-privilege permissions, and SailPoint-governed groups, alongside personal service accounts and JIT elevation for privileged access</li>
<li><strong>Observability &amp; Compliance:</strong> Configure centralized logging and monitoring via Azure Monitor, ensuring all telemetry and audit logs are strictly configured to exclude Protected Health Information (PHI)</li>

</div>
<h2>About you:</h2><div>

<li><strong>Cloud Platforms:</strong> Expert-level experience with Microsoft Azure enterprise data platforms, hybrid multi-cloud architectures, and integration points with Google Cloud and Microsoft Fabric</li>
<li><strong>Containerization &amp; Orchestration:</strong> Proven experience deploying and operating Apache Airflow, Milvus, Neo4j operators, and custom workloads on Azure Kubernetes Service (AKS), including GPU node pool configuration</li>
<li><strong>Automation &amp; Tooling:</strong> Deep proficiency in Infrastructure-as-Code (Terraform, Spacelift/Terraform Enterprise) and CI/CD pipeline implementation via Azure DevOps and GitHub Enterprise</li>
<li><strong>Healthcare Compliance &amp; Security:</strong> Strong working knowledge of HIPAA/PHI compliance controls, data residency restrictions (US-based storage/processing), encryption standards (TLS 1.2+, at-rest key management), and secure SDLC practices</li>

</div>

Lead Cloud Engineer (Azure)

<div><strong>About Egen: </strong></div>
<div> </div>
<div><span>Egen is a fast-growing and entrepreneurial company with a data-first mindset. We bring together the best engineering talent working with the most advanced technology platforms, including Google Cloud and Salesforce, to help clients drive action and impact through data and insights. We are committed to being a place where the best people choose to work so they can apply their engineering and technology expertise to envision what is next for how data and platforms can change the world for the better. We are dedicated to learning, thrive on solving tough problems, and continually innovate to achieve fast, effective results. If this describes you, we want you on our team.</span></div>
<div> </div>
<div><em>Want to learn more about life at Egen? Check out these resources in addition to the job description.</em></div>
<div> </div>
<div><a rel="noopener noreferrer ugc" href="https://egen.ai/company/" target="_blank">Meet Egen</a></div>
<div><a rel="noopener noreferrer ugc" href="https://egen.ai/people/#careers" target="_blank">Life at Egen</a></div>
<div><a rel="noopener noreferrer ugc" href="https://egen.ai/people/" target="_blank">Culture and Values at Egen</a></div>
<div><a rel="noopener noreferrer ugc" href="https://egen.ai/people/#career-development" target="_blank">Career Development at Egen</a></div>
<div><u><a rel="noopener noreferrer ugc" href="https://egen.ai/people/#benefits" target="_blank">Benefits at Egen</a></u></div>
<div> </div>
<div>
<h2><strong>About the opportunity</strong></h2>
<p>We're looking for an engineer who is equally at home running a GPU training platform and the large-scale model training that runs on it. You'll be embedded with a healthcare and life-sciences AI client that trains foundation models on a large GKE-based H200 and GB200 cluster. The client is reorganizing so that its foundation-model platform and its agent platform teams work more closely together.</p>
<p>The role centres on the training stack and the scientists who use it. Solid infrastructure knowledge and experience are required, but the client already has strong SRE and infrastructure capacity, so running the cluster isn't the focus: you'll partner with that team and understand the training workloads well enough to tune, benchmark, and debug them end to end. You'll work alongside other Egen engineers and client scientists, and you'll own your area outright.</p>
</div>
<h2>What You Will Do:</h2><h3><strong>Training platform</strong></h3>

<li>Work with the client's infrastructure team on the GKE training platform (GPU node pools, job queueing and scheduling, quota and capacity) from the training workload's side.</li>
<li>Diagnose and fix platform failures at the root (for example a controller that keeps running out of memory) rather than restarting them.</li>
<li>Tune multi-node GPU networking (RDMA/GPUDirect, NCCL) for training throughput.</li>

<h3><strong>Distributed training</strong></h3>

<li>Run, scale and debug multi-node distributed training with PyTorch (FSDP, DeepSpeed, Megatron or NeMo-style) and Ray Train.</li>
<li>Profile jobs and remove bottlenecks across compute, memory, input pipelines and interconnect.</li>

<h3><strong>Benchmarking and evals</strong></h3>

<li>Design and run performance benchmarks for the cluster and training stack: throughput, scaling efficiency, cost per run.</li>
<li>Build and maintain model evaluation harnesses so changes to the platform or training recipes are measured, not assumed.</li>
<li>Instrument training and the platform with telemetry and monitoring, and keep end-to-end pipeline tests running against what users actually run.</li>

<div><strong>Ownership on the client team</strong></div>

<li>Treat the platform as your own: proactively find issues, plug gaps and propose solutions before anyone asks, keep the team informed of status and risks, and look after the scientists who use it.</li>
<li>Explain and defend your design decisions to client engineers and scientists in detail.</li>
<li>Stay effective while the client's priorities and team structure change.</li>
<h2>Your Technical Toolkit:</h2><ul><li><strong>Core Languages</strong>: Mastery of Python and shell scripting</li>
<li>
<p><strong>AI/LLM Ecosystem</strong>: Extensive experience with Google Gemini, GPT-4, or LLaMA; deep knowledge of Prompt Engineering and Fine-tuning</p>
</li>
<li>
<p><strong>Data &amp; Search</strong>: Expertise in Vector Databases (Vertex AI Vector Search, pgvector, etc.) and implementing Semantic Search</p>
</li>
<li>
<p><strong>Infrastructure</strong>: Hands-on experience with Google Cloud (Vertex AI) and building scalable software architectures</p>
</li>
<li>
<p><strong>Frameworks</strong>: Proficiency in LangChain, LlamaIndex, or similar orchestration layers</p>
</li>
<li>
<p><strong>Mindset:</strong> A strong software engineering foundation—you write clean, maintainable code and understand the full SDLC</p>
</li></ul>
<h2>Basic Qualifications:</h2><div>

<li>
<p>5+ years of engineering experience, with recent hands-on work on GPU training infrastructure and large-scale training.</p>
</li>
<li>
<p><strong>Strong Python software engineering:</strong> production-quality code, tests and code review. Coding is central to this role, not an afterthought.</p>
</li>
<li>
<p><strong>GKE and GPU scheduling:</strong> Kubernetes on Google Cloud with GPU node pools, job queueing (Kueue or similar), and capacity and quota management.</p>
</li>
<li>
<p><strong>Distributed PyTorch and Ray Train: </strong>has personally run and debugged multi-node training jobs.</p>
</li>
<li>
<p><strong>RDMA and GPU networking:</strong> GPUDirect/RDMA, NCCL tuning, multi-node GB200 (or H200) topology.</p>
</li>
<li>
<p><strong>Benchmarking and evals:</strong> has designed performance benchmarks and model evaluations, and used them to make decisions.</p>
</li>
<li>
<p>Deep enough understanding to explain how and why a system works, without leaning on AI coding tools to carry the reasoning.</p>
</li>
<li>
<p>A track record of visible ownership: present, communicative, and accountable on a demanding team.</p>
</li>

</div>
<h2>Personal Attributes:</h2><div>
<h3><strong>Nice to have</strong></h3>

<li>
<p>Healthcare or life-sciences data (genomics, clinical, biomedical), and working under HIPAA.</p>
</li>
<li>
<p>Observability and telemetry for training platforms: metrics, tracing, alerting, end-to-end pipeline testing.</p>
</li>

</div>

ML Platform Engineer, Distributed Training

<div>
<p><span>We are seeking a Principal Data Architect to define and drive enterprise data architecture, modernization, and governance strategies across complex multi-cloud (Google Cloud Platform and Microsoft Azure) and hybrid environments. This role will design scalable, secure, and high-performing data platforms while partnering with business and engineering leaders to translate strategic objectives into impactful data solutions.</span></p>
<p><span>A key focus of this role will be spearheading scalable, agent-driven data migration pipelines to transition legacy on-premises data assets to the cloud, setting the technical direction for engineering teams, and actively managing the customer's technical backlog and delivery roadmap.</span></p>
</div>
<h2>Key Responsibilities</h2><div>
<ul>
<li>
<p><span>Multi-Cloud Lakehouse Architecture: Lead the end-to-end architecture, design, and implementation of scalable Data Lakehouse solutions across Google Cloud Platform (BigQuery, GCS, BigLake, Dataplex) and Microsoft Azure (Azure Synapse Analytics, Microsoft Fabric, ADLS Gen2, Azure Databricks).</span></p>
</li>
<li>
<p><span>On-Prem to Cloud Migration &amp; Agentic Pipelines: Architect and deploy automated, scalable, agent-driven data pipelines (leveraging GenAI agents and metadata-driven frameworks) to orchestrate, validate, and accelerate complex migrations from on-premises legacy warehouses/databases (Oracle, Teradata, SQL Server) to GCP and Azure.</span></p>
</li>
<li>
<p><span>Client Engagement &amp; Advisory: Collaborate directly with customers to understand business goals, data challenges, and technical requirements; translate them into robust architectural blueprints, technical roadmaps, and actionable delivery plans.</span></p>
</li>
<li>
<p><span>Customer Technical Backlog Management: Own and actively manage the customer technical backlog—translating business epics into technical user stories, prioritizing technical debt, defining sprint milestones, and aligning backlog items with business ROI.</span></p>
</li>
<li>
<p><span>Modern Ingestion &amp; Streaming: Design and implement data pipelines supporting both real-time and batch ingestion using modern orchestration (Cloud Composer, Azure Data Factory, Airflow) and streaming/messaging frameworks (Kafka, GCP Pub/Sub, Azure Event Hubs).</span></p>
</li>
<li>
<p><span>Governance &amp; Lineage: Establish and enforce best practices for unified data cataloging, metadata management, automated lineage, and data quality across multi-cloud and on-prem systems using Dataplex, Microsoft Purview, or open-source governance tools.</span></p>
</li>
<li>
<p><span>Security &amp; Compliance: Define and implement robust data security, identity management (IAM / Azure Entra ID), encryption, row/column-level access control, and regulatory compliance standards across clouds.</span></p>
</li>
<li>
<p><span>Team Technical Leadership &amp; Mentorship: Steer the overarching technical direction of engineering teams; establish architectural standards, mentor senior and mid-level engineers, conduct architecture review, and ensure engineering excellence.</span></p>
</li>
<li>
<p><span>Hands-on Delivery &amp; Prototyping: Balance strategic architecture discussions with hands-on solutioning, architectural spikes, Proof-of-Concepts (POCs), and performance tuning of complex multi-cloud distributed workloads.</span></p>
</li>
<li>
<p><span>Cross-Functional Partnership: Partner with enterprise architects, delivery leads, DevOps engineers, and customer stakeholders to drive solution adoption, operational scalability, and long-term platform maintainability</span></p>
</li>

</ul></div>
<h2>Technical Experience &amp; Tooling</h2><div>
<p><span>Candidates should demonstrate hands-on architecture and design experience across the following domains:</span></p>
<ul>
<li>
<p><span>Cloud Platforms:  Deep expertise in both Google Cloud Platform (GCP) and Microsoft Azure enterprise data ecosystems.</span></p>
</li>
<li>
<p><span>Processing Engines: Apache Spark, PySpark, Databricks, Azure Synapse / Fabric Spark, GCP Dataproc.</span></p>
</li>
<li>
<p><span>Modern Migration &amp; AI Automation: Building automated, agentic, or metadata-driven data migration pipelines, automated schema conversion, and data validation agents for on-prem to cloud lift-and-shift/re-platforming.</span></p>
</li>
<li>
<p><span>Messaging &amp; Streaming: Apache Kafka, GCP Pub/Sub, Azure Event Hubs.</span></p>
</li>
<li>
<p><span>Orchestration: Apache Airflow, Google Cloud Composer, Azure Data Factory (ADF).</span></p>
</li>
<li>
<p><span>Analytical Storage &amp; Warehouses:  GCP BigQuery, Azure Synapse, Microsoft Fabric, Delta Lake/Parquet.</span></p>
</li>
<li>
<p><span>NoSQL / Cache Storage: MongoDB, Redis, Bigtable, Azure Cosmos DB.</span></p>
</li>
<li>
<p><span>Containerization &amp; Deployment: Kubernetes (GKE and AKS), Cloud Run, Azure Container Apps.</span></p>
</li>
<li>
<p><span>API Frameworks: Python (FastAPI, Flask) for data service endpoints.</span></p>
</li>
<li>
<p><span>Databases &amp; Querying: Advanced SQL validation and extraction from legacy/RDBMS systems (SQL Server, Oracle, PostgreSQL).</span></p>
</li>
<li>
<p><span>CI/CD &amp; Version Control: Git, GitHub, GitLab CI/CD, Azure DevOps, and Infrastructure as Code (Terraform).</span></p>
</li>

</ul></div>
<h2>Required Qualifications &amp; Skills</h2><div>
<ul>
<li>
<p><span>15+ years of progressive experience in Software Engineering, Data Engineering, and Enterprise Data Platform architecture.</span></p>
</li>
<li>
<p><span>5+ years architecting, migrating, and deploying production data platforms across GCP and/or Azure, with proven multi-cloud integration patterns.</span></p>
</li>
<li>
<p><span>Demonstrated track record leading on-premises/cloud to cloud modernization/migration initiatives, including data ingestion, transformation, automated data reconciliation, and cutover strategies.</span></p>
</li>
<li>
<p><span>Experience designing or integrating autonomous, agent-driven, or LLM-augmented data workflows (e.g., automated schema mapping, code conversion, data quality validation agents).</span></p>
</li>
<li>
<p><span>Proven hands-on experience designing and deploying Lakehouse platforms incorporating Medallion Architecture (Bronze/Silver/Gold) and Data Mesh / Data Product principles.</span></p>
</li>
<li>
<p><span>Deep knowledge of multi-cloud data governance, access management, and cataloging (Dataplex, Microsoft Purview, Unity Catalog).</span></p>
</li>
<li>
<p><span>Strong leadership capability in technical backlog management, agile scoping, sprint planning, and setting technical strategy for distributed teams.</span></p>
</li>
<li>
<p><span>Exceptional stakeholder communication skills: capability to articulate architectural trade-offs to C-level executives as smoothly as reviewing code with engineers.</span></p>
</li>

</ul></div>
<h2>Preferred Qualifications (Optional / Good to Have)</h2><div>
<ul>
<li>
<p><span>Multi-cloud certifications: </span><em><span>Google Cloud Professional Data Engineer / Cloud Architect</span></em><span> and/or </span><em><span>Microsoft Certified: Azure Data Engineer Associate / Azure Solutions Architect Expert</span></em><span>.</span></p>
</li>
<li>
<p><span>Hands-on knowledge of Snowflake and Databricks </span></p>
</li>
<li>
<p><span>Familiarity with AWS data services (Amazon S3, Redshift, Glue, EMR).</span></p>
</li>
<li>
<p><span>Experience with LangChain, LlamaIndex, or agentic framework integrations in data operations.</span></p>
</li>

</ul></div>
<h2>Education</h2><div>
<ul>
<li>
<p><span>Bachelor’s or Master’s degree in Computer Science, Information Technology, Engineering, or equivalent practical experience.</span></p>
</li>

</ul></div>

Principal Data Architect - Multi-Cloud (GCP & Azure)

<div>
<div><strong>About Egen: </strong></div>
<div> </div>
<div>Egen is a fast-growing and entrepreneurial company with a data-first mindset. We bring together the best engineering talent working with the most advanced technology platforms, including Google Cloud and Salesforce, to help clients drive action and impact through data and insights. We are committed to being a place where the best people choose to work so they can apply their engineering and technology expertise to envision what is next for how data and platforms can change the world for the better. We are dedicated to learning, thrive on solving tough problems, and continually innovate to achieve fast, effective results. If this describes you, we want you on our team.</div>
<div> </div>
<div><em>Want to learn more about life at Egen? Check out these resources in addition to the job description.</em></div>
<div> </div>
<div><a rel="noopener noreferrer ugc" href="https://egen.ai/company/" target="_blank">Meet Egen</a></div>
<div><a rel="noopener noreferrer ugc" href="https://egen.ai/people/#careers" target="_blank">Life at Egen</a></div>
<div><a rel="noopener noreferrer ugc" href="https://egen.ai/people/" target="_blank">Culture and Values at Egen</a></div>
<div><a rel="noopener noreferrer ugc" href="https://egen.ai/people/#career-development" target="_blank">Career Development at Egen</a></div>
<div><u><a rel="noopener noreferrer ugc" href="https://egen.ai/people/#benefits" target="_blank">Benefits at Egen</a></u></div>
</div>
<h2>RESPONSIBILITIES</h2><div>

<li><strong>Data Ingestion: </strong>Design and build the pipelines that bring data in for feature engineering and machine learning.</li>
<li><strong>Feature Engineering:</strong> Use your knowledge of healthcare data, statistics, and data science to create features with real predictive power.</li>
<li><strong>Feature Extraction and Selection:</strong> Use techniques like clustering, text analytics, and PCA to find the most useful signals while preserving the meaning in the original data, so models stay accurate and scale well.</li>
<li><strong>Exploratory Data Analysis: </strong>Explore the data to find patterns, trends, and relationships, and use what you learn to match features with the right modeling approach.</li>
<li><strong>Model Development:</strong> Experiment with statistical and machine learning methods to build, tune, and optimize predictive models that fit the use case.</li>
<li><strong>Deployment: </strong>Ship production code that follows engineering best practices.</li>
<li><strong>Documentation and Communication:</strong> Document your work and share your approach, insights, and solutions with technical and non-technical audiences.</li>
<li><strong>Monitoring:</strong> Build and maintain the pipelines and processes that keep our enterprise data science products running smoothly.</li>
<li><strong>Quality and Integrity:</strong> Keep data and outputs accurate and reliable from start to finish.</li>
<li><strong>Collaboration:</strong> Partner with stakeholders, data scientists, data engineers, and product managers to build and deliver great data products.</li>

</div>
<h2>QUALIFICATIONS</h2><div>

<li><strong>Experience:</strong> 6+ years working across data science and machine learning (for the Lead level).</li>
<li><strong>Core Languages:</strong> Expert-level SQL and Python.</li>
<li><strong>Data Types:</strong> Hands-on work with both structured and unstructured data, such as tables, text, images, and video.</li>
<li><strong>Databases:</strong> Experience with both relational (SQL) and non-relational (NoSQL) databases.</li>
<li><strong>Data Processing:</strong> Familiarity with ETL and data processing tools like Apache Spark.</li>
<li><strong>Visualization and Monitoring:</strong> Experience using tools to visualize data and monitor its health.</li>
<li><strong>Engineering Practices:</strong> Comfort with modern software practices like automated testing and CI/CD.</li>
<li><strong>Containers:</strong> Experience with Docker.</li>
<li><strong>Cloud:</strong> Experience deploying to GCP or another cloud platform.</li>

</div>
<p> </p>
<div> </div>
<h2>BONUS EXPERIENCE</h2><ul><li>Consulting or Professional Services Background</li>
<li>Domain Knowledge of Healthcare Data</li>
<li>Knowledge of Healthcare Protocols such as HL7, FHIR, DICOM</li></ul>

Egen

Open jobs at Egen

Lead Cloud Engineer (Azure)

ML Platform Engineer, Distributed Training

Principal Data Architect - Multi-Cloud (GCP & Azure)

Principal Data Architect (Data Science)

Working for Egen?

Featured companies