Senior AI DevOps Engineer - Azure/AWS & Python

Senior AI DevOps Engineer

Role Summary

We are seeking a Senior AI DevOps Engineer to design, automate, secure, and operate scalable platforms for artificial intelligence and machine learning workloads.

This hands-on role combines cloud engineering, DevOps, Kubernetes, infrastructure as code, MLOps, and LLMOps. The engineer will support the full AI lifecycle—from experimentation and model packaging through deployment, monitoring, optimization, retraining, and retirement.

The primary cloud environment is Microsoft Azure, with additional experience in AWS, including Amazon Bedrock, highly valued. The successful candidate will partner with data scientists, ML engineers, software engineers, security teams, and business stakeholders to deliver reliable, production-grade AI solutions.

Key Responsibilities

AI/ML Platform Engineering

• Design and operate secure, scalable, and highly available platforms for machine learning, generative AI, and LLM workloads.

• Develop standardized deployment patterns for traditional ML models, deep learning models, LLMs, RAG applications, and AI agents.

• Support real-time, batch, asynchronous, and event-driven model-serving architectures.

• Deploy and manage model endpoints, embedding services, prompt-processing services, retrieval pipelines, and inference APIs.

• Optimize AI platforms for performance, availability, scalability, resilience, and cost.

• Partner with data science and ML engineering teams to productionize experimental models.

• Establish repeatable processes for model packaging, validation, deployment, rollback, versioning, and retirement.

CI/CD and DevSecOps Automation

• Design and maintain CI/CD pipelines using GitHub Actions and Azure DevOps.

• Automate deployments for application code, infrastructure, container images, model artifacts, and ML workflows.

• Implement automated testing for code, infrastructure, data pipelines, model packages, security controls, and deployment configurations.

• Establish promotion workflows across development, test, staging, and production environments.

• Automate model validation, deployment-readiness checks, approval gates, rollback, canary releases, and controlled production promotion.

• Integrate source control, artifact repositories, container registries, model registries, cloud services, and observability platforms.

• Apply GitOps, pull-request-based deployment, reusable workflow templates, and environment protection practices.

Containers and Kubernetes

• Design, deploy, and operate production-grade Kubernetes and Azure Container Instances environments for AI and ML workloads.

• Build secure, efficient Docker images for model-serving APIs, data-processing jobs, feature services, and LLM applications.

• Manage Kubernetes deployments, services, ingress, configuration, secrets, storage, namespaces, resource quotas, and autoscaling.

• Implement workload scheduling for CPU-, GPU-, and memory-intensive applications.

• Use Helm, Kustomize, Kubernetes operators, and platform integrations to support model serving, distributed training, experiment tracking, and batch inference.

• Troubleshoot production issues involving containers, networking, storage, scheduling, performance, and resource utilization.

• Establish standards for cluster security, upgrades, capacity planning, backup, and disaster recovery.

Cloud Engineering and Infrastructure as Code

• Develop reusable Terraform modules and Azure Bicep templates for multi-environment cloud infrastructure.

• Build secure cloud foundations using identity and access management, private networking, encryption, secrets management, policy enforcement, and centralized logging.

• Automate infrastructure provisioning, validation, drift detection, controlled changes, and rollback.

• Integrate infrastructure deployment with CI/CD pipelines and approval workflows.

• Monitor and optimize cloud costs, including compute, storage, data transfer, GPU capacity, and AI inference usage.

• Support high availability, business continuity, disaster recovery, and regional resiliency requirements.

MLOps and Model Lifecycle Management

• Implement MLOps practices covering experimentation, training, validation, registry management, deployment, monitoring, retraining, and retirement.

• Manage model registries and experiment-tracking platforms such as MLflow, Kubeflow, or comparable tools.

• Govern model versions, metadata, lineage, ownership, approval status, and deployment history.

• Establish reproducibility across code, data, model artifacts, dependencies, configurations, and runtime environments.

• Implement model rollback, challenger models, A/B testing, canary deployments, shadow deployments, and automated model-refresh workflows.

• Monitor model, data, and concept drift and support remediation through retraining or controlled model updates.

LLMOps and Generative AI Operations

• Deploy and operate hosted and self-managed LLMs in secure, production-grade environments.

• Build deployment workflows for prompt templates, embedding models, rerankers, retrieval services, and AI application components.

• Support RAG, semantic search, agentic workflows, tool-using applications, and vector database integrations.

• Manage model configuration, context windows, token limits, concurrency, batching, caching, rate limits, quotas, and RPM/TPM usage.

• Optimize model selection, latency, throughput, response quality, token consumption, and overall GenAI cost.

• Implement controls for sensitive-data exposure, prompt injection, jailbreaks, unauthorized access, unsafe responses, and data leakage.

• Apply GenAI SDKs and orchestration frameworks such as OpenAI/Azure AI SDKs, Google ADK, LangChain, and LangGraph.

Observability, Reliability, and Incident Management

• Implement observability across AI platforms, infrastructure, applications, models,