RESPONSIBILITIES:<br>ML Model Deployment & Platform Management<br>• Lead the design, implementation, and ongoing maintenance of scalable ML infrastructure on Databricks, including ML flow for experiment tracking, model registry, and model serving endpoints.<br>• Oversee the development of the ML Ops platform and automated pipelines for deploying, monitoring, and maintaining models within production environments.<br>• Implement robust solutions for model versioning, systematic retraining, and comprehensive artifact management using Databricks Unity Catalog for ML governance.<br>• Design and manage Databricks Feature Store for consistent feature engineering across training and inference pipelines.<br>Generative AI & LLM Operations<br>• Architect and implement Retrieval-Augmented Generation (RAG) systems for document Q& A, enabling business teams to query fund documents, investor letters, and market research.<br>• Design, deploy, and manage vector database solutions (Databricks Vector Search, Pinecone, or similar) for semantic search and retrieval across enterprise documents.<br>• Lead LLM fine-tuning and customization initiatives, training models like Claude or open-source alternatives with CIM proprietary data while ensuring data privacy and compliance.<br>• Develop and optimize document processing pipelines including PDF parsing, chunking strategies, and embedding generation for RAG applications.<br>• Implement prompt engineering best practices and LLM evaluation frameworks to ensure output quality, relevance, and factual accuracy.<br>• Build guardrails and safety measures for GenAI applications, including hallucination detection, output validation, and source attribution.<br>Automation & CI/CD Pipelines<br>• Design and implement extensive automation across the ML workflow, covering model training, testing, validation, and deployment using Databricks Workflows and Asset Bundles.<br>• Set up robust CI/CD pipelines for both traditional ML models and GenAI applications, leveraging GitHub Actions, Azure DevOps, or similar tools.<br>• Automate complex data and model workflows utilizing orchestration tools such as Airflow, Prefect, or Databricks Workflows.