MLOps to GenAIOps: Orchestrating the Transition from Scikit-learn APIs to Multi-Step Agent Pipelines

[manishdwibedy]

What format do you have in mind?

Talk (20-25 minutes + Q&A)

What domain would you say your talk/session falls under?

Data Science, AI/ML, Deep learning, Neural Networks

Abstract

This talk explores the architectural shift from deploying Scikit-learn REST APIs (MLOps) to managing complex RAG pipelines and Agents (GenAIOps).

We demonstrate how FastAPI and Docker transition from model serving tools to orchestration layers for multi-step generative workflows, focusing on new operational challenges.

Description

Many data scientists are familiar with the MLOps principles of deploying a predictive model: training a Scikit-learn model, saving it with joblib, and serving it as a stateless endpoint via FastAPI and Docker. This talk explores how these foundational MLOps skills are extended, not replaced, to manage the complexity of Generative AI Operations (GenAIOps).

GenAI systems, such as RAG (Retrieval Augmented Generation) pipelines or multi-step AI Agents, are not single models; they are choreographed workflows involving multiple components: embedding models, vector databases, LLM calls, and business logic.

We will use a practical, code-focused approach to contrast the two operational paradigms:

• MLOps Case Study: Deploying a simple, predictive Scikit-learn model using FastAPI and containerizing it with Docker.

• GenAIOps Case Study: Transitioning to a multi-step RAG pipeline. We will demonstrate how FastAPI transforms from a model server into an Orchestration Layer that manages sequential calls (e.g., query pre-processing, vector search, LLM invocation via a framework like LangChain).

• The DevOps Difference: Highlighting the new GenAIOps operational concerns: Prompt Versioning, Tracer/Observability (to debug multi-step agents), and managing the data pipeline for the vector store alongside the code deployment.

Attendees will leave with a clear understanding of the architectural shift from a simple REST API to a complex, observable GenAI system, anchored by their existing knowledge of Python, FastAPI, and Docker.

Key Takeaways

What the Audience Will Learn and Be Able to Do:

Understand the Architectural Shift (Δ):

Differentiate MLOps from GenAIOps by focusing on the transition from stateless model serving (Scikit-learn) to multi-step workflow orchestration (RAG/Agents).

Identify the new "model artifacts" in GenAI—namely Prompt Templates and the Vector Store/Retrieval Corpus—and how they impact deployment pipelines (CI/CD).

Master FastAPI as an Orchestrator:

Learn how to transform a simple FastAPI prediction endpoint into a sophisticated GenAI Orchestration Layer that manages sequential calls to an embedding model, a vector database, and an LLM.

Be able to use Pydantic in GenAIOps for structured output validation from Large Language Models (LLMs), moving beyond simple API schemas.

Operationalize GenAI Components:

Identify and plan for new operational concerns unique to GenAIOps, such as Hallucination Monitoring, Prompt Versioning and Evaluation (Prompt Engineering), and cost management for LLM API usage.

Gain confidence in using Docker to containerize a complex, multi-dependency GenAI application, ensuring reproducible environments that include both traditional (e.g., joblib file) and modern components (e.g., LangChain/LlamaIndex dependencies).

Bridge the Skill Gap:

Recognize that their existing scientific Python stack (e.g., NumPy/Pandas for data ingestion/pre-processing) remains critically important for DataOps and vector preparation within a GenAI pipeline.

Pre-requisites/Intended audience & reading material

🧑‍💻 Pre-requisites / Intended Audience

This talk is primarily aimed at data science practitioners and backend engineers with a foundational background in Python and traditional machine learning concepts.

Intended Audience

Data Scientists & ML Engineers (Primary Audience): Individuals who have experience building and deploying traditional models (e.g., using Scikit-learn, Pandas, NumPy) and are looking to transition their skills into the Generative AI and MLOps/GenAIOps space.

Backend & DevOps Engineers: Those familiar with FastAPI or other web frameworks and Docker who want to understand the unique architectural and operational challenges presented by integrating Large Language Models (LLMs) into production systems.

Students & Researchers: Individuals with a solid scientific Python foundation who are curious about the current industry best practices for deploying modern AI solutions.

Pre-requisites (What the Audience Should Know)

To get the most value from this session, familiarity with the following concepts is helpful:

Core Python: Intermediate proficiency with Python programming.

Basic Scientific Python: Exposure to foundational libraries like NumPy and Pandas.

ML Concepts: A general understanding of what a machine learning model is and how it is serialized (e.g., using joblib).

API Fundamentals: Basic knowledge of what a REST API is (POST/GET requests). Experience with FastAPI or Flask is a plus, but not mandatory.

Containerization: Basic understanding of Docker concepts (images, containers).

Resources

Suggested Reading / Pre-session Material

No mandatory reading is required, but for those wishing to prepare, exploring these concepts will enhance their understanding:

A quick review of FastAPI's basic structure (Endpoints and Pydantic).

Familiarity with the high-level concept of RAG (Retrieval Augmented Generation)—understanding that it combines information retrieval with an LLM.

A brief overview of an MLOps workflow (Model → Serialization → API → Deployment).

Time required for the talk/session

30 mins

About the Author

I am a seasoned Senior Software Engineer and technical leader, specializing in the intersection of Full Stack development, DevOps, and Generative AI.

My professional journey began in traditional software engineering, leading product development in areas like telecom and IoT. Since 2022, I have immersed myself fully in the new AI landscape, serving as a Lead Engineer at Social Protocol where I was responsible for architecting and developing a unified AI/LLM Aggregator Platform and a custom Retrieval-Augmented Generation (RAG) solution. More recently, I have continued this focus, building an AI-driven tool (QualiMed) that uses an LLM to automate comprehensive test case creation.

This background provides the perfect context for this talk, "MLOps to GenAIOps." I have hands-on experience in both worlds:

MLOps Baseline: Collaborating with data scientists to feed model outputs via APIs at Tatras Data and contributing to predictive maintenance AI pipelines at Scry Analytics. I have extensive experience deploying APIs using FastAPI, Docker, and AWS Serverless architecture.

GenAIOps Orchestration: Using Python, Flask, and React to design the full technical architecture for complex LLM-driven platforms, which transforms the simple model serving API into a true workflow orchestrator.

I'm here to show how your existing MLOps skills are the essential foundation for mastering GenAIOps.

Connect with me:
LinkedIn: http://linkedin.com/in/manishdwibedy
GitHub: https://github.com/manishdwibedy
Personal Site: https://manishd.in

Availability

6th Dec

Talk/session as a Bluesky post (300 characters max)

MLOps → GenAIOps: How do you deploy an LLM Agent vs. a Scikit-learn model? 🤯

I'm sharing a code-first look at how your FastAPI/Docker skills evolve from single-model MLOps to multi-step GenAIOps orchestration. SciPy India, Dec 6. #Python #FastAPI #GenAI #MLOps

Any comments

I am submitting this proposal for a 30-minute Talk slot (20 minutes of content + 10 minutes for Q&A), as this duration is essential to cover both the baseline MLOps architecture and the new, more complex GenAIOps components effectively.

Specific Information for Reviewers:

Relevance to SciPy/Python Community: This talk bridges the established SciPy ecosystem (NumPy, Scikit-learn, Flask/FastAPI) with the emerging Generative AI stack (LangChain, vector stores). It provides a high-value, practical roadmap for engineers familiar with traditional scientific Python who are looking to move their skills into production LLM and Agent systems.

Focus on Open Source: The entire talk is based on Python-first, open-source-compatible tools (FastAPI, Docker, LangChain, etc.). I will be sharing links to the full source code for my RAG solution and other GenAI projects (e.g., QualiMed GitHub) to maximize learning and community contribution.

Non-Commercial Guarantee: While I will reference my professional experience to validate the architectures, the talk is purely technical and focused on best practices, not on promoting any commercial product or specific company.

Speaker Experience: I have substantial public speaking experience, including talks at developer events (GDG, PyDelhi, PyData). I am comfortable managing time and engaging the audience in a 30-minute technical session.

I believe this topic is timely, practical, and directly aligns with SciPy India's goal of showcasing the application of scientific computing to solve industrial problems. I am eager to share these lessons with the community.

Slides link

Work in Progress, would be submitting it in a weeks to come

Accessibility & special requirements

Technical Requirements:

Projector/Screen: High-resolution projector and screen capable of displaying a 16:9 aspect ratio at a minimum resolution of 1920x1080. This is crucial for displaying code snippets clearly and comfortably for the entire audience.

Audio: A dedicated microphone (lapel or handheld) connected to the room's sound system for clear voice amplification.

Presenter Laptop Hookup: Access to a standard HDMI or USB-C connection is required to connect my personal laptop.

Reliable Internet: Stable, high-speed Wi-Fi access is needed for a brief, live demo/run of the GenAIOps orchestration pipeline, if time permits.

Checklist

• I am a first time speaker (We're here to help!)
• This talk/session has been delivered before
• I consent to recording this talk/session and publishing it on YouTube/other platforms
• I consent to using my picture (or preferred visual) and name on social media
• I have read, understood, and have/will be following the guidelines and SciPy India's CoC (must tick this one)