⚓ Eurybia monitors model drift over time and securizes model deployment with data validation
A curated list of awesome open source tools and commercial products for monitoring data quality, monitoring model performance, and profiling data 🚀
In this repository, we will present techniques to detect covariate drift, and demonstrate how to incorporate your own custom drift detection algorithms and visualizations with SageMaker model monitor.
These are my notes of the Udacity Nanodegree Machine Learning DevOps Engineer.
資料科學的日常研究議題
Simulation, testing and comparison of state of the art Unsupervised Concept Drift Detectors used in a batch Machine Learning scenario.
PipeRoll Seismograph - daily witnessed behavioural-drift readings for LLM APIs: 39 models measured against their own past, deterministic grading, Rekor-witnessed. Readings, not ratings.
In this project, we illustrate how the Kolmogorov Smirnov (KS) statistical test works, and why it is commonly used in Machine Learning (ML), Deep Learning (DL) and Artificial Intelligence (AI).
Detect behavioural drift between LLM versions before you upgrade. Compare model responses, classify regressions, and generate migration reports with validated prompt patches.
Learn how to handle model drift and perform test-based model monitoring
Daily canary for silent LLM degradation, with confidence intervals and a measured noise floor
Continuous, model-version-bound verification of agent skills: run a skill's eval suite with and without the skill, emit hash-verified dated receipts with confidence bands, and diff them into drift reports.
Behavioural attestation for deployed language models: is the model you serve the model you validated? 1 KB/position fingerprints that catch silent sampling filters, quantisation and template bugs.
End-to-end MLOps platform for sovereign country-risk prediction with MLflow, DVC, Docker, AWS SageMaker, CI/CD and production drift monitoring.
Hey LLM, you okay? — pyramid-ordered LLM testing CLI for CI/CD. One YAML for every layer, LLM-as-a-judge gates, and A/B triage that tells prompt regressions from model drift.
Git-native CI regression testing and behavioral drift monitoring for LLM prompts.
Verifiable trust infrastructure for AI agents: model-drift detection, tool-schema translation, consent receipts and a serverless skill registry, over one signed append-only ledger. No server, no dependencies.
The Taravangian Test for AI: catch silent model degradation before your users do. SPC-based reasoning-quality monitoring for Claude, GPT, Gemini, and Grok.
LLMs as production extraction infrastructure: rule-vs-LLM triage, validated structured outputs, cost-capped model routing, a measured precision/recall eval + vendor-drift detection, entity resolution, and an Airflow 3 ETL DAG. Runs fully offline.
A reproducible benchmark for whether commercial LLMs silently drift over time. Deterministic graders, balanced tasks, every raw response in git.
To associate your repository with the model-drift topic, visit your repo's landing page and select "manage topics."