Torchserve server using a YoloV5 model running on docker with GPU and static batch inference to perform production ready and real time inference.
-
Updated
Feb 10, 2023 - Python
#
batch-inference
Here are 38 public repositories matching this topic...
Production-ready batch inference for Cohere’s Arabic/English ASR model, with optimized Silero VAD, multi-file GPU batching, subtitles, and optional word-level timestamps.
-
Updated
Jul 15, 2026 - Python
Optimize and run consistent AI Functions.
s3 prompt data-engineering data-pipelines observability evaluation-framework synthetic-data unstructured-data mlops batch-inference prompt-engineering llm-inference evals distributed-inference evals-engineering
-
Updated
Oct 16, 2026 - Python
RWKV7 推理服务 | 高并发批量推理，动态扩缩容，兼容OpenAI API，CUDA优化，低显存部署大模型推理 RWKV linear attention batch inference server — auto-queue, dynamic capacity, OpenAI compatible.
high-throughput fastapi linear-attention batch-inference dynamic-batching rwkv llm-serving llm-inference rwkv7 openai-compatible
-
Updated
Jun 20, 2026 - Python
Batch inference for datasets: rows in, any OpenAI-compatible endpoint, resumable parquet out. Congestion-aware concurrency — no tuning.
-
Updated
Sep 25, 2026 - Python
Serve pytorch inference requests using batching with redis for faster performance.
-
Updated
Apr 11, 2021 - Python
Make the call. Run local Laya, Von, CLM or hosted Jev for typed classification and scoring through one HTTP and MCP API, with resource-aware batches.
python docker mcp decision-making cuda scoring self-hosted pytorch embeddings classification typesafe clm von laya batch-inference local-ai mcp-server qwen3 decision-models jev
-
Updated
Oct 21, 2026 - Python
Fake Anthropic / Bedrock (InvokeModel, Converse, batch inference, bundled S3) / OpenAI server for zero-cost human/AI-in-the-loop LLM debugging: scripted test harness (scenario rules, latency / rate-limit simulation, pytest fixture), cross-provider relay, observability (history API, JSON logs, Prometheus metrics, cache diagnostics, cost provenance)
testing debugging proxy mock-server s3 pytest bedrock openai gpt boto3 observability test-harness grok human-in-the-loop claude fastapi batch-inference llm anthropic cross-provider
-
Updated
Oct 18, 2026 - Python
Self-hosted batch LLM pipeline for analyzing customer feedback from Excel. Upload xlsx, describe the task, configure output fields — get structured results. Works with any OpenAI-compatible API and Ollama.
sentiment-analysis text-analytics customer-feedback fraud-detection no-code human-in-the-loop unstructured-data structured-output excel-automation russian-nlp batch-inference prompt-engineering llm-pipeline ai-data-analysis
-
Updated
Apr 24, 2026 - Python
Tutorial: mass-parallelized RAI compliance checks with vLLM batch inference on Cloud TPU v5e, using Gemma 4 as an LLM-as-a-Judge
-
Updated
Aug 4, 2026 - Python
Queue-driven, scale-from-zero GPU inference for any Kubernetes — bursts to cross-region VMs when GPUs run dry
python kubernetes machine-learning gpu helm inference self-hosted gpu-computing autoscaling inference-server model-serving ml-infrastructure mlops scale-to-zero serverless-gpu batch-inference serverless-inference gpu-inference ai-infrastructure
-
Updated
Jun 20, 2026 - Python
One-command agentic AI workshop for NVIDIA DGX Spark: LoRA fine-tuning, RAG and agents in a NemoClaw sandbox, plus enterprise batch AI on vLLM (invoices, inbox, call QA) with measured accuracy. Bilingual VI/EN, works offline.
workshop vietnamese nvidia speech-to-text lora ai-agents fine-tuning rag batch-inference vllm local-llm retrieval-augmented-generation ollama nvfp4 dgx-spark openclaw nemoclaw openshell
-
Updated
Oct 7, 2026 - Python
Fine-tune anime-seg (ISNetDIS) bằng LoRA trên 1306 cặp ảnh-mask để tách nền anime; có split cố định, evaluation trên mask thô, hậu xử lý alpha_gain cho RGBA cutout, merge adapter vào checkpoint, và CLI rmbg cho train/eval/predict/report/make-split với manifest tái lập đầy đủ.
computer-vision deep-learning pytorch transfer-learning lora image-segmentation rgba model-evaluation fine-tuning segmentation-model alpha-matting background-removal checkpointing pytorch-lightning batch-inference anime-segmentation
-
Updated
Aug 4, 2026 - Python
AWS-oriented ML platform for fraud detection, churn prediction, anomaly detection, experimentation, monitoring, and risk inference.
python aws data-science machine-learning risk-analysis scikit-learn cloudwatch experimentation ab-testing churn-prediction fraud-detection anomaly-detection sagemaker mlops root-cause-analysis drift-detection model-monitoring batch-inference real-time-inference champion-challenger
-
Updated
Jun 2, 2026 - Python
Summarise ready streaming records in an overnight batch and deliver the results as a PDF.
-
Updated
Oct 14, 2026 - Python
Production-style batch ML pipeline for manufacturing claims anomaly detection with drift checks (synthetic data).
-
Updated
Jan 29, 2026 - Python
B2B Fintech SaaS Churn Intelligence & Customer Lifetime Value (CLV) Engine with Kimball Star Schema, LightGBM, Tweedie Regressor, and TreeSHAP.
python machine-learning data-engineering fintech lightgbm churn-prediction customer-lifetime-value cohort-analysis explainable-ai b2b-saas feature-store shap github-actions azure-sql streamlit duckdb batch-inference saas-metrics
-
Updated
Sep 27, 2026 - Python
A lightweight, production-style Kubernetes setup that deploys, mounts volumes to, and executes a containerized Python ML inference job on a local cluster using KinD (Kubernetes in Docker).
-
Updated
Oct 2, 2026 - Python
End-to-end AWS SageMaker model training, real-time deployment, and batch inference workflow
-
Updated
Aug 18, 2026 - Python
RankShift Serving — batch engagement prediction with per-row fault isolation
python machine-learning fault-tolerance rest-api model-serving mlops fastapi streamlit engagement-prediction batch-inference
-
Updated
Sep 14, 2026 - Python
Add this topic to your repo
To associate your repository with the batch-inference topic, visit your repo's landing page and select "manage topics."