[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"skill-77bac531-77b5-4df7-ad3b-73ac157298a7":3,"$fAN-qmCADPkAYQCWMF-b5yMnHj2CRuMq-nvB5b6P5aaE":43},{"id":4,"title":5,"description":6,"categoryId":7,"moduleId":8,"tags":9,"prompt":10,"icon":11,"source":12,"sourceUrl":13,"authorId":14,"authorName":15,"isPublic":16,"stars":17,"runs":18,"createdAt":19,"updatedAt":19,"module":20,"category":27,"packages":34},"77bac531-77b5-4df7-ad3b-73ac157298a7","senior-ml-engineer","机器学习工程技能，涵盖模型生产化、构建MLOps管道和集成LLMs。包括模型部署、特征存储、漂移监控、RAG系统以及成本优化。当用户询问将ML模型部署到生产、设置MLOps基础设施（MLflow、Kubeflow、Kubernetes、Docker）、监控模型性能或漂移、构建RAG管道或集成LLM API并具有重试逻辑和成本控制时使用。专注于生产和运营问题。","cat_coding_devops","mod_coding","alirezarezvani,coding","---\nname: \"senior-ml-engineer\"\ndescription: ML engineering skill for productionizing models, building MLOps pipelines, and integrating LLMs. Covers model deployment, feature stores, drift monitoring, RAG systems, and cost optimization. Use when the user asks about deploying ML models to production, setting up MLOps infrastructure (MLflow, Kubeflow, Kubernetes, Docker), monitoring model performance or drift, building RAG pipelines, or integrating LLM APIs with retry logic and cost controls. Focused on production and operational concerns rather than model research or initial training.\ntriggers:\n  - MLOps pipeline\n  - model deployment\n  - feature store\n  - model monitoring\n  - drift detection\n  - RAG system\n  - LLM integration\n  - model serving\n  - A\u002FB testing ML\n  - automated retraining\n---\n\n# Senior ML Engineer\n\nProduction ML engineering patterns for model deployment, MLOps infrastructure, and LLM integration.\n\n---\n\n## Table of Contents\n\n- [Model Deployment Workflow](#model-deployment-workflow)\n- [MLOps Pipeline Setup](#mlops-pipeline-setup)\n- [LLM Integration Workflow](#llm-integration-workflow)\n- [RAG System Implementation](#rag-system-implementation)\n- [Model Monitoring](#model-monitoring)\n- [Reference Documentation](#reference-documentation)\n- [Tools](#tools)\n\n---\n\n## Model Deployment Workflow\n\nDeploy a trained model to production with monitoring:\n\n1. Export model to standardized format (ONNX, TorchScript, SavedModel)\n2. Package model with dependencies in Docker container\n3. Deploy to staging environment\n4. Run integration tests against staging\n5. Deploy canary (5% traffic) to production\n6. Monitor latency and error rates for 1 hour\n7. Promote to full production if metrics pass\n8. **Validation:** p95 latency \u003C 100ms, error rate \u003C 0.1%\n\n### Container Template\n\n```dockerfile\nFROM python:3.11-slim\n\nCOPY requirements.txt .\nRUN pip install --no-cache-dir -r requirements.txt\n\nCOPY model\u002F \u002Fapp\u002Fmodel\u002F\nCOPY src\u002F \u002Fapp\u002Fsrc\u002F\n\nHEALTHCHECK CMD curl -f http:\u002F\u002Flocalhost:8080\u002Fhealth || exit 1\n\nEXPOSE 8080\nCMD [\"uvicorn\", \"src.server:app\", \"--host\", \"0.0.0.0\", \"--port\", \"8080\"]\n```\n\n### Serving Options\n\n| Option | Latency | Throughput | Use Case |\n|--------|---------|------------|----------|\n| FastAPI + Uvicorn | Low | Medium | REST APIs, small models |\n| Triton Inference Server | Very Low | Very High | GPU inference, batching |\n| TensorFlow Serving | Low | High | TensorFlow models |\n| TorchServe | Low | High | PyTorch models |\n| Ray Serve | Medium | High | Complex pipelines, multi-model |\n\n---\n\n## MLOps Pipeline Setup\n\nEstablish automated training and deployment:\n\n1. Configure feature store (Feast, Tecton) for training data\n2. Set up experiment tracking (MLflow, Weights & Biases)\n3. Create training pipeline with hyperparameter logging\n4. Register model in model registry with version metadata\n5. Configure staging deployment triggered by registry events\n6. Set up A\u002FB testing infrastructure for model comparison\n7. Enable drift monitoring with alerting\n8. **Validation:** New models automatically evaluated against baseline\n\n### Feature Store Pattern\n\n```python\nfrom feast import Entity, Feature, FeatureView, FileSource\n\nuser = Entity(name=\"user_id\", value_type=ValueType.INT64)\n\nuser_features = FeatureView(\n    name=\"user_features\",\n    entities=[\"user_id\"],\n    ttl=timedelta(days=1),\n    features=[\n        Feature(name=\"purchase_count_30d\", dtype=ValueType.INT64),\n        Feature(name=\"avg_order_value\", dtype=ValueType.FLOAT),\n    ],\n    online=True,\n    source=FileSource(path=\"data\u002Fuser_features.parquet\"),\n)\n```\n\n### Retraining Triggers\n\n| Trigger | Detection | Action |\n|---------|-----------|--------|\n| Scheduled | Cron (weekly\u002Fmonthly) | Full retrain |\n| Performance drop | Accuracy \u003C threshold | Immediate retrain |\n| Data drift | PSI > 0.2 | Evaluate, then retrain |\n| New data volume | X new samples | Incremental update |\n\n---\n\n## LLM Integration Workflow\n\nIntegrate LLM APIs into production applications:\n\n1. Create provider abstraction layer for vendor flexibility\n2. Implement retry logic with exponential backoff\n3. Configure fallback to secondary provider\n4. Set up token counting and context truncation\n5. Add response caching for repeated queries\n6. Implement cost tracking per request\n7. Add structured output validation with Pydantic\n8. **Validation:** Response parses correctly, cost within budget\n\n### Provider Abstraction\n\n```python\nfrom abc import ABC, abstractmethod\nfrom tenacity import retry, stop_after_attempt, wait_exponential\n\nclass LLMProvider(ABC):\n    @abstractmethod\n    def complete(self, prompt: str, **kwargs) -> str:\n        pass\n\n@retry(stop=stop_after_attempt(3), wait=wait_exponential(min=1, max=10))\ndef call_llm_with_retry(provider: LLMProvider, prompt: str) -> str:\n    return provider.complete(prompt)\n```\n\n### Cost Management\n\n| Provider | Input Cost | Output Cost |\n|----------|------------|-------------|\n| GPT-4 | $0.03\u002F1K | $0.06\u002F1K |\n| GPT-3.5 | $0.0005\u002F1K | $0.0015\u002F1K |\n| Claude 3 Opus | $0.015\u002F1K | $0.075\u002F1K |\n| Claude 3 Haiku | $0.00025\u002F1K | $0.00125\u002F1K |\n\n---\n\n## RAG System Implementation\n\nBuild retrieval-augmented generation pipeline:\n\n1. Choose vector database (Pinecone, Qdrant, Weaviate)\n2. Select embedding model based on quality\u002Fcost tradeoff\n3. Implement document chunking strategy\n4. Create ingestion pipeline with metadata extraction\n5. Build retrieval with query embedding\n6. Add reranking for relevance improvement\n7. Format context and send to LLM\n8. **Validation:** Response references retrieved context, no hallucinations\n\n### Vector Database Selection\n\n| Database | Hosting | Scale | Latency | Best For |\n|----------|---------|-------|---------|----------|\n| Pinecone | Managed | High | Low | Production, managed |\n| Qdrant | Both | High | Very Low | Performance-critical |\n| Weaviate | Both | High | Low | Hybrid search |\n| Chroma | Self-hosted | Medium | Low | Prototyping |\n| pgvector | Self-hosted | Medium | Medium | Existing Postgres |\n\n### Chunking Strategies\n\n| Strategy | Chunk Size | Overlap | Best For |\n|----------|------------|---------|----------|\n| Fixed | 500-1000 tokens | 50-100 | General text |\n| Sentence | 3-5 sentences | 1 sentence | Structured text |\n| Semantic | Variable | Based on meaning | Research papers |\n| Recursive | Hierarchical | Parent-child | Long documents |\n\n---\n\n## Model Monitoring\n\nMonitor production models for drift and degradation:\n\n1. Set up latency tracking (p50, p95, p99)\n2. Configure error rate alerting\n3. Implement input data drift detection\n4. Track prediction distribution shifts\n5. Log ground truth when available\n6. Compare model versions with A\u002FB metrics\n7. Set up automated retraining triggers\n8. **Validation:** Alerts fire before user-visible degradation\n\n### Drift Detection\n\n```python\nfrom scipy.stats import ks_2samp\n\ndef detect_drift(reference, current, threshold=0.05):\n    statistic, p_value = ks_2samp(reference, current)\n    return {\n        \"drift_detected\": p_value \u003C threshold,\n        \"ks_statistic\": statistic,\n        \"p_value\": p_value\n    }\n```\n\n### Alert Thresholds\n\n| Metric | Warning | Critical |\n|--------|---------|----------|\n| p95 latency | > 100ms | > 200ms |\n| Error rate | > 0.1% | > 1% |\n| PSI (drift) | > 0.1 | > 0.2 |\n| Accuracy drop | > 2% | > 5% |\n\n---\n\n## Reference Documentation\n\n### MLOps Production Patterns\n\n`references\u002Fmlops_production_patterns.md` contains:\n\n- Model deployment pipeline with Kubernetes manifests\n- Feature store architecture with Feast examples\n- Model monitoring with drift detection code\n- A\u002FB testing infrastructure with traffic splitting\n- Automated retraining pipeline with MLflow\n\n### LLM Integration Guide\n\n`references\u002Fllm_integration_guide.md` contains:\n\n- Provider abstraction layer pattern\n- Retry and fallback strategies with tenacity\n- Prompt engineering templates (few-shot, CoT)\n- Token optimization with tiktoken\n- Cost calculation and tracking\n\n### RAG System Architecture\n\n`references\u002Frag_system_architecture.md` contains:\n\n- RAG pipeline implementation with code\n- Vector database comparison and integration\n- Chunking strategies (fixed, semantic, recursive)\n- Embedding model selection guide\n- Hybrid search and reranking patterns\n\n---\n\n## Tools\n\n### Model Deployment Pipeline\n\n```bash\npython scripts\u002Fmodel_deployment_pipeline.py --model model.pkl --target staging\n```\n\nGenerates deployment artifacts: Dockerfile, Kubernetes manifests, health checks.\n\n### RAG System Builder\n\n```bash\npython scripts\u002Frag_system_builder.py --config rag_config.yaml --analyze\n```\n\nScaffolds RAG pipeline with vector store integration and retrieval logic.\n\n### ML Monitoring Suite\n\n```bash\npython scripts\u002Fml_monitoring_suite.py --config monitoring.yaml --deploy\n```\n\nSets up drift detection, alerting, and performance dashboards.\n\n---\n\n## Tech Stack\n\n| Category | Tools |\n|----------|-------|\n| ML Frameworks | PyTorch, TensorFlow, Scikit-learn, XGBoost |\n| LLM Frameworks | LangChain, LlamaIndex, DSPy |\n| MLOps | MLflow, Weights & Biases, Kubeflow |\n| Data | Spark, Airflow, dbt, Kafka |\n| Deployment | Docker, Kubernetes, Triton |\n| Databases | PostgreSQL, BigQuery, Pinecone, Redis |\n","","imported","https:\u002F\u002Fgithub.com\u002Falirezarezvani\u002Fclaude-skills","user_system_seed","SkillOPIC",true,232,1059,"2026-05-16 13:57:47",{"id":8,"name":21,"slug":22,"icon":23,"description":24,"sort":25,"createdAt":26},"编程开发","coding","mdi-code-braces","代码生成、调试、审查，提升开发效率",2,"2026-05-16 12:53:40",{"id":7,"name":28,"slug":29,"icon":30,"description":31,"moduleId":8,"sort":32,"skillCount":33,"createdAt":26},"DevOps","devops","mdi-cog-outline","CI\u002FCD、容器化、部署运维",3,162,[35],{"id":36,"skillId":4,"version":37,"fileName":38,"fileSize":39,"filePath":40,"fileHash":41,"manifest":42,"createdAt":19},"5be2574d-09ed-40cb-87e8-75e0c0a0624b","1.0.0","senior-ml-engineer.zip",17372,"uploads\u002Fskills\u002F77bac531-77b5-4df7-ad3b-73ac157298a7\u002Fsenior-ml-engineer.zip","8a17e2853c4bc71bf89bb4865f9f16c4b1f29dfad8d03c2fa60f6ab01f4b866e","[{\"path\":\"SKILL.md\",\"isDirectory\":false,\"size\":9119},{\"path\":\"references\u002Fllm_integration_guide.md\",\"isDirectory\":false,\"size\":8398},{\"path\":\"references\u002Fmlops_production_patterns.md\",\"isDirectory\":false,\"size\":7307},{\"path\":\"references\u002Frag_system_architecture.md\",\"isDirectory\":false,\"size\":10286},{\"path\":\"scripts\u002Fml_monitoring_suite.py\",\"isDirectory\":false,\"size\":2782},{\"path\":\"scripts\u002Fmodel_deployment_pipeline.py\",\"isDirectory\":false,\"size\":2812},{\"path\":\"scripts\u002Frag_system_builder.py\",\"isDirectory\":false,\"size\":2777}]",{"code":44,"message":45,"data":46},200,"success",{"items":47,"stats":48,"page":51},[],{"averageRating":49,"totalRatings":49,"ratingCounts":50},0,[49,49,49,49,49],{"limit":52,"offset":49,"hasMore":53,"nextOffset":52,"ratedOnly":16},15,false]