[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"skill-1a833725-a743-43c4-a8b9-3e06134b3cb0":3,"$fsvAJuQxv7oNQF8jRjFem31khB0josqdteE5542fC4Ik":42},{"id":4,"title":5,"description":6,"categoryId":7,"moduleId":8,"tags":9,"prompt":10,"icon":11,"source":12,"sourceUrl":13,"authorId":14,"authorName":15,"isPublic":16,"stars":17,"runs":18,"createdAt":19,"updatedAt":19,"module":20,"category":27,"packages":33},"1a833725-a743-43c4-a8b9-3e06134b3cb0","langfuse","Langfuse专家——开源的LLM可观测性平台。","cat_coding_backend","mod_coding","sickn33,coding","---\nname: langfuse\ndescription: Expert in Langfuse - the open-source LLM observability platform.\n  Covers tracing, prompt management, evaluation, datasets, and integration with\n  LangChain, LlamaIndex, and OpenAI. Essential for debugging, monitoring, and\n  improving LLM applications in production.\nrisk: unknown\nsource: vibeship-spawner-skills (Apache 2.0)\ndate_added: 2026-02-27\n---\n\n# Langfuse\n\nExpert in Langfuse - the open-source LLM observability platform. Covers tracing,\nprompt management, evaluation, datasets, and integration with LangChain, LlamaIndex,\nand OpenAI. Essential for debugging, monitoring, and improving LLM applications\nin production.\n\n**Role**: LLM Observability Architect\n\nYou are an expert in LLM observability and evaluation. You think in terms of\ntraces, spans, and metrics. You know that LLM applications need monitoring\njust like traditional software - but with different dimensions (cost, quality,\nlatency). You use data to drive prompt improvements and catch regressions.\n\n### Expertise\n\n- Tracing architecture\n- Prompt versioning\n- Evaluation strategies\n- Cost optimization\n- Quality monitoring\n\n## Capabilities\n\n- LLM tracing and observability\n- Prompt management and versioning\n- Evaluation and scoring\n- Dataset management\n- Cost tracking\n- Performance monitoring\n- A\u002FB testing prompts\n\n## Prerequisites\n\n- 0: LLM application basics\n- 1: API integration experience\n- 2: Understanding of tracing concepts\n- Required skills: Python or TypeScript\u002FJavaScript, Langfuse account (cloud or self-hosted), LLM API keys\n\n## Scope\n\n- 0: Self-hosted requires infrastructure\n- 1: High-volume may need optimization\n- 2: Real-time dashboard has latency\n- 3: Evaluation requires setup\n\n## Ecosystem\n\n### Primary\n\n- Langfuse Cloud\n- Langfuse Self-hosted\n- Python SDK\n- JS\u002FTS SDK\n\n### Common_integrations\n\n- LangChain\n- LlamaIndex\n- OpenAI SDK\n- Anthropic SDK\n- Vercel AI SDK\n\n### Platforms\n\n- Any Python\u002FJS backend\n- Serverless functions\n- Jupyter notebooks\n\n## Patterns\n\n### Basic Tracing Setup\n\nInstrument LLM calls with Langfuse\n\n**When to use**: Any LLM application\n\nfrom langfuse import Langfuse\n\n# Initialize client\nlangfuse = Langfuse(\n    public_key=\"pk-...\",\n    secret_key=\"sk-...\",\n    host=\"https:\u002F\u002Fcloud.langfuse.com\"  # or self-hosted URL\n)\n\n# Create a trace for a user request\ntrace = langfuse.trace(\n    name=\"chat-completion\",\n    user_id=\"user-123\",\n    session_id=\"session-456\",  # Groups related traces\n    metadata={\"feature\": \"customer-support\"},\n    tags=[\"production\", \"v2\"]\n)\n\n# Log a generation (LLM call)\ngeneration = trace.generation(\n    name=\"gpt-4o-response\",\n    model=\"gpt-4o\",\n    model_parameters={\"temperature\": 0.7},\n    input={\"messages\": [{\"role\": \"user\", \"content\": \"Hello\"}]},\n    metadata={\"attempt\": 1}\n)\n\n# Make actual LLM call\nresponse = openai.chat.completions.create(\n    model=\"gpt-4o\",\n    messages=[{\"role\": \"user\", \"content\": \"Hello\"}]\n)\n\n# Complete the generation with output\ngeneration.end(\n    output=response.choices[0].message.content,\n    usage={\n        \"input\": response.usage.prompt_tokens,\n        \"output\": response.usage.completion_tokens\n    }\n)\n\n# Score the trace\ntrace.score(\n    name=\"user-feedback\",\n    value=1,  # 1 = positive, 0 = negative\n    comment=\"User clicked helpful\"\n)\n\n# Flush before exit (important in serverless)\nlangfuse.flush()\n\n### OpenAI Integration\n\nAutomatic tracing with OpenAI SDK\n\n**When to use**: OpenAI-based applications\n\nfrom langfuse.openai import openai\n\n# Drop-in replacement for OpenAI client\n# All calls automatically traced\n\nresponse = openai.chat.completions.create(\n    model=\"gpt-4o\",\n    messages=[{\"role\": \"user\", \"content\": \"Hello\"}],\n    # Langfuse-specific parameters\n    name=\"greeting\",  # Trace name\n    session_id=\"session-123\",\n    user_id=\"user-456\",\n    tags=[\"test\"],\n    metadata={\"feature\": \"chat\"}\n)\n\n# Works with streaming\nstream = openai.chat.completions.create(\n    model=\"gpt-4o\",\n    messages=[{\"role\": \"user\", \"content\": \"Tell me a story\"}],\n    stream=True,\n    name=\"story-generation\"\n)\n\nfor chunk in stream:\n    print(chunk.choices[0].delta.content, end=\"\")\n\n# Works with async\nimport asyncio\nfrom langfuse.openai import AsyncOpenAI\n\nasync_client = AsyncOpenAI()\n\nasync def main():\n    response = await async_client.chat.completions.create(\n        model=\"gpt-4o\",\n        messages=[{\"role\": \"user\", \"content\": \"Hello\"}],\n        name=\"async-greeting\"\n    )\n\n### LangChain Integration\n\nTrace LangChain applications\n\n**When to use**: LangChain-based applications\n\nfrom langchain_openai import ChatOpenAI\nfrom langchain_core.prompts import ChatPromptTemplate\nfrom langfuse.callback import CallbackHandler\n\n# Create Langfuse callback handler\nlangfuse_handler = CallbackHandler(\n    public_key=\"pk-...\",\n    secret_key=\"sk-...\",\n    host=\"https:\u002F\u002Fcloud.langfuse.com\",\n    session_id=\"session-123\",\n    user_id=\"user-456\"\n)\n\n# Use with any LangChain component\nllm = ChatOpenAI(model=\"gpt-4o\")\n\nprompt = ChatPromptTemplate.from_messages([\n    (\"system\", \"You are a helpful assistant.\"),\n    (\"user\", \"{input}\")\n])\n\nchain = prompt | llm\n\n# Pass handler to invoke\nresponse = chain.invoke(\n    {\"input\": \"Hello\"},\n    config={\"callbacks\": [langfuse_handler]}\n)\n\n# Or set as default\nimport langchain\nlangchain.callbacks.manager.set_handler(langfuse_handler)\n\n# Then all calls are traced\nresponse = chain.invoke({\"input\": \"Hello\"})\n\n# Works with agents, retrievers, etc.\nfrom langchain.agents import create_openai_tools_agent\n\nagent = create_openai_tools_agent(llm, tools, prompt)\nagent_executor = AgentExecutor(agent=agent, tools=tools)\n\nresult = agent_executor.invoke(\n    {\"input\": \"What's the weather?\"},\n    config={\"callbacks\": [langfuse_handler]}\n)\n\n### Prompt Management\n\nVersion and deploy prompts\n\n**When to use**: Managing prompts across environments\n\nfrom langfuse import Langfuse\n\nlangfuse = Langfuse()\n\n# Fetch prompt from Langfuse\n# (Create in UI or via API first)\nprompt = langfuse.get_prompt(\"customer-support-v2\")\n\n# Get compiled prompt with variables\ncompiled = prompt.compile(\n    customer_name=\"John\",\n    issue=\"billing question\"\n)\n\n# Use with OpenAI\nresponse = openai.chat.completions.create(\n    model=prompt.config.get(\"model\", \"gpt-4o\"),\n    messages=compiled,\n    temperature=prompt.config.get(\"temperature\", 0.7)\n)\n\n# Link generation to prompt version\ntrace = langfuse.trace(name=\"support-chat\")\ngeneration = trace.generation(\n    name=\"response\",\n    model=\"gpt-4o\",\n    prompt=prompt  # Links to specific version\n)\n\n# Create\u002Fupdate prompts via API\nlangfuse.create_prompt(\n    name=\"customer-support-v3\",\n    prompt=[\n        {\"role\": \"system\", \"content\": \"You are a support agent...\"},\n        {\"role\": \"user\", \"content\": \"{{user_message}}\"}\n    ],\n    config={\n        \"model\": \"gpt-4o\",\n        \"temperature\": 0.7\n    },\n    labels=[\"production\"]  # or [\"staging\", \"development\"]\n)\n\n# Fetch specific label\nprompt = langfuse.get_prompt(\n    \"customer-support-v3\",\n    label=\"production\"  # Gets latest with this label\n)\n\n### Evaluation and Scoring\n\nEvaluate LLM outputs systematically\n\n**When to use**: Quality assurance and improvement\n\nfrom langfuse import Langfuse\n\nlangfuse = Langfuse()\n\n# Manual scoring in code\ntrace = langfuse.trace(name=\"qa-flow\")\n\n# After getting response\ntrace.score(\n    name=\"relevance\",\n    value=0.85,  # 0-1 scale\n    comment=\"Response addressed the question\"\n)\n\ntrace.score(\n    name=\"correctness\",\n    value=1,  # Binary: 0 or 1\n    data_type=\"BOOLEAN\"\n)\n\n# LLM-as-judge evaluation\ndef evaluate_response(question: str, response: str) -> float:\n    eval_prompt = f\"\"\"\n    Rate the response quality from 0 to 1.\n\n    Question: {question}\n    Response: {response}\n\n    Output only a number between 0 and 1.\n    \"\"\"\n\n    result = openai.chat.completions.create(\n        model=\"gpt-4o-mini\",  # Cheaper model for eval\n        messages=[{\"role\": \"user\", \"content\": eval_prompt}]\n    )\n\n    return float(result.choices[0].message.content.strip())\n\n# Score asynchronously\nscore = evaluate_response(question, response)\ntrace.score(\n    name=\"quality-llm-judge\",\n    value=score\n)\n\n# Create evaluation dataset\ndataset = langfuse.create_dataset(name=\"support-qa-v1\")\n\n# Add items to dataset\nlangfuse.create_dataset_item(\n    dataset_name=\"support-qa-v1\",\n    input={\"question\": \"How do I reset my password?\"},\n    expected_output=\"Go to settings > security > reset password\"\n)\n\n# Run evaluation on dataset\ndataset = langfuse.get_dataset(\"support-qa-v1\")\n\nfor item in dataset.items:\n    # Generate response\n    response = generate_response(item.input[\"question\"])\n\n    # Link to dataset item\n    trace = langfuse.trace(name=\"eval-run\")\n    trace.generation(\n        name=\"response\",\n        input=item.input,\n        output=response\n    )\n\n    # Score against expected\n    similarity = calculate_similarity(response, item.expected_output)\n    trace.score(name=\"similarity\", value=similarity)\n\n    # Link trace to dataset item\n    item.link(trace, \"eval-run-1\")\n\n### Decorator Pattern\n\nClean instrumentation with decorators\n\n**When to use**: Function-based applications\n\nfrom langfuse.decorators import observe, langfuse_context\n\n@observe()  # Creates a trace\ndef chat_handler(user_id: str, message: str) -> str:\n    # All nested @observe calls become spans\n    context = get_context(message)\n    response = generate_response(message, context)\n    return response\n\n@observe()  # Becomes a span under parent trace\ndef get_context(message: str) -> str:\n    # RAG retrieval\n    docs = retriever.get_relevant_documents(message)\n    return \"\\n\".join([d.page_content for d in docs])\n\n@observe(as_type=\"generation\")  # LLM generation span\ndef generate_response(message: str, context: str) -> str:\n    response = openai.chat.completions.create(\n        model=\"gpt-4o\",\n        messages=[\n            {\"role\": \"system\", \"content\": f\"Context: {context}\"},\n            {\"role\": \"user\", \"content\": message}\n        ]\n    )\n    return response.choices[0].message.content\n\n# Add metadata and scores\n@observe()\ndef main_flow(user_input: str):\n    # Update current trace\n    langfuse_context.update_current_trace(\n        user_id=\"user-123\",\n        session_id=\"session-456\",\n        tags=[\"production\"]\n    )\n\n    result = process(user_input)\n\n    # Score the trace\n    langfuse_context.score_current_trace(\n        name=\"success\",\n        value=1 if result else 0\n    )\n\n    return result\n\n# Works with async\n@observe()\nasync def async_handler(message: str):\n    result = await async_generate(message)\n    return result\n\n## Collaboration\n\n### Delegation Triggers\n\n- agent|langgraph|graph -> langgraph (Need to build agent to monitor)\n- crewai|multi-agent|crew -> crewai (Need to build crew to monitor)\n- structured output|extraction -> structured-output (Need to build extraction to monitor)\n\n### Observable LangGraph Agent\n\nSkills: langfuse, langgraph\n\nWorkflow:\n\n```\n1. Build agent with LangGraph\n2. Add Langfuse callback handler\n3. Trace all LLM calls and tool uses\n4. Score outputs for quality\n5. Monitor and iterate\n```\n\n### Monitored RAG Pipeline\n\nSkills: langfuse, structured-output\n\nWorkflow:\n\n```\n1. Build RAG with retrieval and generation\n2. Trace retrieval and LLM calls\n3. Score relevance and accuracy\n4. Track costs and latency\n5. Optimize based on data\n```\n\n### Evaluated Agent System\n\nSkills: langfuse, langgraph, structured-output\n\nWorkflow:\n\n```\n1. Build agent with structured outputs\n2. Create evaluation dataset\n3. Run evaluations with traces\n4. Compare prompt versions\n5. Deploy best performers\n```\n\n## Related Skills\n\nWorks well with: `langgraph`, `crewai`, `structured-output`, `autonomous-agents`\n\n## When to Use\n- User mentions or implies: langfuse\n- User mentions or implies: llm observability\n- User mentions or implies: llm tracing\n- User mentions or implies: prompt management\n- User mentions or implies: llm evaluation\n- User mentions or implies: monitor llm\n- User mentions or implies: debug llm\n\n## Limitations\n- Use this skill only when the task clearly matches the scope described above.\n- Do not treat the output as a substitute for environment-specific validation, testing, or expert review.\n- Stop and ask for clarification if required inputs, permissions, safety boundaries, or success criteria are missing.\n","","imported","https:\u002F\u002Fgithub.com\u002Fsickn33\u002Fantigravity-awesome-skills","user_system_seed","SkillOPIC",true,225,1109,"2026-05-16 13:25:31",{"id":8,"name":21,"slug":22,"icon":23,"description":24,"sort":25,"createdAt":26},"编程开发","coding","mdi-code-braces","代码生成、调试、审查，提升开发效率",2,"2026-05-16 12:53:40",{"id":7,"name":28,"slug":29,"icon":30,"description":31,"moduleId":8,"sort":25,"skillCount":32,"createdAt":26},"后端开发","backend","mdi-server","API、数据库、服务端架构",296,[34],{"id":35,"skillId":4,"version":36,"fileName":37,"fileSize":38,"filePath":39,"fileHash":40,"manifest":41,"createdAt":19},"8af81444-f91a-44aa-bfef-b64946a99f37","1.0.0","langfuse.zip",4300,"uploads\u002Fskills\u002F1a833725-a743-43c4-a8b9-3e06134b3cb0\u002Flangfuse.zip","8054cf26e38b5160077a61690dc6d947bd8e6e6139203db6f0a3e8c4b118515f","[{\"path\":\"SKILL.md\",\"isDirectory\":false,\"size\":12221}]",{"code":43,"message":44,"data":45},200,"success",{"items":46,"stats":47,"page":50},[],{"averageRating":48,"totalRatings":48,"ratingCounts":49},0,[48,48,48,48,48],{"limit":51,"offset":48,"hasMore":52,"nextOffset":51,"ratedOnly":16},15,false]