[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"skill-aa394235-9f8c-4863-b1c7-29e331d18884":3,"$fmxcDwUXFIhApbF53enUr8bIwQETjf2ZhCeCjGnSSxG4":42},{"id":4,"title":5,"description":6,"categoryId":7,"moduleId":8,"tags":9,"prompt":10,"icon":11,"source":12,"sourceUrl":13,"authorId":14,"authorName":15,"isPublic":16,"stars":17,"runs":18,"createdAt":19,"updatedAt":19,"module":20,"category":27,"packages":33},"aa394235-9f8c-4863-b1c7-29e331d18884","hugging-face-datasets","在Hugging Face Hub上创建和管理数据集。支持初始化仓库、定义配置\u002F系统提示、流式行更新以及基于SQL的数据集查询\u002F转换。专为与HF MCP服务器协同工作以实现全面的数据集工作流程而设计。","cat_coding_backend","mod_coding","sickn33,coding","---\nname: hugging-face-datasets\ndescription: Create and manage datasets on Hugging Face Hub. Supports initializing repos, defining configs\u002Fsystem prompts, streaming row updates, and SQL-based dataset querying\u002Ftransformation. Designed to work alongside HF MCP server for comprehensive dataset workflows.\nrisk: unknown\nsource: community\n---\n\n# Overview\nThis skill provides tools to manage datasets on the Hugging Face Hub with a focus on creation, configuration, content management, and SQL-based data manipulation. It is designed to complement the existing Hugging Face MCP server by providing dataset editing and querying capabilities.\n\n## When to Use\n- You need to create, configure, or update datasets on the Hugging Face Hub.\n- You want SQL-style querying, transformation, or export flows over Hub datasets.\n- You are managing dataset content and metadata directly rather than only searching existing datasets.\n\n## Integration with HF MCP Server\n- **Use HF MCP Server for**: Dataset discovery, search, and metadata retrieval\n- **Use This Skill for**: Dataset creation, content editing, SQL queries, data transformation, and structured data formatting\n\n# Version\n2.1.0\n\n# Dependencies\n# This skill uses PEP 723 scripts with inline dependency management\n# Scripts auto-install requirements when run with: uv run scripts\u002Fscript_name.py\n\n- uv (Python package manager)\n- Getting Started: See \"Usage Instructions\" below for PEP 723 usage\n\n# Core Capabilities\n\n## 1. Dataset Lifecycle Management\n- **Initialize**: Create new dataset repositories with proper structure\n- **Configure**: Store detailed configuration including system prompts and metadata\n- **Stream Updates**: Add rows efficiently without downloading entire datasets\n\n## 2. SQL-Based Dataset Querying (NEW)\nQuery any Hugging Face dataset using DuckDB SQL via `scripts\u002Fsql_manager.py`:\n- **Direct Queries**: Run SQL on datasets using the `hf:\u002F\u002F` protocol\n- **Schema Discovery**: Describe dataset structure and column types\n- **Data Sampling**: Get random samples for exploration\n- **Aggregations**: Count, histogram, unique values analysis\n- **Transformations**: Filter, join, reshape data with SQL\n- **Export & Push**: Save results locally or push to new Hub repos\n\n## 3. Multi-Format Dataset Support\nSupports diverse dataset types through template system:\n- **Chat\u002FConversational**: Chat templating, multi-turn dialogues, tool usage examples\n- **Text Classification**: Sentiment analysis, intent detection, topic classification\n- **Question-Answering**: Reading comprehension, factual QA, knowledge bases\n- **Text Completion**: Language modeling, code completion, creative writing\n- **Tabular Data**: Structured data for regression\u002Fclassification tasks\n- **Custom Formats**: Flexible schema definition for specialized needs\n\n## 4. Quality Assurance Features\n- **JSON Validation**: Ensures data integrity during uploads\n- **Batch Processing**: Efficient handling of large datasets\n- **Error Recovery**: Graceful handling of upload failures and conflicts\n\n# Usage Instructions\n\nThe skill includes two Python scripts that use PEP 723 inline dependency management:\n\n> **All paths are relative to the directory containing this SKILL.md\nfile.**\n> Scripts are run with: `uv run scripts\u002Fscript_name.py [arguments]`\n\n- `scripts\u002Fdataset_manager.py` - Dataset creation and management\n- `scripts\u002Fsql_manager.py` - SQL-based dataset querying and transformation\n\n### Prerequisites\n- `uv` package manager installed\n- `HF_TOKEN` environment variable must be set with a Write-access token\n\n---\n\n# SQL Dataset Querying (sql_manager.py)\n\nQuery, transform, and push Hugging Face datasets using DuckDB SQL. The `hf:\u002F\u002F` protocol provides direct access to any public dataset (or private with token).\n\n## Quick Start\n\n```bash\n# Query a dataset\nuv run scripts\u002Fsql_manager.py query \\\n  --dataset \"cais\u002Fmmlu\" \\\n  --sql \"SELECT * FROM data WHERE subject='nutrition' LIMIT 10\"\n\n# Get dataset schema\nuv run scripts\u002Fsql_manager.py describe --dataset \"cais\u002Fmmlu\"\n\n# Sample random rows\nuv run scripts\u002Fsql_manager.py sample --dataset \"cais\u002Fmmlu\" --n 5\n\n# Count rows with filter\nuv run scripts\u002Fsql_manager.py count --dataset \"cais\u002Fmmlu\" --where \"subject='nutrition'\"\n```\n\n## SQL Query Syntax\n\nUse `data` as the table name in your SQL - it gets replaced with the actual `hf:\u002F\u002F` path:\n\n```sql\n-- Basic select\nSELECT * FROM data LIMIT 10\n\n-- Filtering\nSELECT * FROM data WHERE subject='nutrition'\n\n-- Aggregations\nSELECT subject, COUNT(*) as cnt FROM data GROUP BY subject ORDER BY cnt DESC\n\n-- Column selection and transformation\nSELECT question, choices[answer] AS correct_answer FROM data\n\n-- Regex matching\nSELECT * FROM data WHERE regexp_matches(question, 'nutrition|diet')\n\n-- String functions\nSELECT regexp_replace(question, '\\n', '') AS cleaned FROM data\n```\n\n## Common Operations\n\n### 1. Explore Dataset Structure\n```bash\n# Get schema\nuv run scripts\u002Fsql_manager.py describe --dataset \"cais\u002Fmmlu\"\n\n# Get unique values in column\nuv run scripts\u002Fsql_manager.py unique --dataset \"cais\u002Fmmlu\" --column \"subject\"\n\n# Get value distribution\nuv run scripts\u002Fsql_manager.py histogram --dataset \"cais\u002Fmmlu\" --column \"subject\" --bins 20\n```\n\n### 2. Filter and Transform\n```bash\n# Complex filtering with SQL\nuv run scripts\u002Fsql_manager.py query \\\n  --dataset \"cais\u002Fmmlu\" \\\n  --sql \"SELECT subject, COUNT(*) as cnt FROM data GROUP BY subject HAVING cnt > 100\"\n\n# Using transform command\nuv run scripts\u002Fsql_manager.py transform \\\n  --dataset \"cais\u002Fmmlu\" \\\n  --select \"subject, COUNT(*) as cnt\" \\\n  --group-by \"subject\" \\\n  --order-by \"cnt DESC\" \\\n  --limit 10\n```\n\n### 3. Create Subsets and Push to Hub\n```bash\n# Query and push to new dataset\nuv run scripts\u002Fsql_manager.py query \\\n  --dataset \"cais\u002Fmmlu\" \\\n  --sql \"SELECT * FROM data WHERE subject='nutrition'\" \\\n  --push-to \"username\u002Fmmlu-nutrition-subset\" \\\n  --private\n\n# Transform and push\nuv run scripts\u002Fsql_manager.py transform \\\n  --dataset \"ibm\u002Fduorc\" \\\n  --config \"ParaphraseRC\" \\\n  --select \"question, answers\" \\\n  --where \"LENGTH(question) > 50\" \\\n  --push-to \"username\u002Fduorc-long-questions\"\n```\n\n### 4. Export to Local Files\n```bash\n# Export to Parquet\nuv run scripts\u002Fsql_manager.py export \\\n  --dataset \"cais\u002Fmmlu\" \\\n  --sql \"SELECT * FROM data WHERE subject='nutrition'\" \\\n  --output \"nutrition.parquet\" \\\n  --format parquet\n\n# Export to JSONL\nuv run scripts\u002Fsql_manager.py export \\\n  --dataset \"cais\u002Fmmlu\" \\\n  --sql \"SELECT * FROM data LIMIT 100\" \\\n  --output \"sample.jsonl\" \\\n  --format jsonl\n```\n\n### 5. Working with Dataset Configs\u002FSplits\n```bash\n# Specify config (subset)\nuv run scripts\u002Fsql_manager.py query \\\n  --dataset \"ibm\u002Fduorc\" \\\n  --config \"ParaphraseRC\" \\\n  --sql \"SELECT * FROM data LIMIT 5\"\n\n# Specify split\nuv run scripts\u002Fsql_manager.py query \\\n  --dataset \"cais\u002Fmmlu\" \\\n  --split \"test\" \\\n  --sql \"SELECT COUNT(*) FROM data\"\n\n# Query all splits\nuv run scripts\u002Fsql_manager.py query \\\n  --dataset \"cais\u002Fmmlu\" \\\n  --split \"*\" \\\n  --sql \"SELECT * FROM data LIMIT 10\"\n```\n\n### 6. Raw SQL with Full Paths\nFor complex queries or joining datasets:\n```bash\nuv run scripts\u002Fsql_manager.py raw --sql \"\n  SELECT a.*, b.* \n  FROM 'hf:\u002F\u002Fdatasets\u002Fdataset1@~parquet\u002Fdefault\u002Ftrain\u002F*.parquet' a\n  JOIN 'hf:\u002F\u002Fdatasets\u002Fdataset2@~parquet\u002Fdefault\u002Ftrain\u002F*.parquet' b\n  ON a.id = b.id\n  LIMIT 100\n\"\n```\n\n## Python API Usage\n\n```python\nfrom sql_manager import HFDatasetSQL\n\nsql = HFDatasetSQL()\n\n# Query\nresults = sql.query(\"cais\u002Fmmlu\", \"SELECT * FROM data WHERE subject='nutrition' LIMIT 10\")\n\n# Get schema\nschema = sql.describe(\"cais\u002Fmmlu\")\n\n# Sample\nsamples = sql.sample(\"cais\u002Fmmlu\", n=5, seed=42)\n\n# Count\ncount = sql.count(\"cais\u002Fmmlu\", where=\"subject='nutrition'\")\n\n# Histogram\ndist = sql.histogram(\"cais\u002Fmmlu\", \"subject\")\n\n# Filter and transform\nresults = sql.filter_and_transform(\n    \"cais\u002Fmmlu\",\n    select=\"subject, COUNT(*) as cnt\",\n    group_by=\"subject\",\n    order_by=\"cnt DESC\",\n    limit=10\n)\n\n# Push to Hub\nurl = sql.push_to_hub(\n    \"cais\u002Fmmlu\",\n    \"username\u002Fnutrition-subset\",\n    sql=\"SELECT * FROM data WHERE subject='nutrition'\",\n    private=True\n)\n\n# Export locally\nsql.export_to_parquet(\"cais\u002Fmmlu\", \"output.parquet\", sql=\"SELECT * FROM data LIMIT 100\")\n\nsql.close()\n```\n\n## HF Path Format\n\nDuckDB uses the `hf:\u002F\u002F` protocol to access datasets:\n```\nhf:\u002F\u002Fdatasets\u002F{dataset_id}@{revision}\u002F{config}\u002F{split}\u002F*.parquet\n```\n\nExamples:\n- `hf:\u002F\u002Fdatasets\u002Fcais\u002Fmmlu@~parquet\u002Fdefault\u002Ftrain\u002F*.parquet`\n- `hf:\u002F\u002Fdatasets\u002Fibm\u002Fduorc@~parquet\u002FParaphraseRC\u002Ftest\u002F*.parquet`\n\nThe `@~parquet` revision provides auto-converted Parquet files for any dataset format.\n\n## Useful DuckDB SQL Functions\n\n```sql\n-- String functions\nLENGTH(column)                    -- String length\nregexp_replace(col, '\\n', '')     -- Regex replace\nregexp_matches(col, 'pattern')    -- Regex match\nLOWER(col), UPPER(col)           -- Case conversion\n\n-- Array functions  \nchoices[0]                        -- Array indexing (0-based)\narray_length(choices)             -- Array length\nunnest(choices)                   -- Expand array to rows\n\n-- Aggregations\nCOUNT(*), SUM(col), AVG(col)\nGROUP BY col HAVING condition\n\n-- Sampling\nUSING SAMPLE 10                   -- Random sample\nUSING SAMPLE 10 (RESERVOIR, 42)   -- Reproducible sample\n\n-- Window functions\nROW_NUMBER() OVER (PARTITION BY col ORDER BY col2)\n```\n\n---\n\n# Dataset Creation (dataset_manager.py)\n\n### Recommended Workflow\n\n**1. Discovery (Use HF MCP Server):**\n```python\n# Use HF MCP tools to find existing datasets\nsearch_datasets(\"conversational AI training\")\nget_dataset_details(\"username\u002Fdataset-name\")\n```\n\n**2. Creation (Use This Skill):**\n```bash\n# Initialize new dataset\nuv run scripts\u002Fdataset_manager.py init --repo_id \"your-username\u002Fdataset-name\" [--private]\n\n# Configure with detailed system prompt\nuv run scripts\u002Fdataset_manager.py config --repo_id \"your-username\u002Fdataset-name\" --system_prompt \"$(cat system_prompt.txt)\"\n```\n\n**3. Content Management (Use This Skill):**\n```bash\n# Quick setup with any template\nuv run scripts\u002Fdataset_manager.py quick_setup \\\n  --repo_id \"your-username\u002Fdataset-name\" \\\n  --template classification\n\n# Add data with template validation\nuv run scripts\u002Fdataset_manager.py add_rows \\\n  --repo_id \"your-username\u002Fdataset-name\" \\\n  --template qa \\\n  --rows_json \"$(cat your_qa_data.json)\"\n```\n\n### Template-Based Data Structures\n\n**1. Chat Template (`--template chat`)**\n```json\n{\n  \"messages\": [\n    {\"role\": \"user\", \"content\": \"Natural user request\"},\n    {\"role\": \"assistant\", \"content\": \"Response with tool usage\"},\n    {\"role\": \"tool\", \"content\": \"Tool response\", \"tool_call_id\": \"call_123\"}\n  ],\n  \"scenario\": \"Description of use case\",\n  \"complexity\": \"simple|intermediate|advanced\"\n}\n```\n\n**2. Classification Template (`--template classification`)**\n```json\n{\n  \"text\": \"Input text to be classified\",\n  \"label\": \"classification_label\",\n  \"confidence\": 0.95,\n  \"metadata\": {\"domain\": \"technology\", \"language\": \"en\"}\n}\n```\n\n**3. QA Template (`--template qa`)**\n```json\n{\n  \"question\": \"What is the question being asked?\",\n  \"answer\": \"The complete answer\",\n  \"context\": \"Additional context if needed\",\n  \"answer_type\": \"factual|explanatory|opinion\",\n  \"difficulty\": \"easy|medium|hard\"\n}\n```\n\n**4. Completion Template (`--template completion`)**\n```json\n{\n  \"prompt\": \"The beginning text or context\",\n  \"completion\": \"The expected continuation\",\n  \"domain\": \"code|creative|technical|conversational\",\n  \"style\": \"description of writing style\"\n}\n```\n\n**5. Tabular Template (`--template tabular`)**\n```json\n{\n  \"columns\": [\n    {\"name\": \"feature1\", \"type\": \"numeric\", \"description\": \"First feature\"},\n    {\"name\": \"target\", \"type\": \"categorical\", \"description\": \"Target variable\"}\n  ],\n  \"data\": [\n    {\"feature1\": 123, \"target\": \"class_a\"},\n    {\"feature1\": 456, \"target\": \"class_b\"}\n  ]\n}\n```\n\n### Advanced System Prompt Template\n\nFor high-quality training data generation:\n```text\nYou are an AI assistant expert at using MCP tools effectively.\n\n## MCP SERVER DEFINITIONS\n[Define available servers and tools]\n\n## TRAINING EXAMPLE STRUCTURE\n[Specify exact JSON schema for chat templating]\n\n## QUALITY GUIDELINES\n[Detail requirements for realistic scenarios, progressive complexity, proper tool usage]\n\n## EXAMPLE CATEGORIES\n[List development workflows, debugging scenarios, data management tasks]\n```\n\n### Example Categories & Templates\n\nThe skill includes diverse training examples beyond just MCP usage:\n\n**Available Example Sets:**\n- `training_examples.json` - MCP tool usage examples (debugging, project setup, database analysis)\n- `diverse_training_examples.json` - Broader scenarios including:\n  - **Educational Chat** - Explaining programming concepts, tutorials\n  - **Git Workflows** - Feature branches, version control guidance\n  - **Code Analysis** - Performance optimization, architecture review\n  - **Content Generation** - Professional writing, creative brainstorming\n  - **Codebase Navigation** - Legacy code exploration, systematic analysis\n  - **Conversational Support** - Problem-solving, technical discussions\n\n**Using Different Example Sets:**\n```bash\n# Add MCP-focused examples\nuv run scripts\u002Fdataset_manager.py add_rows --repo_id \"your-username\u002Fdataset-name\" \\\n  --rows_json \"$(cat examples\u002Ftraining_examples.json)\"\n\n# Add diverse conversational examples\nuv run scripts\u002Fdataset_manager.py add_rows --repo_id \"your-username\u002Fdataset-name\" \\\n  --rows_json \"$(cat examples\u002Fdiverse_training_examples.json)\"\n\n# Mix both for comprehensive training data\nuv run scripts\u002Fdataset_manager.py add_rows --repo_id \"your-username\u002Fdataset-name\" \\\n  --rows_json \"$(jq -s '.[0] + .[1]' examples\u002Ftraining_examples.json examples\u002Fdiverse_training_examples.json)\"\n```\n\n### Commands Reference\n\n**List Available Templates:**\n```bash\nuv run scripts\u002Fdataset_manager.py list_templates\n```\n\n**Quick Setup (Recommended):**\n```bash\nuv run scripts\u002Fdataset_manager.py quick_setup --repo_id \"your-username\u002Fdataset-name\" --template classification\n```\n\n**Manual Setup:**\n```bash\n# Initialize repository\nuv run scripts\u002Fdataset_manager.py init --repo_id \"your-username\u002Fdataset-name\" [--private]\n\n# Configure with system prompt\nuv run scripts\u002Fdataset_manager.py config --repo_id \"your-username\u002Fdataset-name\" --system_prompt \"Your prompt here\"\n\n# Add data with validation\nuv run scripts\u002Fdataset_manager.py add_rows \\\n  --repo_id \"your-username\u002Fdataset-name\" \\\n  --template qa \\\n  --rows_json '[{\"question\": \"What is AI?\", \"answer\": \"Artificial Intelligence...\"}]'\n```\n\n**View Dataset Statistics:**\n```bash\nuv run scripts\u002Fdataset_manager.py stats --repo_id \"your-username\u002Fdataset-name\"\n```\n\n### Error Handling\n- **Repository exists**: Script will notify and continue with configuration\n- **Invalid JSON**: Clear error message with parsing details\n- **Network issues**: Automatic retry for transient failures\n- **Token permissions**: Validation before operations begin\n\n---\n\n# Combined Workflow Examples\n\n## Example 1: Create Training Subset from Existing Dataset\n```bash\n# 1. Explore the source dataset\nuv run scripts\u002Fsql_manager.py describe --dataset \"cais\u002Fmmlu\"\nuv run scripts\u002Fsql_manager.py histogram --dataset \"cais\u002Fmmlu\" --column \"subject\"\n\n# 2. Query and create subset\nuv run scripts\u002Fsql_manager.py query \\\n  --dataset \"cais\u002Fmmlu\" \\\n  --sql \"SELECT * FROM data WHERE subject IN ('nutrition', 'anatomy', 'clinical_knowledge')\" \\\n  --push-to \"username\u002Fmmlu-medical-subset\" \\\n  --private\n```\n\n## Example 2: Transform and Reshape Data\n```bash\n# Transform MMLU to QA format with correct answers extracted\nuv run scripts\u002Fsql_manager.py query \\\n  --dataset \"cais\u002Fmmlu\" \\\n  --sql \"SELECT question, choices[answer] as correct_answer, subject FROM data\" \\\n  --push-to \"username\u002Fmmlu-qa-format\"\n```\n\n## Example 3: Merge Multiple Dataset Splits\n```bash\n# Export multiple splits and combine\nuv run scripts\u002Fsql_manager.py export \\\n  --dataset \"cais\u002Fmmlu\" \\\n  --split \"*\" \\\n  --output \"mmlu_all.parquet\"\n```\n\n## Example 4: Quality Filtering\n```bash\n# Filter for high-quality examples\nuv run scripts\u002Fsql_manager.py query \\\n  --dataset \"squad\" \\\n  --sql \"SELECT * FROM data WHERE LENGTH(context) > 500 AND LENGTH(question) > 20\" \\\n  --push-to \"username\u002Fsquad-filtered\"\n```\n\n## Example 5: Create Custom Training Dataset\n```bash\n# 1. Query source data\nuv run scripts\u002Fsql_manager.py export \\\n  --dataset \"cais\u002Fmmlu\" \\\n  --sql \"SELECT question, subject FROM data WHERE subject='nutrition'\" \\\n  --output \"nutrition_source.jsonl\" \\\n  --format jsonl\n\n# 2. Process with your pipeline (add answers, format, etc.)\n\n# 3. Push processed data\nuv run scripts\u002Fdataset_manager.py init --repo_id \"username\u002Fnutrition-training\"\nuv run scripts\u002Fdataset_manager.py add_rows \\\n  --repo_id \"username\u002Fnutrition-training\" \\\n  --template qa \\\n  --rows_json \"$(cat processed_data.json)\"\n```\n\n## Limitations\n- Use this skill only when the task clearly matches the scope described above.\n- Do not treat the output as a substitute for environment-specific validation, testing, or expert review.\n- Stop and ask for clarification if required inputs, permissions, safety boundaries, or success criteria are missing.\n","","imported","https:\u002F\u002Fgithub.com\u002Fsickn33\u002Fantigravity-awesome-skills","user_system_seed","SkillOPIC",true,216,493,"2026-05-16 13:22:34",{"id":8,"name":21,"slug":22,"icon":23,"description":24,"sort":25,"createdAt":26},"编程开发","coding","mdi-code-braces","代码生成、调试、审查，提升开发效率",2,"2026-05-16 12:53:40",{"id":7,"name":28,"slug":29,"icon":30,"description":31,"moduleId":8,"sort":25,"skillCount":32,"createdAt":26},"后端开发","backend","mdi-server","API、数据库、服务端架构",296,[34],{"id":35,"skillId":4,"version":36,"fileName":37,"fileSize":38,"filePath":39,"fileHash":40,"manifest":41,"createdAt":19},"1c5a6f42-bd91-439d-8b3f-a3053711abc6","1.0.0","hugging-face-datasets.zip",5727,"uploads\u002Fskills\u002Faa394235-9f8c-4863-b1c7-29e331d18884\u002Fhugging-face-datasets.zip","30d53327ce4e9aa3212e75155d1f80b7e4642a7dd908f36092e09fcd9b8c8d59","[{\"path\":\"SKILL.md\",\"isDirectory\":false,\"size\":17003}]",{"code":43,"message":44,"data":45},200,"success",{"items":46,"stats":47,"page":50},[],{"averageRating":48,"totalRatings":48,"ratingCounts":49},0,[48,48,48,48,48],{"limit":51,"offset":48,"hasMore":52,"nextOffset":51,"ratedOnly":16},15,false]