hxxra

# hxxra

This skill is a Research Assistant that helps users search, download, analyze, report, and save research papers.

## Recommended Directory Structure

For better organization, it is recommended to create a dedicated workspace for `hxxra` under your OpenClaw working directory:

```
📁 workspace/                              # OpenClaw current working directory
└── 📁 hxxra/
    ├── 📁 searches/                       # Stores all search result JSON files
        ├── 2025-03-07_neural_radiance_fields_arxiv.json
        ├── 2025-03-07_transformer_architectures_scholar.json
        └── ...
    ├── 📁 papers/                           # Stores downloaded PDF files and per-paper analysis results (each as a subfolder)
        ├── papers_report.md                # Generated Markdown report summarizing all analyzed papers
        ├── 2023_Smith_NeRF_Explained/      # Folder named after the PDF (without extension)
          ├── 2023_Smith_NeRF_Explained.pdf
          ├── analysis.json                 # Structured output from LLM analysis
          └── notes.md                      # (Optional) User-added notes
        ├── 2024_Zhang_Transformer_Survey/
          ├── 2024_Zhang_Transformer_Survey.pdf
          ├── analysis.json
          └── ...
        └── ...
    └── 📁 logs/ # Stores execution logs
        └── hxxra_2025-03-07.log
```

This structure keeps all related files organized and easily accessible for review and further processing.

## Core Commands

### 1. **hxxra search** - Search for research papers

**Dependencies**: `pip install scholarly`

**Purpose**: Search for papers using Google Scholar and arXiv APIs

**Academic Note**: To account for the distinct characteristics of each data source, the tool adopts a differentiated sorting strategy—**arXiv results are ordered by submission date in descending order**, prioritizing the timeliness of recent research; **Google Scholar results retain the source's default relevance ranking**, ensuring strong alignment with the query keywords while appropriately weighing influential or classical literature.

**Parameters**:

- `-q, --query <string>` (Required): Search keywords
- `-s, --source <string>` (Optional): Data source: `arxiv` (default), `scholar`
- `-l, --limit <number>` (Optional): Number of results (default: 10)
- `-o, --output <path>` (Optional): JSON output file (default: `{workspace}/hxxra/searches/search_results.json`)

**Input Examples**:

```json
{"command": "search", "query": "neural radiance fields", "source": "arxiv", "limit": 10, "output": "results.json"} | python scripts/hxxra.py
{"command": "search", "query": "transformer architecture", "source": "scholar", "limit": 15} | python scripts/hxxra.py
```

**Output Structure**:

```json
{
  "ok": true,
  "command": "search",
  "query": "<query>",
  "source": "<source>",
  "results": [
    {
      "id": "1",
      "title": "Paper Title",
      "authors": ["Author1", "Author2"],
      "year": "2023",
      "source": "arxiv",
      "abstract": "Abstract text...",
      "url": "https://arxiv.org/abs/xxxx.xxxxx",
      "pdf_url": "https://arxiv.org/pdf/xxxx.xxxxx.pdf",
      "citations": 123
    }
  ],
  "total": 10,
  "output_file": "/path/to/results.json"
}
```

------

### 2. **hxxra download** - Download PDF files

**Purpose**: Download PDFs for specified papers

**Parameters**:

- `-f, --from-file <path>` (Required): JSON file with search results
- `-i, --ids <list>` (Optional): Paper IDs (comma-separated or range)
- `-d, --dir <path>` (Optional): Download directory (default: `{workspace}/hxxra/papers/`)

**Input Examples**:

```json
{"command": "download", "from-file": "results.json", "ids": ["1", "3", "5"], "dir": "./downloads"} | python scripts/hxxra.py
{"command": "download", "from-file": "results.json", "dir": "./downloads"} | python scripts/hxxra.py
```

**Output Structure**:

```json
{
  "ok": true,
  "command": "download",
  "downloaded": [
    {
      "id": "1",
      "title": "Paper Title",
      "status": "success",
      "pdf_path": "{workspace}/hxxra/papers/2023_Smith_NeRF_Explained/2023_Smith_NeRF_Explained.pdf",
      "size_bytes": 1234567,
      "url": "https://arxiv.org/pdf/xxxx.xxxxx.pdf"
    }
  ],
  "failed": [],
  "total": 3,
  "successful": 3,
  "download_dir": "{workspace}/hxxra/papers"
}
```

------

### 3. **hxxra analyze** - Analyze PDF content

**Dependencies**: `pip install pymupdf pdfplumber openai`

**Purpose**: Analyze paper content using LLM

**Parameters**:

- `-p, --pdf <path>` (Optional*): Single PDF file to analyze
- `-d, --directory <path>` (Optional*): Directory with multiple PDFs
- `-o, --output <path>` (Optional): Output directory. If not specified, analysis results will be saved in the same subfolder as the PDF (default: `{workspace}/hxxra/papers/{paper_title}/analysis.json`)

** Note: Either `--pdf` or `--directory` must be provided, but not both*

**Input Examples**:

```json
{"command": "analyze", "pdf": "paper.pdf", "output": "./analysis/"} | python scripts/hxxra.py
{"command": "analyze", "directory": "hxxra/papers/"} | python scripts/hxxra.py
```

**Output Structure**:

```json
{
  "ok": true,
  "command": "analyze",
  "analyzed": [
    {
      "id": "paper_1",
      "original_file": "paper.pdf",
      "analysis_file": "{workspace}/hxxra/papers/2023_Smith_NeRF_Explained/analysis.json",
      "metadata": {
        "title": "Paper Title",
        "authors": ["Author1", "Author2"],
        "year": "2023",
        "abstract": "Abstract text..."
      },
      "analysis": {
        "background": "Problem background...",
        "methodology": "Proposed method...",
        "results": "Experimental results...",
        "conclusions": "Conclusions..."
      },
      "status": "success"
    }
  ],
  "summary": {
    "total": 1,
    "successful": 1,
    "failed": 0
  }
}
```

------

### 4. **hxxra report** - Generate Markdown report

**Purpose**: Generate a comprehensive Markdown report from all `analysis.json` files in a directory

**Parameters**:

- `-d, --directory <path>` (Required): Directory containing paper folders with `analysis.json` files
- `-o, --output <path>` (Optional): Output Markdown file path (default: `{directory}/report.md`)
- `-t, --title <string>` (Optional): Report title (default: "Research Papers Report")
- `-s, --sort <string>` (Optional): Sort by: `year` (default, descending), `title`, or `author`

**Input Examples**:

```json
{"command": "report", "directory": "hxxra/papers/", "output": "hxxra/papers/report.md", "title": "My Research Papers", "sort": "year"} | python scripts/hxxra.py
{"command": "report", "directory": "hxxra/papers/"} | python scripts/hxxra.py
```

**Output Structure**:

```json
{
  "ok": true,
  "command": "report",
  "total_papers": 10,
  "output_file": "/path/to/hxxra/papers/report.md"
}
```

**Generated Markdown Format**:

The generated report includes:
- **Header**: Title, generation date, total papers, data source
- **Keywords Table**: Top 15 most frequent keywords across all papers
- **Overview Table**: Quick summary of all papers (title, author, year, keywords)
- **Detailed Content**: For each paper:
  - Title, authors, year, keywords, code link (if available)
  - Abstract
  - Research background
  - Methodology
  - Main results
  - Conclusions
  - Limitations
  - Impact
  - Source folder path

**Note**: The report command recursively scans all subdirectories for `analysis.json` files and only includes papers with `status: "success"`.

------

### 5. **hxxra save** - Save to Zotero

**Purpose**: Save papers to Zotero collection

**Parameters**:

- `-f, --from-file <path>` (Required): JSON file with search results (e.g., `hxxra/searches/search_results.json`)
- `-i, --ids <list>` (Optional): Paper IDs to save
- `-c, --collection <string>` (Required): Zotero collection name

**Input Examples**:

```json
{"command": "save", "from-file": "hxxra/searches/search_results.json", "ids": ["1", "2", "3"], "collection": "AI Research"} | python scripts/hxxra.py
{"command": "save", "from-file": "hxxra/searches/search_results.json", "collection": "My Collection"} | python scripts/hxxra.py
```

**Output Structure**:

```json
{
  "ok": true,
  "command": "save",
  "collection": "AI Research",
  "saved_items": [
    {
      "id": "1",
      "title": "Paper Title",
      "zotero_key": "ABCD1234",
      "url": "https://www.zotero.org/items/ABCD1234",
      "status": "success"
    }
  ],
  "failed_items": [],
  "total": 3,
  "successful": 3,
  "zotero_collection": "ABCD5678"
}
```

------

## Workflow Examples

### Complete Workflow

```bash
# 1. Search for papers
{"command": "search", "query": "graph neural networks", "source": "arxiv", "limit": 10, "output": "hxxra/searches/gnn_arxiv.json"} | python scripts/hxxra.py

# 2. Download papers
{"command": "download", "from-file": "hxxra/searches/gnn_arxiv.json", "dir": "hxxra/papers"} | python scripts/hxxra.py

# 3. Analyze downloaded papers
{"command": "analyze", "directory": "hxxra/papers/"} | python scripts/hxxra.py

# 4. Generate comprehensive report
{"command": "report", "directory": "hxxra/papers/", "output": "hxxra/papers/report.md", "sort": "year"} | python scripts/hxxra.py

# 5. Save to Zotero
{"command": "save", "from-file": "hxxra/searches/gnn_arxiv.json", "collection": "GNN Papers"} | python scripts/hxxra.py
```

### Single Command Examples

```bash
# Search with scholar
{"command": "search", "query": "reinforcement learning", "source": "scholar", "limit": 15} | python scripts/hxxra.py

# Download specific papers
{"command": "download", "from-file": "hxxra/searches/search_results.json", "ids": ["2", "4", "6"], "dir": "hxxra/papers"} | python scripts/hxxra.py

# Analyze single PDF in detail
{"command": "analyze", "pdf": "hxxra/papers/2024_Zhang_Transformer_Survey/2024_Zhang_Transformer_Survey.pdf"} | python scripts/hxxra.py

# Generate report sorted by title
{"command": "report", "directory": "hxxra/papers/", "sort": "title", "output": "hxxra/papers/report_by_title.md"} | python scripts/hxxra.py

# Save with custom notes
{"command": "save", "from-file": "hxxra/searches/search_results.json", "ids": ["1"], "collection": "To Read"} | python scripts/hxxra.py
```

## Configuration Requirements

### API Credentials(config.json)

1. **arXiv API**: No key required for basic access

2. **Google Scholar**: May require authentication for large queries

3. **Zotero API**: Required credentials:

   ```json
   {
     "api_key": "YOUR_ZOTERO_API_KEY", # Create at https://www.zotero.org/settings/keys/new
     "user_id": "YOUR_ZOTERO_USER_ID", # Found on the same page (numeric, not username)
     "library_type": "user"  # or "group"
   }
   ```

4. **LLM API**: OpenAI or compatible API key for analysis

## Notes

- All commands are executed via stdin/stdout JSON communication
- Error handling returns `{"ok": false, "error": "Error message"}`
- Large operations support progress reporting via intermediate messages
- Configuration is loaded from `config.json` or environment variables
- Concurrent operations have configurable limits to avoid rate limiting

## Error Handling

Each command returns standard error format:

```json
{
  "ok": false,
  "command": "<command>",
  "error": "Error description",
  "error_code": "ERROR_TYPE",
  "suggestion": "How to fix it"
}
```

## Development Status

### Current Version: v1.2.0 (2026/3/8)

### Version History

**v1.2.0 · 2026/3/8**

- Added `report` command to generate comprehensive Markdown reports from all `analysis.json` files
- Report includes keyword statistics, overview table, and detailed content for each paper
- Supports sorting by year (default), title, or author
- Generates clean, readable Markdown format with tables, headers, and structured content
- Updated documentation to include the new report command in workflows and examples

**v1.1.1 · 2026/3/7**

- Added `sanitize_filename()` function to unify filename and folder name handling for downloaded papers.
- Modified `handle_download` function to use the new sanitization function for author names and titles.
- Improved filename safety: now only allows letters, numbers, and underscores; multiple consecutive underscores are merged; length limited to 50 characters.

**v1.1.0 · 2026/3/7**

- Added a recommended directory structure for optimal organization of search results, papers, analysis, and logs.
- Updated all examples and default output locations to align with the new `{workspace}/hxxra/` folder layout.
- Clarified file storage practices: each downloaded paper now has its own subfolder containing the PDF and analysis files.
- Improved documentation for command parameters and outputs to reflect the directory structure changes.
- Enhanced clarity of workflow steps, making it easier to manage, locate, and share research outputs.
- Fixed ids data handling: improved ID matching logic to support both string and numeric ID comparisons in download and save commands.
- Fixed analyze output parameter: output directory is now only created when explicitly specified, otherwise analysis results are saved in the same subfolder as the PDF.
- Fixed Zotero API "400 Bad Request" error: changed data format from object to array (`[item_data]`) to comply with Zotero API requirements

**v1.0.2 · 2026/3/6**

- Modified hxxra.py script to add fix_proxy_env() function call, resolving the issue where ALL_PROXY and all_proxy are reset to socks://127.0.0.1:7897/ in new OpenClaw sessions, causing search failures

**v1.0.1 · 2026/3/6**

- Added academic note clarifying that arXiv search results are sorted by most recent submission date, while Google Scholar results use the source's default relevance ranking
- No changes to command structure, parameters, or output formats

**v1.0.0 · 2026/2/9**

Initial release of hxxra – a research assistant tool for searching, downloading, analyzing, and saving research papers.

- Introduces four core JSON-based commands: search, download, analyze, save
- Supports searching papers via Google Scholar and arXiv, with flexible parameters and output structure
- Enables PDF downloads using search results, with fine-grained ID selection and status reporting
- Integrates LLM-driven PDF content analysis, providing structured output for one or many papers
- Allows saving papers to Zotero collections, requiring user API credentials
- Features robust parameter validation, error handling, and documentation with usage examples