๐Ÿ“–
Auditing LLM Alpha-Factor Mining: Metric Provenance and Full-Cohort Reanalysis
Yufeng He

Auditing LLM Alpha-Factor Mining: Metric Provenance and Full-Cohort Reanalysis

Author / Organization:
Science & Mathematics Published 2026 Verification Pending

Overview & Synopsis

Reliable evaluation of LLM-based alpha mining requires more than positive backtest summaries. We audit a preserved project at three levels: historical run counters, outcome retention, and a separately scoped public reanalysis. Three historical runs report 67, 20, and 11 keep decisions, but the comparison mixes metric populations: a mean over kept candidates is compared with a mean of per-round maxima, and successful evaluations with all returned outcomes. These artifacts cannot identify an architecture advantage. A released 1,837-expression catalog also excludes non-positive OOS outcomes, making its zero-reversal rate non-informative. We then account for all 1,047 unique expressions in a frozen, IS-screened preserved pool using forty A-shares and 2024-2025 rank associations. A versioned public-operator correction yields 923 scorable records, 99 with insufficient valid dates, and 25 unsupported, without modifying the other 990 prior diagnostics. Among scorable records, 671 have positive and 252 negative signed mean rank-IC. Simple untrained references also show positive associations, so this distribution does not establish incremental agent skill or profitability. The case study motivates explicit metric grains, immutable cohort accounting, and provenance-aware interpretation rather than architectural rankings from incomplete records.

Indexed from repository: arXiv (arxiv.org)
Loading document from verified source...
Keyboard Shortcuts: ← / → for pages • + / - for zoom • 0 to fit width Powered by Mozilla PDF.js v3.11

Document Information

Document Title Auditing LLM Alpha-Factor Mining: Metric Provenance and Full-Cohort Reanalysis
Author / Organization Yufeng He
Subject & Discipline cs.CE
Categories Science & Mathematics
Publication Year 2026
License / Rights arXiv Open Access / Creative Commons
Resource Verification Listed PDF (Verification Pending)
Original Host Website arXiv (arxiv.org)
File Format application/pdf (Portable Document Format)
Language EN
Catalog ID DOC-267187
Original Source Link Visit Source Page on arxiv.org »