How To Backtest Your Trading Strategy With AI

§ 01 Executive Snapshot

• What: Six frontier AI models participated in a trading experiment on Hyperliquid, focusing on crypto perpetuals.
• Who: AI models including Qwen3 Max, DeepSeek, GPT-5, Gemini 2.5 Pro, and Claude 4.5 Sonnet, managed by Nof1.ai under Alpha Arena.
• Why it matters: The experiment highlights the limitations of AI in trading and raises questions about the efficacy of backtesting AI-driven trading strategies.

§ 02 Key Developments

• In late October 2025, six AI models were given $10,000 each to trade crypto perpetuals on Hyperliquid.
• The experiment ran until November 4, with Qwen3 Max and DeepSeek leading, while GPT-5, Gemini 2.5 Pro, and Claude 4.5 Sonnet struggled.
• The results underscored the headline that "LLMs can’t trade crypto," emphasizing the inherent challenges of AI in live trading scenarios.

§ 03 Strategic Context

• The experiment demonstrates the ongoing evolution of AI in trading, particularly in high-volatility environments like crypto markets.
• It fits into the broader narrative of AI's role in financial markets, especially regarding the limitations of traditional backtesting methods for non-deterministic AI models.

§ 04 Strategic Implications

• The immediate consequence is a reevaluation of how AI-driven strategies are developed and tested, moving towards forward testing instead of traditional backtesting.
• Long-term implications include the need for new methodologies and tools to assess AI performance in live trading conditions without relying on historical data alone.

§ 05 Risks & Constraints

• One potential risk is the overfitting of AI models to historical data, which can lead to poor performance in live environments.
• Another risk involves the inherent non-determinism of AI models, which complicates the ability to predict outcomes based on past performance.

§ 06 Watchlist / Forward Signals

• Future developments to watch include the evolution of AI trading platforms and the emergence of new methodologies for forward testing AI strategies.
• Monitoring the performance of AI models in real-time trading environments will signal their viability and effectiveness compared to traditional trading strategies.