title: Chapter 11: Backtesting In Depth¶
Chapter 11: Backtesting In Depth¶
This chapter takes a detailed look at the configuration, usage, and result analysis of the Lean backtesting system.
11.1 Backtest Configuration¶
11.1.1 The config.json Configuration File¶
Note that the engine's config.json does not contain start-date/end-date/initial-cash keys for backtest dates and capital. A backtest's start/end dates and initial capital are determined by SetStartDate()/SetEndDate()/SetCash() in the algorithm's Initialize() (see Section 11.1.3). Common backtest-related keys in config.json are:
{
// 运行环境:"backtesting"(回测)、"live-paper" 等
"environment": "backtesting",
// 数据文件夹
"data-folder": "./Data",
// 算法配置
"algorithm-type-name": "MyStrategy",
"algorithm-language": "Python",
"algorithm-location": "MyStrategy.py",
// 是否实盘模式
"live-mode": false,
// 历史数据提供商(值为类名,此处为内置默认值)
"data-provider": "QuantConnect.Lean.Engine.DataFeeds.DefaultDataProvider"
}
11.1.2 Configuring a Backtest with the Lean CLI¶
# 基本回测(项目名是位置参数,指向项目目录)
lean backtest "MyStrategy"
# 指定结果输出目录
lean backtest "MyStrategy" --output "./backtests/custom"
# 指定历史数据提供商(值为类名)
lean backtest "MyStrategy" \
--data-provider-historical "QuantConnect.Lean.Engine.DataFeeds.DefaultDataProvider"
Note:
lean backtesthas no--start/--end/--initial-cashoptions; the backtest period and initial capital are determined bySetStartDate()/SetEndDate()/SetCash()in the algorithm'sInitialize().
11.1.3 Programmatic Configuration¶
from AlgorithmImports import *
class ConfigurableStrategy(QCAlgorithm):
"""
可配置的策略
"""
def Initialize(self):
# 回测日期与初始资金在 Initialize() 中设置(也可从参数读取后转换)
self.SetStartDate(2020, 1, 1)
self.SetEndDate(2023, 12, 31)
self.SetCash(100000)
# 从参数读取可覆盖的值(GetParameter 返回 str,需自行转换)
# initial_cash = int(self.GetParameter("initial-cash") or 100000)
# self.SetCash(initial_cash)
self.spy = self.AddEquity("SPY", Resolution.Daily).Symbol
11.2 Backtest Execution Modes¶
11.2.1 Data Normalization Modes¶
def Initialize(self):
spy = self.AddEquity("SPY", Resolution.Daily)
# 数据归一化模式
# Adjusted: 使用复权价格(推荐)
spy.SetDataNormalizationMode(DataNormalizationMode.Adjusted)
# Raw: 使用原始价格(包含分红拆股)
# spy.SetDataNormalizationMode(DataNormalizationMode.Raw)
# SplitAdjusted: 仅调整拆股
# spy.SetDataNormalizationMode(DataNormalizationMode.SplitAdjusted)
# TotalReturn: 包含分红的总收益
# spy.SetDataNormalizationMode(DataNormalizationMode.TotalReturn)
11.2.2 Fill Mode¶
def Initialize(self):
# 默认填充数据(fillForward 默认为 True)
spy = self.AddEquity("SPY", Resolution.Daily, fillForward=True)
# 不填充
spy = self.AddEquity("SPY", Resolution.Daily, fillForward=False)
11.2.3 Extended Market Hours¶
def Initialize(self):
# 包含盘前盘后数据(extendedMarketHours 默认为 False)
spy = self.AddEquity("SPY", Resolution.Minute, extendedMarketHours=True)
11.3 Corporate Action Handling¶
11.3.1 Dividend Handling¶
def OnDividends(self, dividends):
"""
分红事件处理
Args:
dividends: Dividends(DataDictionary[Symbol, Dividend])
"""
for dividend in dividends.Values:
self.Debug(f"Dividend: {dividend.Symbol.Value}")
self.Debug(f" Distribution: ${dividend.Distribution}")
self.Debug(f" Reference Price: ${dividend.ReferencePrice}")
# 分红会自动计入账户现金
# 如果持有股票,现金分红会自动添加
11.3.2 Split Handling¶
def OnSplits(self, splits):
"""
拆股事件处理
Args:
splits: Splits(DataDictionary[Symbol, Split])
"""
for split in splits.Values:
self.Debug(f"Split: {split.Symbol.Value}")
self.Debug(f" Type: {split.Type}")
self.Debug(f" Split Factor: {split.SplitFactor}")
self.Debug(f" Reference Price: ${split.ReferencePrice}")
# 拆股会自动处理
# 持仓数量和价格会自动调整
11.3.3 Delisting Handling¶
def OnDelistings(self, delistings):
"""
退市事件处理
Args:
delistings: Delistings(DataDictionary[Symbol, Delisting])
"""
for delisting in delistings.Values:
self.Debug(f"Delisting: {delisting.Symbol.Value}")
self.Debug(f" Type: {delisting.Type}")
# 平仓(Liquidate 没有 reason 关键字参数)
if self.Portfolio[delisting.Symbol].Invested:
self.Liquidate(delisting.Symbol)
11.4 Backtest Result Analysis¶
11.4.1 Basic Statistics¶
def Initialize(self):
# Portfolio 没有 StartingCash 属性,初始资金在 Initialize 中保存
self.starting_cash = self.Portfolio.TotalPortfolioValue
def OnEndOfAlgorithm(self):
"""算法结束时分析结果"""
# 基本统计
total_return = (self.Portfolio.TotalPortfolioValue /
self.starting_cash - 1)
# 从 Statistics 获取更多信息
# self.Statistics 是 StatisticsResults,只有 TotalPerformance/RollingPerformances/Summary 三个成员
# 数值指标位于 TotalPerformance.PortfolioStatistics,Summary 是字符串字典
portfolio_stats = self.Statistics.TotalPerformance.PortfolioStatistics
summary = self.Statistics.Summary
self.Debug("=" * 60)
self.Debug("BACKTEST RESULTS")
self.Debug("=" * 60)
self.Debug(f"Starting Capital: ${self.starting_cash:,.2f}")
self.Debug(f"Final Portfolio Value: ${self.Portfolio.TotalPortfolioValue:,.2f}")
self.Debug(f"Total Return: {total_return:.2%}")
self.Debug(f"Annual Return: {portfolio_stats.CompoundingAnnualReturn:.2%}")
self.Debug(f"Sharpe Ratio: {portfolio_stats.SharpeRatio:.2f}")
self.Debug(f"Sortino Ratio: {portfolio_stats.SortinoRatio:.2f}")
self.Debug(f"Max Drawdown: {portfolio_stats.Drawdown:.2%}")
self.Debug(f"Win Rate: {portfolio_stats.WinRate:.2%}")
self.Debug(f"Total Orders: {summary['Total Orders']}")
self.Debug("=" * 60)
11.4.2 Detailed Performance Metrics¶
| Metric | Description | Calculation |
|---|---|---|
| Total Return | Total rate of return | (final value - initial value) / initial value |
| Compounding Annual Return | Compound annual growth rate | (final value / initial value)^(1/years) - 1 |
| Sharpe Ratio | Sharpe ratio | annualized return / annualized volatility |
| Sortino Ratio | Sortino ratio | annualized return / downside volatility |
| Drawdown | Drawdown | decline from peak to trough |
| Win Rate | Win rate | winning trades / total trades |
| Profit-Loss Ratio | Profit-loss ratio | average profit per trade / average loss per trade |
| Annual Variance | Annualized variance | daily return variance × 252 |
11.4.3 Generating an HTML Report¶
# 生成回测报告(--backtest-results 指向回测输出目录中的结果 JSON,
# --report-destination 指定报告输出路径)
lean report \
--backtest-results "MyStrategy/backtests/2023-01-01_12-00-00/MyStrategy.json" \
--report-destination "./reports/report.html"
11.4.4 Custom Analysis¶
import pandas as pd
def OnEndOfAlgorithm(self):
"""自定义分析
注意:QCAlgorithm 没有内置的 CalculateMonthlyReturns/CalculateHoldingPeriods
方法,需要基于自己记录的数据(或 self.History 拉取的行情)自行计算。
"""
# self.daily_values 为 OnData 中逐日记录的 {日期: 组合总值}
equity = pd.Series(self.daily_values).sort_index()
# 计算月度收益:取每月最后一个净值,再计算环比变化
monthly_last = equity.groupby(equity.index.to_period("M")).last()
monthly_returns = monthly_last.pct_change().dropna()
self.Debug("\n=== Monthly Returns ===")
for month, return_pct in monthly_returns.items():
self.Debug(f"{month}: {return_pct:.2%}")
11.5 Backtest Debugging Tips¶
11.5.1 Verbose Logging¶
def Initialize(self):
# 设置详细日志级别
self.Log("Initializing strategy...")
self.Debug(f"Start Date: {self.StartDate}")
self.Debug(f"End Date: {self.EndDate}")
self.Debug(f"Initial Cash: ${self.Portfolio.Cash}")
def OnData(self, data):
# 记录关键决策点
if self.ShouldBuy(data):
self.Log(f"Buy signal at {self.Time}")
self.Debug(f"Price: {self.Securities[self.spy].Price}")
self.Debug(f"RSI: {self.rsi.Current.Value}")
11.5.2 Plotting with Plot¶
def Initialize(self):
# 自定义图表(PlotIndicator 只用于绘制指标对象;
# 绘制数值请使用 Plot(chart, seriesName, value))
self.AddChart(Chart("Strategy"))
def OnData(self, data):
# 绘制信号数值
if self.ShouldBuy(data):
self.Plot("Strategy", "Signal", 1)
elif self.ShouldSell(data):
self.Plot("Strategy", "Signal", -1)
11.5.3 Saving Intermediate Results¶
import json
def OnData(self, data):
# 保存每日状态(状态持久化使用 ObjectStore,
# 没有 SaveState/GetState 方法)
if self.Time.day == 1:
state = {
'portfolio_value': str(self.Portfolio.TotalPortfolioValue),
'cash': str(self.Portfolio.Cash),
'invested': str(self.Portfolio.Invested)
}
self.ObjectStore.Save("mystrategy/state", json.dumps(state))
def OnEndOfAlgorithm(self):
# 读取保存的状态
if self.ObjectStore.ContainsKey("mystrategy/state"):
state = json.loads(self.ObjectStore.Read("mystrategy/state"))
self.Debug(f"Final state: {state}")
11.6 Backtesting Best Practices¶
11.6.1 In-Sample / Out-of-Sample Testing¶
class OutOfSampleTestStrategy(QCAlgorithm):
"""
样本内外测试策略
"""
def Initialize(self):
# 样本内训练期:2020-2021
self.SetStartDate(2020, 1, 1)
self.SetEndDate(2021, 12, 31)
self.in_sample = True
def OnData(self, data):
# 样本期内:训练和优化参数
if self.in_sample and self.Time.date() == datetime(2021, 12, 31).date():
self.TrainModel()
self.Debug("Training complete. Switching to out-of-sample testing.")
self.in_sample = False
# 样本外:使用固定参数测试
if not self.in_sample:
self.TradeWithFixedParams()
11.6.2 Rolling Window Backtest¶
class RollingWindowBacktest(QCAlgorithm):
"""
滚动窗口回测
每6个月重新训练模型
"""
def Initialize(self):
self.SetStartDate(2020, 1, 1)
self.SetEndDate(2023, 12, 31)
# 每6个月重新训练
self.Schedule.On(
self.DateRules.Every(DayOfWeek.Sunday),
self.TimeRules.Every(TimeSpan.FromDays(30)),
lambda: self.CheckRetrain()
)
self.last_retrain = self.Time
def CheckRetrain(self):
"""检查是否需要重新训练"""
if (self.Time - self.last_retrain).days >= 180:
self.RetrainModel()
self.last_retrain = self.Time
11.6.3 Avoiding Common Pitfalls¶
class RobustStrategy(QCAlgorithm):
"""健壮的策略示例"""
def Initialize(self):
self.SetStartDate(2020, 1, 1)
self.SetEndDate(2023, 12, 31)
self.SetCash(100000)
self.spy = self.AddEquity("SPY", Resolution.Daily).Symbol
# 1. 设置预热避免初始数据不足
self.SetWarmUp(TimeSpan.FromDays(60))
# 2. 使用调整后价格避免分红影响
self.Securities[self.spy].SetDataNormalizationMode(
DataNormalizationMode.Adjusted
)
def OnData(self, data):
# 3. 检查数据可用性
if not data.ContainsKey(self.spy):
return
# 4. 确认指标准备就绪
if not self.indicator.IsReady:
return
# 5. 检查证券可交易
if not self.Securities[self.spy].IsTradable:
return
# 6. 确认账户有足够资金
if self.Portfolio.Cash < self.Securities[self.spy].Price:
return
# 执行交易逻辑
self.ExecuteStrategy()
11.7 Complete Example: A Backtest with Detailed Analysis¶
from AlgorithmImports import *
from datetime import datetime, timedelta
class DetailedBacktestAnalysis(QCAlgorithm):
"""
带详细分析的回测策略
"""
def Initialize(self):
self.SetStartDate(2020, 1, 1)
self.SetEndDate(2023, 12, 31)
self.SetCash(100000)
# Portfolio 没有 StartingCash 属性,在此保存初始资金
self.starting_cash = self.Portfolio.TotalPortfolioValue
self.spy = self.AddEquity("SPY", Resolution.Daily).Symbol
# 设置数据归一化
self.Securities[self.spy].SetDataNormalizationMode(
DataNormalizationMode.Adjusted
)
# 指标
self.sma_fast = self.SMA(self.spy, 20)
self.sma_slow = self.SMA(self.spy, 50)
self.SetWarmUp(TimeSpan.FromDays(60))
# 记录交易
self.trades = []
self.daily_values = {}
def OnData(self, data):
# 记录每日组合价值
self.daily_values[self.Time.date()] = self.Portfolio.TotalPortfolioValue
# 交易逻辑
if self.sma_fast.Current.Value > self.sma_slow.Current.Value:
if not self.Portfolio[self.spy].IsLong:
self.SetHoldings(self.spy, 1)
self.RecordTrade("BUY", self.Securities[self.spy].Price)
else:
if self.Portfolio[self.spy].IsLong:
self.Liquidate(self.spy)
self.RecordTrade("SELL", self.Securities[self.spy].Price)
def RecordTrade(self, direction, price):
"""记录交易"""
self.trades.append({
'time': self.Time,
'direction': direction,
'price': price,
'portfolio_value': self.Portfolio.TotalPortfolioValue
})
def OnEndOfAlgorithm(self):
"""输出详细分析"""
self.PrintBasicStats()
self.PrintTradeAnalysis()
self.PrintMonthlyReturns()
self.PrintDrawdownAnalysis()
def PrintBasicStats(self):
"""打印基本统计"""
total_return = (self.Portfolio.TotalPortfolioValue /
self.starting_cash - 1)
self.Debug("\n" + "="*60)
self.Debug("BASIC STATISTICS")
self.Debug("="*60)
self.Debug(f"Starting Capital: ${self.starting_cash:,.2f}")
self.Debug(f"Final Capital: ${self.Portfolio.TotalPortfolioValue:,.2f}")
self.Debug(f"Total Return: {total_return:.2%}")
self.Debug(f"Annual Return: "
f"{self.Statistics.TotalPerformance.PortfolioStatistics.CompoundingAnnualReturn:.2%}")
self.Debug(f"Sharpe Ratio: "
f"{self.Statistics.TotalPerformance.PortfolioStatistics.SharpeRatio:.2f}")
self.Debug(f"Max Drawdown: "
f"{self.Statistics.TotalPerformance.PortfolioStatistics.Drawdown:.2%}")
self.Debug(f"Total Trades: {len(self.trades)}")
def PrintTradeAnalysis(self):
"""打印交易分析"""
if not self.trades:
return
buy_trades = [t for t in self.trades if t['direction'] == 'BUY']
sell_trades = [t for t in self.trades if t['direction'] == 'SELL']
self.Debug("\n" + "="*60)
self.Debug("TRADE ANALYSIS")
self.Debug("="*60)
self.Debug(f"Buy Trades: {len(buy_trades)}")
self.Debug(f"Sell Trades: {len(sell_trades)}")
# 计算每笔交易收益
returns = []
for i in range(0, len(self.trades)-1, 2):
if i+1 < len(self.trades):
entry = self.trades[i]
exit = self.trades[i+1]
trade_return = (exit['price'] / entry['price'] - 1) * 100
returns.append(trade_return)
if returns:
self.Debug(f"Average Trade Return: {sum(returns)/len(returns):.2f}%")
self.Debug(f"Winning Trades: {sum(1 for r in returns if r > 0)}")
self.Debug(f"Losing Trades: {sum(1 for r in returns if r < 0)}")
def PrintMonthlyReturns(self):
"""打印月度收益"""
monthly = {}
for date, value in self.daily_values.items():
month_key = date.strftime("%Y-%m")
if month_key not in monthly:
monthly[month_key] = []
monthly[month_key].append(value)
self.Debug("\n" + "="*60)
self.Debug("MONTHLY RETURNS")
self.Debug("="*60)
for month in sorted(monthly.keys()):
values = monthly[month]
month_return = (values[-1] / values[0] - 1) * 100
self.Debug(f"{month}: {month_return:.2f}%")
def PrintDrawdownAnalysis(self):
"""打印回撤分析"""
self.Debug("\n" + "="*60)
self.Debug("DRAWDOWN ANALYSIS")
self.Debug("="*60)
values = list(self.daily_values.values())
peaks = []
drawdowns = []
running_max = values[0]
for value in values:
if value > running_max:
running_max = value
peaks.append(value)
drawdown = (value - running_max) / running_max * 100
drawdowns.append(drawdown)
self.Debug(f"Max Drawdown: {min(drawdowns):.2f}%")
self.Debug(f"Average Drawdown: {sum(drawdowns)/len(drawdowns):.2f}%")
# 计算回撤持续时间
in_drawdown = False
drawdown_periods = []
current_period = 0
for dd in drawdowns:
if dd < 0:
in_drawdown = True
current_period += 1
elif in_drawdown:
in_drawdown = False
drawdown_periods.append(current_period)
current_period = 0
if drawdown_periods:
self.Debug(f"Avg Drawdown Duration: {sum(drawdown_periods)/len(drawdown_periods):.0f} days")
self.Debug(f"Max Drawdown Duration: {max(drawdown_periods)} days")