coco
26 KiB
title, source, author, published, created, description, tags
| title | source | author | published | created | description | tags | ||
|---|---|---|---|---|---|---|---|---|
| Building a Prediction Market Trading Bot with Python: From Idea to Deployment | https://x.com/sopersone/status/2045488274915901765 |
|
2026-04-18 | 2026-04-26 | Part 6 of 12 Event-Driven Backtesting and Parameter OptimizationIn the previous article, we built a vectorized backtest and compared three ... |
|
Part 6 of 12 Event-Driven Backtesting and Parameter Optimization
In the previous article, we built a vectorized backtest and compared three strategies on real BTC data. Fast, clean, and useful. But this approach has limitations: it does not handle commissions properly, does not model slippage, and does not support complex position management rules.
In this article, we move to the next level: event-driven backtesting. We will process data bar by bar - the way it actually happens in live trading. We will add commissions and slippage, walk through parameter optimization with grid search, learn to fight overfitting with walk-forward analysis, and get acquainted with the Kelly Criterion for position sizing.
Event-Driven vs Vectorized Backtesting
In a vectorized backtest, we apply operations to the entire data array at once. It is fast, but it means there is no concept of "state" - we simply multiply two arrays and get returns.
In an event-driven backtest, we step through the data one bar at a time, as if we are inside that moment in time. At each bar:
1. We observe new data 2. We check strategy conditions 3. We make a decision: buy, sell, or do nothing 4. We update portfolio state: cash balance, position, trade history
It is slower, but it lets us model reality accurately: commissions are deducted on every trade, slippage eats into profits, and position sizing shapes the equity curve.
Building the Event-Driven Backtest
Let us start with a simple but complete implementation. We need three components:
- **Portfolio** -- an object that tracks state: cash on hand, current position, trade history
- **execute_trade()** -- a function that simulates order execution accounting for commission and slippage
- **run_backtest()** -- the main loop that steps through bars and calls the strategy
The Portfolio Class
python
class Portfolio:
"""
Tracks the state of a trading account.
"""
def __init__(self, initial_cash=10_000.0, commission=0.001, slippage=0.0005):
"""
Parameters:
initial_cash -- starting capital in USDT
commission -- commission per trade (0.001 = 0.1%)
slippage -- slippage (0.0005 = 0.05%)
"""
self.initial_cash = initial_cash
self.cash = initial_cash
self.position = 0.0 # BTC held in position
self.commission = commission
self.slippage = slippage
self.trades = [] # trade history
self.equity_curve = [] # equity curve
def buy(self, price, date):
"""Open a long position using all available cash."""
if self.position > 0:
return # already in position
# Slippage: we buy slightly above the price
exec_price = price * (1 + self.slippage)
# Deduct commission
commission_cost = self.cash * self.commission
btc_amount = (self.cash - commission_cost) / exec_price
self.position = btc_amount
self.cash = 0.0
self.trades.append({
"date": date,
"action": "BUY",
"price": exec_price,
"amount": btc_amount,
"cost": commission_cost,
})
def sell(self, price, date):
"""Close the position."""
if self.position == 0:
return # no position to close
# Slippage: we sell slightly below the price
exec_price = price * (1 - self.slippage)
proceeds = self.position * exec_price
commission_cost = proceeds * self.commission
self.cash = proceeds - commission_cost
self.position = 0.0
self.trades.append({
"date": date,
"action": "SELL",
"price": exec_price,
"amount": 0,
"cost": commission_cost,
})
def total_equity(self, current_price):
"""Current portfolio value: cash + position at current price."""
return self.cash + self.position * current_price
def record(self, date, current_price):
"""Record the current state to the equity curve."""
self.equity_curve.append({
"date": date,
"equity": self.total_equity(current_price),
})
The Main Backtest Loop
python
import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
from data_loader import fetch_btc_data, add_returns
from strategies import sma_crossover_strategy, momentum_strategy, mean_reversion_strategy
def run_backtest(df_strategy, initial_cash=10_000.0, commission=0.001, slippage=0.0005):
"""
Event-driven backtest of a strategy.
Parameters:
df_strategy -- DataFrame with a 'position' column (already computed by strategy)
initial_cash -- starting capital
commission -- commission per trade
slippage -- slippage
Returns:
portfolio -- Portfolio object with results
equity_df -- DataFrame with the equity curve
"""
portfolio = Portfolio(initial_cash, commission, slippage)
data = df_strategy.dropna(subset=["position"]).copy()
for date, row in data.iterrows():
price = row["close"]
position = row["position"]
if position == 1:
portfolio.buy(price, date)
elif position == 0:
portfolio.sell(price, date)
portfolio.record(date, price)
equity_df = pd.DataFrame(portfolio.equity_curve).set_index("date")
return portfolio, equity_df
Let us run it:
python
df = fetch_btc_data(interval="1d", lookback="2 years ago UTC")
df = add_returns(df)
df_sma = sma_crossover_strategy(df, fast=20, slow=50)
portfolio, equity_df = run_backtest(df_sma, initial_cash=10_000)
print(f"Starting capital: ${portfolio.initial_cash:,.0f}")
print(f"Final capital: ${equity_df['equity'].iloc[-1]:,.0f}")
print(f"Number of trades: {len(portfolio.trades)}")
print(f"Total commissions: ${sum(t['cost'] for t in portfolio.trades):,.2f}")
Commissions and Slippage: How Much Do They Matter
Let us put some numbers to this and see how costs change the outcome. We run the same backtest with different settings:
python
configs = [
{"commission": 0.0, "slippage": 0.0, "label": "No costs"},
{"commission": 0.001, "slippage": 0.0, "label": "Commission only (0.1%)"},
{"commission": 0.001, "slippage": 0.0005, "label": "Commission + slippage"},
{"commission": 0.002, "slippage": 0.001, "label": "High costs"},
]
results = {}
for cfg in configs:
_, eq = run_backtest(df_sma,
commission=cfg["commission"],
slippage=cfg["slippage"])
results[cfg["label"]] = eq
fig, ax = plt.subplots(figsize=(12, 5))
for label, eq in results.items():
ax.plot(eq.index, eq["equity"], label=label, linewidth=1.2)
ax.set_title("Impact of Commissions and Slippage on Returns (SMA 20/50)")
ax.set_ylabel("Capital (USDT)")
ax.legend()
ax.grid(True, alpha=0.3)
plt.tight_layout()
plt.show()
For SMA Crossover with 8-12 trades per year the difference is small - a 0.1% commission per trade does not add up to much. But for strategies with dozens of trades per month, it is significant. Always account for transaction costs. They can turn a profitable strategy into a losing one.
Parameter Optimization: Grid Search
Every strategy has parameters. For SMA Crossover those are the fast and slow periods. The question is: which values produce the best results?
One way to answer that is to try every combination and pick the best. This is called a grid search.
python
def grid_search_sma(df, fast_range, slow_range, commission=0.001):
"""
Iterate over all SMA Crossover parameter combinations.
Returns a DataFrame with results for each pair.
"""
results = []
for fast in fast_range:
for slow in slow_range:
if fast >= slow:
continue # fast SMA must be shorter than slow SMA
df_strat = sma_crossover_strategy(df, fast=fast, slow=slow)
portfolio, equity_df = run_backtest(df_strat, commission=commission)
final_equity = equity_df["equity"].iloc[-1]
total_return = (final_equity / portfolio.initial_cash) - 1
# Sharpe ratio
daily_returns = equity_df["equity"].pct_change().dropna()
sharpe = (daily_returns.mean() / daily_returns.std()) * np.sqrt(252) \
if daily_returns.std() > 0 else 0
# Maximum drawdown
rolling_max = equity_df["equity"].cummax()
max_drawdown = ((equity_df["equity"] - rolling_max) / rolling_max).min()
results.append({
"fast": fast,
"slow": slow,
"total_return": total_return,
"sharpe": sharpe,
"max_drawdown": max_drawdown,
"trades": len(portfolio.trades),
})
return pd.DataFrame(results)
# Run the search
fast_range = range(5, 51, 5) # 5, 10, 15, ... 50
slow_range = range(10, 201, 10) # 10, 20, 30, ... 200
grid_results = grid_search_sma(df, fast_range, slow_range)
# Top 10 by Sharpe ratio
print(grid_results.sort_values("sharpe", ascending=False).head(10).to_string())
Heatmap of Result
Let us visualize how the Sharpe ratio changes across the parameter space:
python
import seaborn as sns
pivot = grid_results.pivot(index="slow", columns="fast", values="sharpe")
fig, ax = plt.subplots(figsize=(12, 8))
sns.heatmap(pivot, annot=False, fmt=".2f", cmap="RdYlGn",
center=0, ax=ax, cbar_kws={"label": "Sharpe ratio"})
ax.set_title("Sharpe Ratio: SMA Crossover (fast vs slow)")
ax.set_xlabel("Fast SMA (days)")
ax.set_ylabel("Slow SMA (days)")
plt.tight_layout()
plt.show()
The heatmap shows that there is no single "best" parameter pair - there are entire regions where the strategy performs well. That is a good sign. If good results were concentrated in just one narrow spot, that would be a clear signal of overfitting.
The Overfitting Problem and Walk-Forward Analysis
Here is the trap with grid search: if you tune parameters on the same data you use to evaluate the result, you end up with a strategy perfectly fitted to the past. On a live market, it will fail.
This is called **overfitting**.
The classic solution is to split the data into three parts:
- **Train set** - optimize parameters here (grid search) - **Validation set** - check whether you have overfit - **Test set** - the final evaluation on data you never touched during development
With financial data there is a key constraint: it is sequential. You cannot shuffle future data in with the past. The split must be strictly chronological.
Walk-Forward Analysis
Walk-forward is a more realistic version of the same principle. We slide a window across the data:
1. Take a training window (for example, 1 year) 2. Optimize parameters on it 3. Test on the next window (for example, 3 months) - this is out-of-sample 4. Shift both windows forward by 3 months 5. Repeat
The result is a series of out-of-sample tests that together give an honest picture of how the strategy would have performed in real time.
python
def walk_forward_analysis(df, fast_range, slow_range,
train_months=12, test_months=3,
commission=0.001):
"""
Walk-forward analysis for SMA Crossover.
At each step:
1. Optimize parameters on the train window
2. Record the out-of-sample result on the test window
"""
results_oos = []
start = df.index[0]
end = df.index[-1]
window_start = start
while True:
train_end = window_start + pd.DateOffset(months=train_months)
test_end = train_end + pd.DateOffset(months=test_months)
if test_end > end:
break
df_train = df[window_start:train_end]
df_test = df[train_end:test_end]
if len(df_train) < 60 or len(df_test) < 10:
window_start += pd.DateOffset(months=test_months)
continue
# --- Optimize on train ---
best_sharpe = -np.inf
best_fast, best_slow = 20, 50
for fast in fast_range:
for slow in slow_range:
if fast >= slow:
continue
df_s = sma_crossover_strategy(df_train, fast=fast, slow=slow)
_, eq = run_backtest(df_s, commission=commission)
if len(eq) < 5:
continue
dr = eq["equity"].pct_change().dropna()
sharpe = (dr.mean() / dr.std()) * np.sqrt(252) \
if dr.std() > 0 else 0
if sharpe > best_sharpe:
best_sharpe = sharpe
best_fast, best_slow = fast, slow
# --- Out-of-sample test ---
df_s_test = sma_crossover_strategy(df_test, fast=best_fast, slow=best_slow)
_, eq_test = run_backtest(df_s_test, commission=commission)
if len(eq_test) > 0:
oos_return = (eq_test["equity"].iloc[-1] / eq_test["equity"].iloc[0]) - 1
results_oos.append({
"period_start": train_end.date(),
"period_end": test_end.date(),
"best_fast": best_fast,
"best_slow": best_slow,
"oos_return": oos_return,
})
window_start += pd.DateOffset(months=test_months)
return pd.DataFrame(results_oos)
# Run walk-forward
fast_range_wf = range(10, 41, 10)
slow_range_wf = range(20, 101, 20)
wf_results = walk_forward_analysis(df, fast_range_wf, slow_range_wf,
train_months=12, test_months=3)
print(wf_results.to_string())
print(f"\nAverage OOS return per quarter: {wf_results['oos_return'].mean():.2%}")
print(f"Percentage of profitable quarters: {(wf_results['oos_return'] > 0).mean():.1%}")
Walk-forward takes longer - we run the grid search multiple times. But the results are honest: every out-of-sample period was tested on data that was genuinely "the future" at the time of optimization.
The Kelly Criterion: Position Sizing
Until now our strategy invested the entire available capital in every trade. This is called full Kelly - the most aggressive approach possible. In practice, nobody trades this way.
The **Kelly Criterion** answers a specific question: what fraction of capital should you risk on each trade to maximize long-term capital growth?
The Kelly formula for binary outcomes:
f = (p * b - (1 - p)) / b
where:
- `f` - fraction of capital to invest - `p` - win probability (win rate) - `b` - ratio of average win to average loss (win/loss ratio)
python
def kelly_fraction(trades_df):
"""
Computes the Kelly fraction from trade history.
Parameters:
trades_df -- DataFrame with trades (needs a 'pnl' column)
Returns:
kelly_f -- recommended fraction of capital (between 0 and 1)
"""
wins = trades_df[trades_df["pnl"] > 0]["pnl"]
losses = trades_df[trades_df["pnl"] < 0]["pnl"].abs()
if len(wins) == 0 or len(losses) == 0:
return 0.0
p = len(wins) / len(trades_df) # win rate
avg_win = wins.mean()
avg_loss = losses.mean()
b = avg_win / avg_loss # win/loss ratio
kelly_f = (p * b - (1 - p)) / b
return max(0.0, min(kelly_f, 1.0)) # clip to [0, 1]
In practice, traders use **half Kelly**. This delivers roughly 75% of the maximum capital growth but with half the volatility and drawdowns. A sensible trade-off.
python
def run_backtest_kelly(df_strategy, initial_cash=10_000.0,
commission=0.001, kelly_multiplier=0.5):
"""
Event-driven backtest with Kelly-based position sizing.
"""
portfolio = Portfolio(initial_cash, commission)
data = df_strategy.dropna(subset=["position"]).copy()
# First run a standard backtest to estimate Kelly parameters
_, eq_base = run_backtest(df_strategy, initial_cash, commission)
strategy_returns = eq_base["equity"].pct_change().dropna()
in_position = df_strategy["position"].reindex(eq_base.index).fillna(0)
trade_returns = strategy_returns[in_position == 1]
if len(trade_returns) > 10:
wins = trade_returns[trade_returns > 0]
losses = trade_returns[trade_returns < 0].abs()
p = len(wins) / len(trade_returns)
b = wins.mean() / losses.mean() if len(losses) > 0 else 1.0
kelly_f = max(0.0, min((p * b - (1 - p)) / b, 1.0))
else:
kelly_f = 0.5 # default fallback
fraction = kelly_f * kelly_multiplier
print(f"Kelly fraction: {kelly_f:.2%}, using: {fraction:.2%} of capital")
# Run the backtest with this fraction
cash = initial_cash
position = 0.0
equity_curve = []
for date, row in data.iterrows():
price = row["close"]
signal = row["position"]
if signal == 1 and position == 0:
invest = cash * fraction
cost = invest * commission
position = (invest - cost) / price
cash -= invest
elif signal == 0 and position > 0:
proceeds = position * price
cost = proceeds * commission
cash += proceeds - cost
position = 0.0
equity_curve.append({
"date": date,
"equity": cash + position * price,
})
equity_df = pd.DataFrame(equity_curve).set_index("date")
return equity_df, kelly_f
Comparison: Full Position vs Half Kelly vs Fixed Fraction
python
fig, ax = plt.subplots(figsize=(12, 5))
# Full position (as before)
_, eq_full = run_backtest(df_sma)
ax.plot(eq_full.index, eq_full["equity"],
label="Full position (100%)", linewidth=1.2, color="steelblue")
# Half Kelly
eq_hk, kf = run_backtest_kelly(df_sma, kelly_multiplier=0.5)
ax.plot(eq_hk.index, eq_hk["equity"],
label=f"Half Kelly ({kf*0.5:.0%} of capital)", linewidth=1.2, color="darkorange")
# Fixed 25%
eq_25, _ = run_backtest_kelly(df_sma, kelly_multiplier=0.25)
ax.plot(eq_25.index, eq_25["equity"],
label="Fixed 25%", linewidth=1.2, color="forestgreen")
ax.set_title("Position Sizing: Comparing Approaches")
ax.set_ylabel("Capital (USDT)")
ax.legend()
ax.grid(True, alpha=0.3)
plt.tight_layout()
plt.show()
The comparison makes it clear: the aggressive approach produces higher absolute returns, but with significantly larger drawdowns. Moderate position sizing reduces volatility - and makes the strategy much easier to stick with psychologically.
Final Script: Putting It All Together
Everything goes into `backtest_event.py`:
python
"""
backtest_event.py -- event-driven backtest with commissions, grid search, and Kelly
"""
import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
from data_loader import fetch_btc_data, add_returns
from strategies import sma_crossover_strategy, momentum_strategy, mean_reversion_strategy
class Portfolio:
def __init__(self, initial_cash=10_000.0, commission=0.001, slippage=0.0005):
self.initial_cash = initial_cash
self.cash = initial_cash
self.position = 0.0
self.commission = commission
self.slippage = slippage
self.trades = []
self.equity_curve = []
def buy(self, price, date):
if self.position > 0:
return
exec_price = price * (1 + self.slippage)
commission_cost = self.cash * self.commission
self.position = (self.cash - commission_cost) / exec_price
self.cash = 0.0
self.trades.append({"date": date, "action": "BUY",
"price": exec_price, "cost": commission_cost})
def sell(self, price, date):
if self.position == 0:
return
exec_price = price * (1 - self.slippage)
proceeds = self.position * exec_price
commission_cost = proceeds * self.commission
self.cash = proceeds - commission_cost
self.position = 0.0
self.trades.append({"date": date, "action": "SELL",
"price": exec_price, "cost": commission_cost})
def total_equity(self, price):
return self.cash + self.position * price
def record(self, date, price):
self.equity_curve.append({"date": date, "equity": self.total_equity(price)})
def run_backtest(df_strategy, initial_cash=10_000.0, commission=0.001, slippage=0.0005):
portfolio = Portfolio(initial_cash, commission, slippage)
data = df_strategy.dropna(subset=["position"]).copy()
for date, row in data.iterrows():
if row["position"] == 1:
portfolio.buy(row["close"], date)
elif row["position"] == 0:
portfolio.sell(row["close"], date)
portfolio.record(date, row["close"])
equity_df = pd.DataFrame(portfolio.equity_curve).set_index("date")
return portfolio, equity_df
def grid_search_sma(df, fast_range, slow_range, commission=0.001):
results = []
for fast in fast_range:
for slow in slow_range:
if fast >= slow:
continue
df_s = sma_crossover_strategy(df, fast=fast, slow=slow)
_, eq = run_backtest(df_s, commission=commission)
if len(eq) < 5:
continue
dr = eq["equity"].pct_change().dropna()
sharpe = (dr.mean() / dr.std()) * np.sqrt(252) if dr.std() > 0 else 0
rm = eq["equity"].cummax()
mdd = ((eq["equity"] - rm) / rm).min()
ret = (eq["equity"].iloc[-1] / 10_000) - 1
results.append({"fast": fast, "slow": slow,
"sharpe": sharpe, "total_return": ret,
"max_drawdown": mdd})
return pd.DataFrame(results)
if __name__ == "__main__":
df = fetch_btc_data(interval="1d", lookback="2 years ago UTC")
df = add_returns(df)
# Event-driven backtest for SMA Crossover
df_sma = sma_crossover_strategy(df, fast=20, slow=50)
portfolio, equity_df = run_backtest(df_sma)
final = equity_df["equity"].iloc[-1]
ret = (final / 10_000) - 1
print(f"Final capital: ${final:,.0f}")
print(f"Return: {ret:.1%}")
print(f"Trades: {len(portfolio.trades)}")
# Grid search
print("\nRunning grid search...")
grid = grid_search_sma(df,
fast_range=range(5, 51, 5),
slow_range=range(10, 201, 10))
print("\nTop 5 by Sharpe ratio:")
print(grid.sort_values("sharpe", ascending=False).head(5).to_string())
Updated project structure:
algobot/
data/
btc_daily.csv
btc_daily.h5
data_loader.py
strategies.py
backtest_vectorized.py
backtest_event.py <- new file
Practice Exercise
Try these on your own:
1. Run the event-driven backtest for the momentum strategy (window=20). How does the result compare to the vectorized backtest? How large a role did commissions play?
2. Run a grid search for the momentum strategy: sweep the window from 5 to 60 days in steps of 5. Which window gives the best Sharpe ratio?
3. Implement a simple train/test split: first 70% of the data is train, last 30% is test. Optimize the SMA on train, then run the best parameters on test. Do the results hold up?
4. Compute the Kelly fraction for SMA Crossover from your backtest results. Compare the equity curve under full position vs half Kelly. How much did the maximum drawdown change?
Summary
In this article, we moved from the simplified vectorized backtest to a full event-driven one:
- Built a `Portfolio` class that simulates a real trading account with commissions and slippage - Implemented an event-driven loop that processes data bar by bar - Saw how transaction costs affect returns in concrete numbers - Wrote a grid search for parameter tuning and learned to read the heatmap - Worked through walk-forward analysis as a tool for fighting overfitting - Got acquainted with the Kelly Criterion and position sizing
We now have a solid, realistic tool for evaluating strategies. The next step is to teach the bot to predict the market rather than just react to it.
Important Disclaimer
Algorithmic trading involves real financial risk. This series is educational material. It will teach you how to build trading bots, but it does not guarantee profit. Good backtest results do not mean good results on a live market. If you decide to trade with real money, do so with capital you can afford to lose.
What Is Next
In Article 7, we add machine learning. We will take historical BTC data, build features from technical indicators, train logistic regression and random forest models, and find out whether it is possible to predict price direction better than random chance. Spoiler: it is - but with caveats.
If you want to try it yourself → https://polymarket.com/?r=copy
See you in Part 7!