Bob's Bots

Bob's Bot Workshop. Free to Make and Test Profitable, Just Pay for Hosting or Download. Results not guaranteed. Backtests are hypothetical; trading risks loss of capital.

Buffett's Berkshire: ~1.5%/mo for 59 yrs. Ours: ~5.2%/mo backtested, drawdowns to −40%.

In-Sample vs Out-of-Sample Testing, Explained

When you tweak a bot's settings until the backtest looks good, you are fitting it to that stretch of history. That stretch is the in-sample period. With enough knobs you can make almost anything look profitable in-sample.

Out-of-sample data is history the bot was never tuned on. If a strategy only works in-sample, you probably found noise rather than an edge.

How Bob's Bots splits the data

We test each bot on the whole history, then report the most recent portion separately as out-of-sample. A bot that makes money in-sample but not out-of-sample gets the badge Profitable in-sample only. That is a warning, not a pass.

Walk-forward testing

Walk-forward testing cuts history into consecutive windows and checks each one. A real edge should show up in most windows. One that comes from a single lucky stretch won't.

Luck checks

If you try 50 variations, one of them will look good by chance. The deflated Sharpe ratio corrects for the number of variations tried. A random-entry test compares your bot with bots that enter at random times under the same exits and costs. If random entries do as well, the entry rule isn't adding anything.

The rule we follow

A strategy only counts as validated once it has passed on data it never saw.

More guides

What Is a Backtest? A Plain-English Guide for Trading Bots · Trading Fees, Slippage and Funding: The Hidden Costs of Trading Bots · How to Read Bob's Bots Backtest Verdicts

Build a bot freeSee tested templates

Results not guaranteed. Backtests are hypothetical; trading risks loss of capital.