> For the complete documentation index, see [llms.txt](https://www.degenbot.wiki/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://www.degenbot.wiki/performance-and-methodology/evaluating-results-fairly.md).

# Evaluating results fairly

Distinguish backtests, forward observations, delivered alerts, and actual trading records.

Different records answer different questions. Do not treat them as interchangeable.

### Four kinds of evidence

| Record                            | What it can show                                                          | What it does not prove by itself                            |
| --------------------------------- | ------------------------------------------------------------------------- | ----------------------------------------------------------- |
| Backtest                          | How a specified method behaved on historical data under test assumptions. | That it would have reached users in time or will work next. |
| Forward observation / shadow test | What happened after observations were recorded prospectively.             | That members received them or could execute them.           |
| Delivered-alert record            | What was sent to users and when, with subsequent outcomes.                | Members' fills, costs, sizing, or account returns.          |
| Executed trading record           | Orders and realised account outcomes under documented conditions.         | That all subscribers would obtain those results.            |

An alerts-only product can have a live delivered-alert record without executing trades. Calling a deployment “production” does not turn a backtest into a live trading record.

### Keep the future out of the past

A method should use information available at the observation time. Accidentally using future data in preprocessing, feature selection, or evaluation can make historical results look stronger than they are. This is called **data leakage**. See the [scikit-learn guidance on data leakage](https://scikit-learn.org/stable/common_pitfalls.html#data-leakage) for the general principle; this citation is not an audit of Degen.

Changing settings repeatedly after seeing a test period can also make that period less useful as an independent test. A new method needs an appropriate new evaluation, not only a better-looking old chart.

### Keep the complete record

Review failures, expiries, missing data, and excluded observations as well as successes. Overlapping alerts can share the same underlying move; hundreds of notifications do not necessarily represent hundreds of independent events.

A statement that an alert came “before a move” needs a defined move and relevant timestamps. Detection before send, send before receipt, and receipt before the event are separate questions.

### Version changes and corrections

A new trigger, model, target, expiry, or outcome resolver can change what the percentage means. Keep versions separate unless a combined report explicitly explains the comparison.

Corrections should explain what was wrong, which records changed, and how totals were affected. A decrease after a scoring correction may reflect better measurement, not a newly worse strategy.

### A practical review standard

Ask whether the result is reproducible, dated, versioned, and based on a clearly defined sample. Then ask whether it measures the claim being made. An attractive accuracy number is not a substitute for those answers.

This guide does not claim an independent audit, verified current win rate, or a guaranteed minimum sample size for proving an edge.