Validation methods

Tested on the real market,
not on promises.

Before a strategy is offered for deployment, it goes through a validation chain that we make public. Here is how we test, what we measure, and where you can check our execution evidence in real time.

⚠ A backtest is a simulation on historical data. Past performance does not predict future results.

Exclusive strategies · How the default parameters are validated The method, in detail.

The catalog offers exclusive Botlyz strategies (the SIGMA family), shipped with default parameters that you remain free to modify before you sign. They are built with the same validation pipeline as quant funds: walk-forward, testing on unseen data, fees and slippage included. This section walks through it, in pictures, without the jargon.

scroll to discover
01 / 05Concept · Finding the right balance

The perfect tradeoff.

NSGA-II·Non-dominated Sorting Genetic Algorithm

You want to win big, but without losing big. Except those two desires pull against each other. Here is how the algorithm finds the strategy that maximizes one without sacrificing the other, out of thousands of candidates.

STEP 01 / 05

Two opposing desires.

When you trade, you want two things at the same time: win big and lose little. The problem is that the two work against each other: every time you try to win more, the risk climbs.

The chart on the right is going to illustrate this. The vertical axis is the return. The horizontal axis is the worst possible drop (what we call risk).

STEP 02 / 05

A thousand possible strategies.

Each little dot that appears is a trading strategy that has been tested: a different setting, a different behavior. Botlyz regularly tests more than a thousand of them.

You can see they form a cloud: each one has its own return and its own risk.

STEP 03 / 05

The "top-left"
is ideal.

In the top-left corner: what we dream of having. Lots of return, little risk. That is where we want to be.

In the bottom-right corner: disaster. Little return, lots of risk. And nobody knows how to win without risking a bit.

So we are not looking for THE best strategy. We are looking for the best tradeoff between return and risk.
STEP 04 / 05

The frontier of the best.

NSGA-II draws a curved line: the best possible performance for each level of risk you accept. This line is called "the Pareto front".

The green dots are on it: these are the unbeatable strategies. The others, further back, are beaten by at least one green strategy. So we throw them out.

STEP 05 / 05

The final choice.

All that is left is to choose which green dot on this frontier. Botlyz uses a third criterion: stability over time. Not the highest return (often unstable), not the lowest risk (often lazy). The point that holds up over the long run.

That is the strategy we deploy to production. All the others, we set aside.
Visualization · Pareto front · 2D
CONFIG · 0 / 1 000
RISK (drawdown) → ↑ RETURN (gain) low return too risky ideal
Preparing the axes…
02 / 05Concept · Searching without trying everything

The guided search.

TPE·Tree-structured Parzen Estimator

A trading strategy means dozens of settings to tune at the same time. Testing every combination one by one would take several lifetimes. TPE is a statistical method that guesses where to look, like a detective following the clues.

STEP 01 / 05

The problem:
too many settings.

A trading strategy has plenty of parameters to set: when to buy, when to sell, how much to risk, on which pair… If you combine everything, the number of possibilities explodes.

With just 30 parameters to set, you get more combinations than there are stars in the known universe. Testing them one by one would take longer than the age of the Earth.
STEP 02 / 05

First attempt:
at random.

We start by trying about thirty settings at random. On the chart, each dot = one tested setting. The higher the dot, the better the setting works.

The result looks like a cloud: some settings are good, others bad. But at this stage, we don't yet know why.

STEP 03 / 05

The TPE trick.

TPE (Tree-structured Parzen Estimator) is a statistical algorithm that looks at the attempts already made and guesses the shape of the terrain.

The green curve is that guess. The peaks = the zones where it works well. The troughs = the zones to avoid. It is exactly like a treasure map drawn from clues.

STEP 04 / 05

Now we dig in the right spot.

TPE concentrates its new attempts around the peaks of its map. With each new attempt, it refines its guess. The more it makes, the more precise the map.

Instead of searching a whole house, it digs where it heard a noise. Logical, and furiously efficient.

STEP 05 / 05

×12 faster.

Concrete result: with 3,200 well-placed attempts, TPE finds the same optima as a brute-force search that would require more than 40,000.

The time saved, we invest in more tests on other pairs. The same scientific rigor, applied 12 times more widely.
Sampling · trial 0 / 3 200
ESTIMATED DENSITY
SPACE OF POSSIBLE SETTINGS → ↑ PERFORMANCE PEAK #1 PEAK #2 · best PEAK #3
The challenge: an immense space…
03 / 05Concept · Testing for real

The proof, five times over.

Walk-Forward·validation on sliding windows

A strategy that works perfectly on the past can collapse the very next month. It is exactly like a student who memorizes the answers by heart: 20/20 on the exam they know, 0/20 on the next. Here is how we avoid that trap.

STEP 01 / 05

The "by heart" trap.

A student who memorizes the answers by heart will get 20/20 on the exam they know, and 0/20 on the next. They understood nothing, they just memorized.

For a trading algorithm, the trap is exactly the same: it can "learn" the past so well that it becomes incapable of trading the future. The technical term: "overfitting".

STEP 02 / 05

The fix:
hide a part.

Here is our market history: 360 days of real prices. The solution is simple: instead of showing everything to the algorithm, we hide a part from it.

It trains only on the light window on the left (60 days). The rest, it never sees.

STEP 03 / 05

Then we give it the exam.

Once trained, we show it the green window right after (15 days), the one it has never seen. And we check whether its trades are good or bad.

If the results stay good on unknown data, it means it did not memorize: it truly understood the logic.

STEP 04 / 05

We repeat 5 times.

A single exam passed could be luck. So we shift the windows in time and start over. 5 different exams, over 5 different periods.

The algorithm has to pass every exam, not just the average. Otherwise, we reject it.

STEP 05 / 05

Validated.
Or rejected.

There is the filter. Out of 100 strategies that look good in training, only about 13 pass all 5 exams. The other 87, we throw out.

Those 87 rejected strategies would have looked brilliant on paper. And would have lost money in production. That is what walk-forward is: an impassable wall for false good ideas.
Validation · fold 0 / 5
SLIDING WINDOWS
HISTORY · 360 DAYS TRAINING (90d) OOS TEST (30d) raw data VERDICT · 5 / 5 PASSED strategy validated for production deployment d 0 d 360
The complete market history…
04 / 05Concept · Simulating the past honestly

Brutal realism.

Backtest·custom-built simulation engine

A "backtest" means replaying the past with a strategy to see what it would have earned. The trap: if you simulate it poorly, you get flattering numbers that will never repeat in real life. Here are the 3 traps that most people ignore.

STEP 01 / 05

Three traps we forget.

An "idealistic" backtest assumes that you can buy exactly at the displayed price, in unlimited quantity, and without paying any fees. These three assumptions are false in real life.

If you ignore them, the displayed return is flattered. And disappointment is inevitable.

STEP 02 / 05

Trap #1: slippage.

You look at the price at 3,412 €, you decide to buy. By the time the order reaches the market (a few milliseconds), the price has moved to 3,414 €.

That is slippage: the difference between the price you saw and the price you got. Often unfavorable.

On average, this slippage costs 0.04% per trade. For an active strategy (several hundred trades per year), that is more than 1% of capital silently eaten away by the theory-versus-reality gap.
STEP 03 / 05

Trap #2: liquidity.

On the market, there is always someone on the other side. But on some less popular pairs, the order book is thin. If you buy big, you exhaust the book and the price climbs as you go.

Consequence: on a medium-sized order, the average buy price can be noticeably higher than the displayed price.

Botlyz measures the real depth of the book on each Lighter pair, and adapts trade size accordingly.
STEP 04 / 05

Trap #3: fees.

Every traditional exchange charges 0.1% per trade (taker). For an active strategy, fees can eat a huge share of capital before you even talk about PnL. The majority of strategies that are "profitable on paper" are no longer profitable once these fees are counted properly.

Botlyz runs on Lighter (0% maker/taker) and applies 0.10% per trade in on-chain integration fees, capped and revocable. All of the software's simulations include these fees and slippage, without exception: what you backtest is net of costs.

STEP 05 / 05

The result:
the truth.

When you see a return on a Botlyz strategy, it is what it would have actually earned, with slippage, liquidity and fees already deducted. Not a "magic" return that evaporates in production.

3.2 million configurations tested on 12 months of 5-minute history. Each one with the 3 traps modeled. No shortcuts.
Engine · realistic trade simulation
CONDITIONS
i
Overview3 conditions to model
3 traps
1
Slippagegap between decision price → execution price
≈ 0.04% / trade
2
Liquidityreal order book depth
per pair
3
Botlyz feescharged on-chain, included in the P&L
0.01 – 0.10%
Final backtest3.2M+ configurations simulated
12 months · 5min
A backtest is a simulation…
05 / 05Concept · The reading framework

Reading the report card.

OOS·Out-of-Sample · metrics on unseen data

When you run the backtest of an exclusive strategy, the software displays a series of metrics computed over the entire history, unseen data included. Here is how to read them, without the jargon, so you can judge for yourself pair by pair.

·  These metrics are reading tools, not guaranteed thresholds. Results vary depending on the pair, the period and the parameters you choose. No configuration is validated in advance: you backtest each strategy in the app, then you decide.
METRIC 01 / 06 · SHARPE

Sharpe, your first read.

The Sharpe ratio is the universal grade of a strategy: how much you earn for each unit of risk taken. The higher, the better.

Reading benchmarks: above 1 = decent. above 1.5 = good. above 2 = excellent.

Depending on the pair and the period, the Sharpe of a default config can be excellent or downright negative. That is exactly what you look at on the backtest before you sign.
METRIC 02 / 06 · SORTINO

Sortino, risk on the loss side.

Sortino is the Sharpe in an improved version. It looks only at the losses, ignoring sudden surges. Logical: when you win, who cares about volatility.

Reading it above 2 on a backtest means observing that the behavior during losses stayed under control over the period, not just on average.

METRIC 03 / 06 · MAX DRAWDOWN

Drawdown, the worst drop.

The "max drawdown" is the worst drop observed on the capital. From the highest peak down to the lowest trough before recovering.

It is the most honest metric, and it varies enormously: depending on the pair, a default config can show a moderate drop or a very severe one, down to a large part of the capital. The leverage and allocation you choose amplify it.

To look at first: before signing, run the backtest and ask yourself whether you could absorb the worst drop shown, without panicking and cutting everything.
METRIC 04 / 06 · CALMAR

Calmar, return relative to the pain.

Calmar combines two things: the annual return divided by the worst drawdown. It is a measure of "peace of mind": how much you earn per unit of potential pain.

Above 3, we consider that the behavior justifies the drawdown risk over the period. A negative Calmar, on the other hand, says you paid the pain without the return.

METRIC 05 / 06 · PROFIT FACTOR & SAMPLE

Profit factor and sample size.

The profit factor compares everything the strategy wins to everything it loses. Under 1, it loses money. Above 1.3, the gains clearly exceed the losses, fees included.

Also look at the number of trades: our strategies generate several hundred over the history. A good score on 12 trades is luck, not a statistic.

Many strategies "profitable on paper" rest on 2 or 3 huge wins. Measurable consistency is read on the profit factor and the number of trades, not on return alone.
SYNTHESIS

You read, you decide.

All these metrics are computed over the entire available history, pair by pair, unseen data included. Some default configs come out solid, others not: that is precisely why you keep the last word.

No strategy is "validated in advance" for you. Backtest, compare the pairs, adjust the parameters, then sign. Or not.
Metrics · reading framework
OUT-OF-SAMPLE · UNSEEN DATA
SHARPE · benchmark> 0.0return per unit of risk · > 1.5 = good
SORTINO · benchmark> 0.0like Sharpe, but ignores the surges
MAX DRAWDOWNthe worst dropread first · varies by pair
CALMAR · benchmark> 0.0annual return / max drawdown
SAMPLE> 0number of trades over the history
PROFIT FACTOR · benchmark> 0.0gross gains / gross losses
Yours to read, then to decide
The reading framework…
Honest framing

What validation does not do.

For the sake of transparency rather than promise, here are the limits. Validation does not guarantee gains: a favourable backtest over 6 months guarantees nothing for the next 6.

Regime changes
Logic that worked yesterday can stop working when the volatility or the liquidity of the Lighter order book changes.
Extreme events
Crashes, violent gaps, cascading liquidations: sometimes not captured by the backtest's standard slippage.
Candle resolution
The backtest decides at the close; the real market moves continuously, with possible partial fills.
Residual overfitting
Robustness tests reduce the risk of curve-fitting without fully eliminating it.
Real fees and funding
The backtest's 0.10% is a reference; Lighter's real fees and funding apply independently and may change.
Your responsibility
You configure, sign (EIP-712) and deploy under your sole responsibility. API keys are trade-only: no withdrawal possible.
You be the judge

Test.
Decide.

Backtest any strategy on real history and only deploy if the numbers convince you. Your funds stay in your wallet, on the DEX.

Non-custodial · Lighter DEX · No KYC

Cryptocurrency trading carries a risk of partial or total loss of capital. Past performance does not predict future results.