When choosing an EA, the first thing most people look at is the “backtest results.” A clean, upward-sloping equity curve alongside numbers like “profit factor 3.0” or “win rate 95%” can make an EA look impressively strong. But those numbers don’t necessarily mean it will keep winning going forward. In fact, in the world of FX, it’s a common occurrence that the better a backtest looks, the more likely the EA is to fall apart in live trading.
This page translates what an EA’s (reading What Is an EA? Automated Trading Explained first will deepen your understanding) backtest actually does, what to check in the MT4/MT5 Strategy Tester, and the biggest pitfall of all — “the backtest looked great, but it doesn’t win in live trading” — also known as over-optimization (curve fitting) — into terms beginners can judge for themselves. We’ll also look concretely at how to read the numbers, using our own SMC Gold Sniper (GOLD/M30, backtested 2018-2026, PF 1.87, max DD 8.2%) as a real example.
What Is a Backtest? “Taking a Mock Exam on the Past” (and Why It Matters)
A backtest is the process of applying an EA’s trading rules to historical price data to calculate “what results this rule would have produced if it had been trading in the past.” Think of it like school: before the real exam (live trading), you take a mock exam using past questions to gauge your ability.
Why is this necessary? Because an EA is essentially “a program that codifies market rules,” and nobody can know whether that rule actually works until it’s put into action. If you jump straight into live trading with real money and the rule turns out to be flawed, the tuition fee comes back to you as real losses. Testing it against past data first lets you at least weed out “rules that didn’t work in past markets” before you lose any money.
There’s one crucial premise to keep in mind here, though: a backtest is “a performance record against the past,” not “a guarantee of the future.” Just as you can score high on a mock exam and still stumble on the real one, it’s entirely normal for an EA with a great backtest to lose money in live trading. Why a good backtest can still lose is the core question we’ll dig into in the second half of this page.
What Is the MT4/MT5 Strategy Tester?
The tool actually used to run a backtest is a feature called the “Strategy Tester,” built into MT4/MT5 (the trading platforms). You load an EA file into it, set conditions such as the currency pair, period, and timeframe, and run it — the tester then replays historical price movement at high speed and calculates, all at once, where the EA would have bought, where it would have sold, and how much it would have won or lost.
The results are output as a profit/loss chart (equity curve) along with a performance report that includes the profit factor, maximum drawdown, and more. We translate what each item in that performance report means, one by one, in How to Read EA Performance Metrics — if you want to know the pass/fail line for each number, read that page alongside this one.
The Difference Between Test Modes (Every Tick vs. Open Prices Only) and How to Choose
The Strategy Tester has a “test mode” setting that determines how finely it reproduces price movement. The first thing beginners should know is the difference between the following two modes.
- Every tick: A mode that tests by reproducing even the fine price movements inside each candle (i.e. “ticks,” explained below). It’s closest to live conditions and the most accurate, but it takes longer to calculate. EAs that “make decisions mid-candle,” such as ones using martingale-style averaging (nampin) or trailing stops, must always be checked in this mode.
- Open prices only: A simplified, fast mode that judges only at the moment each candle is confirmed (its open price). It’s convenient for getting a rough sense of the trend, but because it ignores conditions triggered mid-candle, results can look either better or worse than reality.
If the backtest image on a sales page or in a distributed EA was made using “open prices only,” it hasn’t accounted for price movement inside each candle, so the results may be skewed from reality. Which mode was used to verify the backtest is something you should always check before taking the results at face value.
How to Run a Backtest (Period, Currency, Timeframe, Spread, and Modeling Quality)
To be able to “read” a backtest yourself, you need to look not just at the resulting numbers but at the conditions under which those numbers were produced. Even the same EA can be made to look as good as you like simply by changing the test conditions. Here we’ll organize the baseline conditions beginners should check at minimum.
| Check Item | What to Look At | Warning Sign (a state worth doubting) |
|---|---|---|
| Test period | How many years of data was used | Too short, e.g. a few months to a year. Cherry-picks a specific market the strategy happens to be good at. |
| Currency pair/instrument | Whether it’s specialized to one instrument or general-purpose | Only shows one instrument, and it’s unclear whether it works on others. |
| Timeframe | Which timeframe it’s designed to trade on | Tested on a different timeframe than the one used in live trading. |
| Spread | How many pips of trading cost were used in the calculation | Assumes zero spread or an unrealistically narrow one. |
| Modeling quality | Data reproduction accuracy (%) | Below 90%, or not shown at all. |
| Number of trades | Whether it’s a large enough sample | Too few, such as only a few dozen trades — high chance the results are due to luck. |
Historical Data and Tick Data
The foundation of any backtest is “historical data” — the recorded history of past price movement. If this record is sparse or full of gaps, then no matter how precisely you run the test, the raw material itself is poor, so the results can’t be trusted either. It’s exactly the case of “garbage in, garbage out.”
This is where the unit called a “tick” comes in. A tick is the record of a single moment when the price moved — essentially, one beat of the market’s pulse. A single candle can contain dozens of these beats. Using “tick data,” recorded down to the individual tick, lets you reproduce even the up-and-down movement inside a candle, so you can test with a level of precision close to live trading. In high-precision EA verification, it’s common practice to separately import and use good-quality tick data like this.
Why 90% or Higher Modeling Quality Is a Good Benchmark
The Strategy Tester’s results report displays a percentage called “modeling quality.” This is a metric showing “how faithfully the past price movement was reproduced” — put simply, it’s the reliability of the backtest itself.
Generally, 90% or higher is considered a reasonably trustworthy test. If it’s 50%, or not shown at all, the price movement inside each candle hasn’t been reproduced well enough, and you need to be on guard that “these results might differ from reality.” Be especially careful with EAs like martingale-style averaging (nampin) systems that make decisions repeatedly mid-candle — when modeling quality is low, the gap versus live trading tends to be larger.
How Execution, Slippage, Swap, and Commissions Are Handled
Another reason backtests diverge from live trading is whether “the real-world costs of trading” are factored into the calculation. These are items that tend to get ignored in an on-screen simulation but definitely occur when real money is on the line.
- Spread: The difference between the bid and ask price — effectively, the trading cost. The narrower it’s set, the better the backtest looks, but in live trading it widens during sudden market moves.
- Slippage: The gap between the price you ordered at and the price you were actually filled at. It tends to move against you more in fast-moving situations, such as around economic indicator releases. Since a backtest basically calculates on the assumption that “the order filled exactly as placed,” this is where it diverges from live trading.
- Swap: The interest-rate-differential adjustment that occurs when a position is carried over to the next day. For EAs designed to hold positions for a long time, or ones designed to carry unrealized losses for a long time through martingale-style averaging, this cost adds up.
- Commissions: Depending on the account type, a separate trading commission may apply on top of the spread.
A backtest that underestimates these costs is, in effect, showing “performance in an idealized, zero-cost world.” Once these costs weigh on live trading, an equity curve that was sloping upward gets gradually eaten away. Keep in mind that scalping-style EAs (ones aiming for many small, thin profits) are especially vulnerable, since their per-trade profit is small enough that the impact of costs can be fatal.
Translated by AI
Conditions like “modeling quality” and “spread assumptions” are, in short, a gauge of “how close to live-trading conditions that backtest was run in.” When we have our AI read a backtest image, it looks at the test period, modeling quality, and spread settings before the performance numbers themselves, and marks a result down if the conditions were too lenient, even if the performance looks good. The trick is to translate “what assumptions the numbers were built on” before translating the numbers themselves.*This is an AI interpretation and does not guarantee future performance.
Why a Good Backtest Can Still Lose in Live Trading (Over-Optimization, a.k.a. Curve Fitting)
This is the part of the page we most want you to take away. If a more perfect backtest always meant more success in live trading, none of this would be a problem. But in reality, there’s a paradox: the better an EA’s backtest looks, the more prone it tends to be to falling apart in live trading. The main culprit behind this is “over-optimization,” also known as curve fitting.
Over-optimization is when an EA’s parameters (numerical settings) are tuned too aggressively “to produce the best possible results on past data.” Picture a student who memorizes past exam questions word for word, scores perfectly on those exact questions, but can’t solve anything once the real exam changes even slightly. An EA fitted precisely to the past (known data) can’t handle the future (unknown markets) — a fundamentally different problem.
What’s frightening is that an over-optimized EA’s backtest looks extremely attractive. The equity curve is an almost perfectly straight line rising to the right, the win rate is high, and there are almost no losing trades. Beginners in particular tend to jump at it, thinking “this is amazing.” But that very smoothness is often evidence that the EA was fitted conveniently to the past, and only the past.
What Long-Term Verification (2018-2026) Means
The simplest weapon for spotting over-optimization is “the length of the test period.” That’s because markets go through many different phases: years with strong sustained trends, years of directionless ranging, and sharp downturns like the COVID shock or sudden rate-change shocks. If you cut the window short, you risk only ever looking at a market that happens to suit that particular strategy.
Our SMC Gold Sniper is backtested over the long stretch from 2018 through 2026. That means it has been run through a full range of market conditions — trends and ranges, calm periods and sudden shocks alike. If performance holds up after passing through multiple market environments over a long period, that’s one piece of evidence that the rule isn’t fragile, fitted only to one particular kind of market. Conversely, if an EA only shows a backtest for the most recent year, it’s fair to suspect that year just happened to be a market it was good at.
An Unusually High PF Can Be a “Warning Sign”
One of the performance metrics is the “profit factor (PF).” It’s calculated as total profit divided by total loss, and a value above 1 means profits exceed losses. We’ll leave the detailed explanation of how to read it to How to Read EA Performance Metrics, but there’s one point worth making here.
Intuitively, you’d think “the higher the PF, the better.” In the world of verification, though, an unusually high value — a PF above 2.5, for example, is sometimes treated as a sign of over-optimization rather than a good thing. It’s hard to imagine a rule that loses that rarely in real markets, so there’s a real possibility that “fitting the rule too conveniently to the past has unnaturally erased the losing trades.” As a general guideline, a PF in the roughly 1.3-2.0 range is considered less likely to reflect extreme fitting. Our SMC Gold Sniper’s published PF of 1.87 falls within this “realistic, no need to over-suspect” zone. Being neither too high nor too low can actually be taken as a sign of health. We go further into “how to be suspicious when PF is too high” in How to Read EA Performance Metrics.
A Note from Our Researchers
The first thing beginners trip up on is the assumption that “a good backtest equals a winning EA.” The truth is closer to the opposite: when you see a backtest that looks too perfect, get in the habit of first asking, “Why are there so few losses here?” Instead of hunting for good results, hunt for inconvenient numbers being hidden — maximum DD, losing months, behavior during sudden market moves. Once you can make that shift in perspective, the odds of putting money into a dangerous EA drop sharply.
How Discretionary “Verification” Relates to an EA’s Backtest
The idea of “verifying a method against the past” isn’t unique to EAs, in fact. People learning discretionary trading (trading where a human makes the buy/sell decisions) do exactly the same thing. The process of going back through past charts and counting up “would this rule have won if I’d entered here?” — verification, or backtesting — is the standard path discretionary traders take to develop their method.
In other words, it makes sense to think of an EA’s backtest as the same “past-chart verification” done in discretionary trading, just handed off to a program and massively sped up. Verification that would take a human several days across hundreds of instances is finished by the Strategy Tester in minutes. This is one of the EA’s biggest advantages.
And the more experience someone has with discretionary verification, the more easily they notice the pitfalls of an EA’s backtest. That’s because they already know, firsthand, the feeling of “my method would have won too if I only cut out this particular period” and “fitting to the past alone doesn’t mean the future will be the same.” If you’ve learned the fundamentals of discretionary trading, such as Dow Theory and trend structure and support/resistance and liquidity, you start to see what an EA’s trades are actually based on. Discretionary trading and EAs aren’t opposites — they’re on a continuum, where an EA is simply SMC logic automated.
Using AI to Translate Backtest Results for Beginners
That said, it’s a lot to ask a beginner to read modeling quality, PF, slippage, and everything else all at once. This is where we put real effort in: having AI read difficult backtest results and translate them into beginner-friendly language.
The method is simple: we have AI read the backtest report and equity curve, and sort the findings into “strengths,” “points of concern,” and “a one-line summary for beginners.” What was once just a list of numbers turns into language you can actually use to make a decision — things like “this period is sufficiently long,” “there are suspiciously few losses here,” or “with a max DD of 8.2%, on a 1,000,000-yen account, the design needs to withstand a temporary drawdown of around 80,000 yen.” It’s also useful that the AI mechanically checks preconditions — test period, modeling quality — that humans tend to overlook.
That said, AI’s translation is only an aid to interpretation — it isn’t magic that can predict the future. Our stance is not “the AI said it’s good, so it will win,” but rather to use the AI’s translation as material so that, in the end, you’re able to make the judgment yourself. And whether something that looked good in a backtest actually holds up needs to be confirmed at the next stage — running it in real markets, i.e. a forward test.
Translated by AI
When we had AI read the SMC Gold Sniper backtest, this is the summary it produced. Strength: “PF 1.87, tested over the long 2018-2026 period, passing through multiple market environments.” Point of concern: “Max DD 8.2% — there was a phase where equity temporarily dropped by 8.2%. Do you have the capital and the mindset to withstand that?” Beginner-friendly translation: “If you’re running this with 1,000,000 yen, plan for a design that needs to withstand a temporary drawdown of around 80,000 yen.” Once you can “see” the numbers this way, you can choose based not on good-versus-bad but on whether it fits you.*This is an AI interpretation and does not guarantee future performance.
Frequently Asked Questions
- Q. If the backtest looks good, is it okay to buy that EA?
A. A good backtest is a “necessary condition,” but not a “sufficient condition.” It can weed out rules that didn’t work in the past, but it can’t guarantee the future. If the results look too good, suspect over-optimization instead, and make your decision together with the results of a forward test run in actual markets. - Q. What profit factor (PF) counts as safe?
A. There’s no level that can guarantee safety, but as a general guideline, a PF in the roughly 1.3-2.0 range is considered less likely to reflect extreme fitting. An unusually high value, such as one above 2.5, may be a sign of over-optimization — fitting too closely to past data. Our SMC Gold Sniper has a PF of 1.87, which falls within this realistic zone. For more detail, see How to Read EA Performance Metrics. - Q. Is a backtest with modeling quality below 90% untrustworthy?
A. It’s not necessarily unusable, but its accuracy is lower, so caution is needed. This is especially true for martingale-style averaging EAs that make decisions mid-candle — when modeling quality is low, the gap versus live trading tends to widen. For performance images that don’t display the quality figure at all, the assumptions are unknown, so it’s safer not to take them at face value.
Summary
A backtest is a “mock exam” that tests an EA’s rules against past markets. It’s an indispensable step that lets you weed out bad rules before you lose any money, but it always comes with a limit: it’s a performance record against the past, not a guarantee of the future. That’s exactly why the ability to read the “preconditions” — test period, modeling quality, spread, number of trades — matters more than the performance numbers themselves.
And the biggest pitfall of all is over-optimization (curve fitting). A backtest that’s too good, an equity curve that’s suspiciously straight, a PF too high above 2.5 — signs like these actually point to the possibility of a fragile EA that has been conveniently fitted only to the past. The reason we verify our SMC Gold Sniper (PF 1.87, max DD 8.2%) over the long 2018-2026 period, and publish both the good numbers and the inconvenient ones, is to embody this stance of “doubt first, then believe.”
As your next step, learn how to confirm “whether it really works in actual markets” in What Is a Forward Test?, which covers what makes up for a backtest’s limits, and learn the “pass/fail line for PF and max DD” in How to Read EA Performance Metrics, which translates each performance number one by one. The defensive fundamentals that underpin any EA operation are covered in Stop-Losses and Money Management, and you can check the overall learning order for EAs in the EA / Automated Trading Learning Hub. The numbers we’ve actually verified ourselves can be seen on our performance dashboard (losing months included). For now, just take away one idea: read a backtest with doubt first.
Risk Disclosure
This page is not investment advice; it is analysis and verification information provided by our research lab. Past performance (including backtests and forward tests) does not guarantee future profit. Offshore brokers (such as HFM) carry high-leverage risk; we treat them as a small-stakes, high-risk verification bucket, while our main trading activity is conducted through domestic brokers (JFX/OANDA). FX and automated trading can result in losses. Always trade with disposable funds and at your own judgment and responsibility.