hi,

 

I have done the AUDUSD course as well. And in that one explains how to do backtest on All History data – 1 Month. And also to select for example Out of sample 10%.

In the initial course I did “Trading portfolio Expert Advisors + 100 EURUSD strategies” I just explains to put “in sample” and montecarlo validation.

My question is. What would be the difference to use Out of Sample 10% or in Sample. To generate strategies which one is more recommendable?

 

thanks

Facundo

Hello Facundo,

Glad to hear from you. When you use In Sample, you are generating strategies over the complete Historical data that you have.

If you do that, it is a must to use Monte Carlo to check for the robustness of the strategies and avoid over-optimization.

When you use Out of Sample of 10%, for example, you are generating the strategies over 90% of your Historical data, and for the last 10%, you are testing the strategies. If your 10% of data is five months, it would mean that you were placing the strategy for trading 5 months ago, and you see the result. This is an excellent method to check if the strategies were over-optimized from the generator.

However, if you combine the Out of Sample with Monte Carlo, it is the best according to me because this way, you are testing the EAs with two different instruments.

Petko A

Good morning Facundo,

Sorry for the storybook I’m about to Write.

In sample and out of sample are two different ways of looking at your strategies. FSB and EA Studios utilize machine learning to find and backtest strategies. “In Sample” is what the software uses to build the strategy on, and depending on your acceptance criteria will then put into the collection for you because it’s saying “i found what you might be looking for”. The problem with this is that you have an extremely high chance of over optimization and wouldn’t know it until you put the strategy onto your demo (which will become valuable real estate as you produce more strategies). “Out of sample” comes into play as a simulation of what the strategy “should have done” if you placed it on a demo or live account when the OOS period started. However as i explained in another post, Market timing is now your next obstacle if the strategy has passed acceptance criteria, OOS, and Monte Carlo. When we place the strategy is also important, it could misread the market structure and get dragged around. This is why we see a huge separation when placing them on demo. Some are in tune with the structure and fly to the top of our FXblue list. Some are not in tune with the current moves in the market and fall the bottom. One of my first strategies I created at the end of 2018 wasn’t in tune and fell fast from my radar and was removed from further testing at the beginning of 2019. March of 2019 i bought a new PC and gave it another go since i didn’t have a huge selection of strategies on demo. It’s currently my 2nd best EURUSD strategy behind Petko’s that he released to us in September of 2018. Moral of the story, keep trying different things to find what you’re comfortable trading, then find your way of managing them on the live accounts. FSB and EA Studios are tools, its up to us to figure out how to use it to our advantages.

Petko posted while i was writing my novel LOL

Thanks to both of you for spending time with such a clear explanation. Appreciated

No problem Facundo. Just keep running the generator/reactor as much as you can to get a good feel for what settings you like.