Hello,

I’ve been using EA studio since 20 February and I must admit that I love the process. I really enjoyed to play with params and generate strategies. I have a dedicated server for my work and I’ve created seven virtual machines that calculate strategies almost 24/7 since I started, so the system analysed millions of strategies up to now. I think I’ve tried all the possible combinations and parameters 🙂

Last week, I was very confident with a collection and I divided it in two portfolios that I put on a real account with minimum lots of 0.01 to try the system on a real account.

And after one week of trading, I’m literally horrified with the results. I’m not talking about the fact that the system didn’t work, I’m aware that even the best systems may have drawdowns, but I’m horrified with the difference between what ea studio calculates with my two systems vs what I have in reality on my account.

Here are the results for this week :

Strategy 1 : 20 eas on one porfolio (15 are selected in portfolio because only 15 opened a trade during the week)

Theory: 

Reality: 

So we get a difference of 6 trades and 239$ (multiplied by 10 my results for 0.1 lot like in ea studio)

Strategy 2 : 26 eas on one portfolio :

theory : 

reality : 

So we get a difference of 64 trades and 377$ (multiplied by 10 my results for 0.1 lot like in ea studio)

And of course, I was on the correct pair, with the correct time frame, and I’m working with the correct historical data for the same broker. And data were imported with the ea studio script where I’ve added the correct commission for IC Markets.

I’m not here to criticize the system. I like it very much and I also like the energy and kindness of Petko.

But after this first week of real trading with the system, I’m totally desperate. I have the impression that I’ve totally lost my money and my time on something that I can’t rely on. For me it was a huge money investment. And I really don’t trust the system anymore. I can assure you that I’ve checked everything 10 times to find out where I made a mistake on my tests and comparaison, but I really don’t find where my mistake is. If someone sees it, please tell me. I hope so much that I made a mistake somewhere and that the system works fine…

Are there some people here that have the same issues/differences ?

Does some people really earn money on real life with the system ?

Thanks for the long lecture…

Have a great day

David

Hi David,

First of all let me assure you without any doubt whatsoever that EA studio is exceptional software and highly accurate. I can guarantee you that the mismatches are your mistakes and not the software’s, which I hope is comforting :-). Early days I remember having similar frustrations but it was always my mistake.

I will get to addressing your main issue in a second but first there is a far bigger issue to address. This is the fact that you are not ready to be trading live yet and certainly not with a meaningful amount of money. I think its good to have a live account and you can reassure yourself how things will compare between demo and live by uploading data from both the demo and live server from the same broker and comparing backtest results etc. BUT besides that you should not even think about trading live until you have had consistent success in demo.

As a rule I think only when you are happy with your demo results over three consistent months should you move to live. Of course that doesn’t mean every month should be great but the aggregate should be and the poorer performing months should be controlled (I.e Month 1 = +15% profit, Month 2 = -5% profit and Month 3 = +10% profit would be a good result in my opinion).

You have made no real mention to how you are using EA studio and how you are generating and selecting your EAs. How many of Petkos courses have you watched and what general guidelines are you following may I ask? Your approach seems pretty brute force from what I have read above?

Its great that you enjoy the process though, this to me is an extremely important indicator of your likely success in the future. But its not as easy as just generating loads of strategies with good looking equity curves and throwing them on a live account unfortunately 🙁

So my first recommendation is stop the live trading account trading immediately and start practicing on a demo. I hope you agree?

Okay, so your main concern seems to be how different your live trading results are from your EA studio comparison during the same period.

Firstly I suggest the following:

Take a single EA from EA studio and compare the results with a backtester using Mt4 backtester using open prices. Petko has a video on doing this which I am sure you can find easily on youtube. In doing this compare the trading journal of each and look at the times of opening closing trades etc. Things should match really well and if not then you have made a mistake somewhere for sure.

After this is done with a single EA feel free to repeat the exercise with a portfolio EA and again things should match fairly well. They wont be exact and in my experience EA studio is cleverer than Mt4 in calculating drawdowns etc even though both are using open prices, however it is an important check and should help build your confidence in the system.

Also note that the MT4 backtester does not include commission but thats not important at this stage, just make sure you are content that both your EA studio trade journal and Mt4 backtester match in terms of times of opening and closing trades (and also method of closing such as stoploss, take profit or exit signal).

Then you can run the EA on demo for a week and compare the MT4 backtesting report, EA studio journal and the actual trade log made on demo to see how things are comparing before finally adding live with smallest trade size to compare also if you really want.

So in a nutshell:

Step 1 – For a single EA compare EA studio trade journal with MT4 backtester report

Step 2 – Then repeat this exercise in a weeks time but also include the EAs trades on Demo as a comparison

Step 3 – Repeat step 2 but include live as a comparison

Step 4 – Can also do this with a portfolio EA but start by keeping things simple

Once you have done all this you will learn a lot and have your faith restored 🙂

 

Hi Samuel,
And thank you very much for your response.
To be honest I’ll only be reassured when I’ll find my error or when I’ll see with my eyes that the system correspond with reality, but I’m happy that my message has been posted on the forum and that assure me that the system is not scam or dream seller.
I didn’t talk about the way I’ve calculated the strategies because in my opinion, it was not relevant concerning my issue. Whatever the system, the results should match. Of course not to the cent, but not with these differences.
I’ve watch a lot of Petko’s videos, here and on Udemy (where I first heard of this software), but not all, I must confess 🙂
In this case, the collection has been build on GBPJPY 5m, with only trailing stops 1->250 (may be that’s the problem) and no TP, with all available data from premium and my acceptance criteria were more than 100 positions and a profit factor >= 1.2. I used the reactor with everything checked except multi market. The software calculated more than one week and I build a collection of 300 filtered with best net profit. I think that more than 1000 strategies were pruned by the software to get this 300 collection.
With that collection, I’ve created two portfolios:
1: 20 best system quality number
2: the stategies that passed multi market validation in validator (in my case 26)

Of course your right with your two points. I’m not ready to trade live and I must analyse the situation with one ea at a time to get my confidence back. With portfolio, it’s merely impossible to really check backwards what happened.

I’ll do that this week and I’ll share my results 🙂

Once again thank you for your response

Have a great day

David

Hi David,

You are welcome regarding my response. Could empathize with your desperation as losing money certainly isn’t fun!

Yes of course you need to see it with your own eyes. I would expect no less! But if you follow those steps a lot will clear up I am sure.

And actually I would suggest starting the suggested tests with a normal stop loss rather than a trailing stop as its clearer in MT4 comparison because of how MT4 handles the trailing stop.

I actually tend to avoid trailing stops for this reason and I haven’t seen much benefit, but I understand that they are useful for trend following approach and all should be well if you use them but keep things simple to start. You will be able to check with portfolio and trailing stops after you have followed those steps through but definitely start simple first as it will make everything clearer and easier.

It sounds like you are targeting a trend following approach with the trailing stop and and no TP, is this correct?

Yes, you can absolutely rest assured that this is fantastic software that has clearly had many years of time and passion put into it. If there were actual mismatches that were a genuine bug then Popov would certainly want to know about it, but I very strongly do not believe what you have shown is evidence of this and needs to be broken down into smaller steps for you to get to the bottom of it properly.

Also if you do find any legitimate bugs you can email Miroslav Popov directly and he will respond and fix them very quickly. But in my experience often these are almost always our own mistakes or misunderstandings as it is highly polished and quality software.

Happy troubleshooting 🙂

 

Oh also, if you are trend following then M5 wouldn’t really be a good time frame for that. Might be an idea to stick to higher timeframes when starting out but that’s a whole other discussion.

Thank you for your concern 🙂
It’s not much the money I’ve lost but more the idea that the system could not work that gives me more despair. Because I had so much faith in it and I really fell in love with the process.
As you assure me that the system works fine, I’ll really try to find out what is the problem. It can be fun too and I’m sure I’ll learn a lot in the process.
I’ll test 8 EAs. 2 with trailing stop and 2 without on two different pairs. And as soon as I get enough data to work with, I’ll come back with the results 🙂
And to answer your question, I don’t have any preferences for trend following. I use these parameters because with all the tests that I’ve made up to now, they give me the best and stable results.
Thank you very much for your time and responses

Hello all,

I think I know the error.

In my case there were two to be exact.

1. wrong server time zone set.

2. using data from the broker.

On the first point I had a 1 hour time shift in my systems. Which led to beautiful equity curves but once it was right made the system useless.

With the 2nd point, there is not much you can do, because all brokers have very miserable historical data.

I imported them, but then adjusted the premium data accordingly like icmarkets. So you have the complete data but with icmarkets settings.

Or you can buy historical data somewhere for a lot of money. But that would be in the Formula 1 league.

I hope this helps to find the error, good luck!

Übersetzt mit DeepL (

Hi Selim,

I am just going to expand on your points to make things clearer for David but they are very good points!

For Point 1:

I also suspect a time issue which will become clear when David compares the trades between MT4 backtester and EA studio as it will be easy to see if all trades are shifted by a set amount of time. Additionally when only checking a week the fact that EA studio only starts trading after 100 bars needs to be accounted for to get the best match which will also become clear to David when he is working on this.

However this only really applies for the premium data as when importing the broker data as David has done the time will match and does not need to be set.

Have you set any specific time settings in EA studio David or is the 00:00 – 24:00 applied?

For Point 2:

What David has done is actually correct in using the broker data to compare rather than using premium data that has the settings applied.

When generating strategies we should use the premium data with settings adjusted because as you say there are no missing bars and we can get this data for. The two main issues with the broker data for generation is missing bars and the fact that there is not enough of it usually (Sometimes they just give a month even although most give more sensible amounts).

But what David is doing is just comparing a weeks live trading results (although will be demo going forward 🙂 with EA studio (and MT4 backtester also until things matching up) and in this sense it is definitely best to import the broker data to EA studio and check using that. In this case David will have collected the broker data himself with the MT4 terminal real-time so isn’t even downloading a chunk of data from their server really. Always check the data quality using EA studio s Data statistics which is super quick and easy to do.

Also the historical data tool/premium data is all we need for generating good strategies in my opinion, you really wont be getting anything by paying for additional data. You can even download free tick data from Dukascopy directly if required but again that is free so there should be no need to pay for anything.

 

Hi Selim and Samuel,
thank you for your insights 🙂
I’ve never thought that time zone could such important here. I naively thought that for most indicators, it’s just a succession of bars where hour has no importance at all.
But I’ll see it very rapidly during my tests 🙂 And if that’s the case, I’ll have to change the timezone in premium data, that’s it ? I didn’t change any time settings up to now.
If it’s possible, I would prefer to keep working with premium data, because it’s free and it has more bars and up to now I use short time frames. I just change the spread and commission to reflect reality. But with higher time frames, I’ll probably use the broker history to be more precise. Because if I test a system with ICMarket-live10 or ICMarket-live12, I have completely different results. I found that amazing and also disturbing. But may be it’s because I work with short time frames too…

Hi David,

If you have imported data from your broker and not applied any specific time settings in EA studio then the time of your trades made on your demo and shown in the EA studio journal should match. Also start of course by checking using the MT4 backtester on your broker MT4 as discussed also.

As you say the time isn’t that important if no time settings are applied and especially over a large time period (It could be significant for only a weeks results but will be insignificant backtesting over a couple of years for example).

Also, I honestly just don’t think there is any need for different data for generating strategies with EA studio other than the premium datahistorical tool. I heard of some people optimizing  strategies for their broker data but personally I think a simple check that your strategies equity line is similar between premium and broker data over the same time period (i.e. for 6 months if that is all the broker data you have) is sensible enough.

As for the concern over the two live servers giving different results, that does seem odd. What are you comparing exactly? Do you mean that you have put identical robots on two separate IC market live accounts with different servers and the results are very different? Or are these checks within EA studio. Seems like a different issue, possibly due to using a short time frame and poor latency (Although in that case I would only expect the actual profits to be different but the times of entry an exit should mostly be close, could be a bit different for stop loss and take profit exits though of course). What latency do you have with your broker? I am assuming you are trading on a VPS?

There is a lot being discussed here actually which can always be a bit confusing. For problems like these I would always advise breaking the problem into as small pieces as possible and avoid trying to solve too much at once.

But I am still sure all will become clear to you once you work through the steps as discussed 🙂

 

 

Hi Samuel,

You’re right, we’ve covered a lot of subjects here 😅

The differences in live accounts were only on EA Studio, where I was testing my portfolio to explain my case here, I never tried same EAs on different accounts. For my curiosity, I’ll come back to all these questions afterwards. But my primary goal now is to prove myself that the system is reliable, following your steps, because without this assomption, everything else is useless.

So I really can’t wait to check the results this we, hoping that I’ll have enough data to see something interesting 🙂

And to answer your question, my mt4 are running on a virtual machine on a dedicated server in Germany. For my two IC Market live accounts, the average latency is 90-95, but I don’t know it it’s good or bad…

Hey David,

The first and most important check really shouldn’t take any time to collect data. Just grab an EA from EA studio and compare the trade journal in EA studio (using the imported broker data) with MT4 backtester open prices – certainly cant doubt MT4 backtester. Should only take 10-15 minutes to do that really. This first check should definitely quickly put your mind at ease regarding EA studios backtesting quality (Which is honestly absolutely excellent ;-).

Then you can set that EA on demo and wait to see how it compares with the backtester in a weeks time for the second check.

 

Hello,
for those who are interested by the results, here there are.
colors on first block compares reality with mt4 tester, and colors of second block compares mt4 tester with EA Studio.

Methodology:
I’ve generated 20 strategies on 1 minute time frame. It’s not a time frame that I would use in real world but it was the only one that could give me enough trades in one week.
The ea traded all week but as my wonderful windows automatically restarted on Thursday evening, I’ve only took the trades from Tuesday to Thursday evening. So three full days.
In my strategies, there was 5 fixed SL on GBPJPY AND AUDUSD, and 5 trailing SL on same pairs, so I had 20 EAs running at the same time.
I’ve only analysed the fixed ones, but if someone here is interested in the trailing ones, he can tell me and I’ll make the effort to analyse them.
And what I’ve discovered completely confirm my doubts and first impressions, there are real differences between real trades and calculations.
Out the 10 strategies analysed, only two have full correspondance between real trades, mt4 tester and ea calculations. But EA Studio make a good job with 8 full correspondance with mt4 tester (only 4 different trades on 34, every others correspond to the minute).
So in summary, for 8 strategies out of 10, the results are not the same between reality and theory.
The worst strategy is probably the first one (101001), where no single trade correspond between the three systems (I’ve verified everything 10x, for those who wonder). So I checked the system to see if it’s a complicated one, but here is the configuration. It could not be more easy…

I’m so depressed. I’ve already lost so much time on this for nothing but frustration.
I’ve been thinking a lot about where I could make an error, but there is no difficult operation in the process. I’ve renamed all the robots and initialized every magic numbers. I’ve imported directly the robot in EA Studio to be certain that I compare the right things. Some times there are more trades, sometimes less, sometime same, sometime different. I really can not see any explanations…
I’ll put tomorrow more realistic strategies on 5min and 30min time frame, that I’ll put on two different demo accounts on two different computers and I’ll let them run for a few weeks until I have enough trades. I hope that it’s only the 1m time frame that give more chaos and that I’ll get better results like that…

Hi David,

First of all well done for your efforts. I can see that you have put in some work and the results are no doubt frustrating!

Firstly I have some very good news for you. You may have read on the forum that I will be releasing videos on Pekto’s new YouTube channel. For my first couple of video’s I will be addressing this very issue and will walk through the comparison of MT4, EA Studio and then a demo account.

You can expect the first video within a couple of weeks from now at the latest.

Do not despair, I assure you there is no issue with EA studio and that there is a mistake somewhere in your working.

You are also trying to do way too much at once which will further be clouding the issue.

As per my initial recommendation was to first only compare MT4 backtester and EA journal, only then move onto comparing demo results but you have tried to do everything all at once (less haste more speed).

While you are awaiting these videos lets take a look at things anyway as I can feel the frustration 😉

So firstly I suggest you focus your matching efforts ONLY on your strategy 101002 for now. I can see that in that case all the times of trades match but the prices don’t. (In fact between the MT4 tester and ea journal that’s actually the case for all of them except the 101001). Let’s completely forget the reality for the time being and address one issue at a time.

So I can see that all the trades are opening and closing at the same times which is great. The profit not matching of course is not so great.

Here are the first few obvious things to check:

1 – Are your swaps matching both for EA studio and MT4 at the time of performing your tests? Remember they change so make sure you check when you test

2 – Have you used the same spread for both the MT4 and EA studio backtest

3 – Do you have commission included in EA studio? (To match with MT4 backtest you will need to turn off the commission as it doesn’t include it – but obviously EA studio is the more accurate check in this case and turning it off is just for comparison purposes)