Scientifically evaluating robots and signal providers

When choosing a robot or a signal provider you are bombarded with promises of high gains and quick profits. This overload is hard to handle, especially for new traders, and can be overwhelming.

This article will help you understand how to handle these claims and the scientific proof they claim to have for the success of their service.

[table id=5/]

What to watch out for

Many providers of signals and robots claim to be able to generate a certain success rate for you. “Use our robot to generate 75 percent winning trades” is a typical example for how many providers advertise their service. How should you handle these claims as a trader?

Simply pick the provider with the highest winning percentage and trust that it will be able to achieve the good results he promised, right? Not so fast. First, let us consider how these numbers are created. All a provider needs to have to claim a winning percentage of 75 percent is an algorithm that would have won 75 percent of its trades in the past. This algorithm can even be written in hindsight and can therefore be so uniquely adapted to the past market situation that it is useless for the changing conditions future market’s will inevitably bring.

Even if the algorithm was created before the market movements occurred, there is no guarantee its claimed 75 percent accuracy is any indication of future results. The provider could simply have created ten different algorithms, each uniquely suited for one market condition. One of these algorithms is bound to do well.

The provider can throw the other nine away and advertise the one that worked as the new big thing. Still, it’s incredible success rate over the past months may simply due to a coincidently fit of market developments and the design of its algorithm that is unlikely to repeat any time soon.

If you invest your money based on such an algorithm, you will not only lose the money you invested to buy the signal or robot, you will also lose a significant amount of your capital.

The scientific truth about signals and robots

The simple truth is this: No prediction can be tied to a certain percentage. There are too many unknown factors to attach a probability factor to any prediction. Still, the past is the only way to predict the future. If you do not want to choose your robot or your signal provider randomly, you therefore have to use past results in some way to evaluate the service in front of you. Therefore, you need a rule how to do that.

The simplest way to determine a shaky claim from a solid one is to take a look at the sample size.

To draw any conclusion from the data in front of you, the sample should ideally include at least 200 cases. Any study based on less than 50 samples is completely useless; any study based on less than 100 samples is at least questionable. A signal or robot provider offering a solid service should have no problem providing you with the 200 samples you need to judge his service thoroughly.

When a provider splits a large sample into monthly winning percentage, make sure to always calculate the winning percentage for all accumulated trades. Sometimes, splitting a large data sample into many smaller samples is used to hide the truth and make the numbers look better than they really are.

Many robots and signal providers offer you a free trial period. Use this trial period to use your robot / signal with a demo account, if possible, or with as little money as possible. This will enable you to evaluate its real-time performance. Make sure to get a good sample size, and by the end of your trial period you will know whether you can trust your robot / signal or not.

Default Broker – US – NADEX