A little rant on statistical modeling in finance and my personal motivation

There are very sophisticated ones out there but I would consider most approaches I see as complete failures at least for intended applications (e.g. very large volumes and profits). To understand trading bots, we have to understand the environment they are operating in. Further, we have to understand that building a trading bot is something completely different than simply predicting stock prices.

In most European countries, I observed a general denial that machine learning could ever work and deliver something useful. Well, if you don’t have any skills, then there is a tendency to deny that everyone else has it. I’ve talked to my people working on the application side of ML/AI. I found a lot of common mistakes in what I’ve been told, especially when it comes to trading bots and stock price predictions. I would like to put these people in two categories.

The ones in the first categories are the ones using trading bots (robo trader). They are quite disappointed that their growth rates are much lower than expected but relatively happy that they usually don’t lose money over a period of 12 months.

The second group are people that either build AI/ML startups/established that build a trade system or people who build a bitcoin trade bot for themselves. Additionally, I would consider most AI/ML courses that cover stock price prediction a part of this. In this group we could distinguish between providing

  1. stock price prediction
  2. trade bots

While I can understand group number one, I have serious questions regarding the qualification of group two. It seems like even some people working in finance have no little idea understanding of what they doing in terms of understanding how things work/interact/interfere. However, regarding mathematical models I observed a few things:

  • linear thinking
  • everybody aims at exponential growth which, by the way, is easy - hardly anyone aims at hyperbolic growth ;)
  • spreadsheet and org-chart thinking - the real world is much messier and could best be described as set of interconnected graphs
  • it works at company A therefore it has to work at company B as well
  • oversimplification
  • … probably a lot more things in this direction

In the following sections, I’m going to provide some views on the (IMHO) most common misconceptions.

Key difference to other machine learning fields

Like many real-world applications of machine learning, this is a time-series problem. Well, it is a spatio-temporal problem because have to consider different market places (stock exchanges) operating in different time zones. I’m not covering trade outside stock exchanges here. Stock price predictions look like a classical time series problem at first. Let’s have a look at this very simple stock price prediction example:

Predicting prices for the next day/month/second (at time i+1) is relatively simple as we see. The blue line is hardly visible. If we look a the red line, then we can see that we predict prices too early. The orange line indicates that we predict prices too late. If we combine them, then we can create a range of prices with some probabilities attached to it.

So far, so good - it looks like every time series problem. But there is a big difference. Let’s say that we predict stock prices to feed it into our trading bot. This changes the situation. We are no longer a third-party observer of price development, but we are involved generating prices. This changes everything. We are not only bidding for a price or offering a price to buy, we are triggering other parties as well, for example high-frequency trader or manual trader interference. My very personal opinion is that these systems are triggered by a threshold volume and price. Therefore, I assume that low volume trading bots could work reasonable well. IMHO it is probably possible to build trading bots that deliver profits in the USD 1 - 5k range per month (before taxes and transaction costs). This is a reasonable range claimed by people building crypto trading bots.

Let’s conclude. Our task changes from predicting stock prices to predicting stock prices by estimating how every body else acts as well. The dimensionality of such a task simply explodes. Further, we have to include transaction costs, taxes etc. IMHO it is pretty clear why so many people building such systems are facing the problem of ending with “close to break even results”.

Extremely short-term/high frequency trading bots

I’m not going to cover this. Yes, I have moral objections covering this since it is mostly used to drain money out of the stock market without providing any benefit. In the context of ecosystems, we would call this a parasite. If you are interested in building such a system, then let me simply say one thing here: your math skills should exceed those of a typical MBA person. If you studied hard science (at the level of quantum physics and above) or engineering (at least at the level of laser and photonics, error correction for RF communication), then you are probably good to go on this one. However, you should keep in mind that there are trading limits that you simply can’t exceed without causing some interference and you probably have to spend a ton of money to cut transmission length and therefore latency to a particular stock market.

Short-term trading bots

Let’s define short-term trading as buy-hold-sell strategies that cover a few hours to a few weeks. We can assume that the shorter our holding periods, the more we will trigger other parties that try to intercept orders and drive prices. Further, our transaction costs are likely to increase. However, it is reasonable to assume that short-term price estimates have a much higher certainty than. Hence, in a way this might be the most appropriate trading bot category.

With increasing time span, other factors are important that influence the behavior of other parties. For example, information that is not related to a companies performance but to customer interaction, press releases etc. can be considered:

  • twitter and press release sentiment analysis
  • information extraction of press releases, product announcements, earning reports etc.
  • massively shorting of stocks (e.g. Amazon, Tesla)

Again, this introduces different kinds of uncertainty that increases complexity of building trading bots.

Mid-term trading bots

Let’s define mid-term trading as buy-hold-sell strategies that cover a few weeks to a few months (years). Other things are increasingly important (e.g. correlations between quarterly earning calls). The short-term and extremely short-term behavior of some entities are probably less important than other competitors, who have mid-term investment goals and therefore have a different behavior, are more important.

Long-term trading bots

That is a whole different category of trading bots. To be honest, that is probably the most useful category (also for evaluation of venture capital investments) but definitely the most challenging. It would be fun to work on that ;).