At
old thread illiki23 left several responses so I asked our CTO Alexander Gandzha to comment it. So here are his replies.
Deep learning is really hot right now and has a lot of potential though has its downfalls such as model interpretability given the complexity of the resulting models and takes a lot of training data before you get decent classification and prediction accuracy.
We agree that you will definitely need to analyze more than the history of price movement. Algorithmic traders moved beyond price history analysis decades ago and to keep up you will have to as well. The efficient market hypothesis is somewhat relevant. Feature generation is one of the most critical parts of the data mining process as it leads to more information (you can have an awesome machine learning algorithm which fails to produce good results if given the wrong features). Lately I have seen people on the crypto scene using GitHub indicators (for example if a project posts an update), detecting trends in sentiment on Twitter and in forum threads, and so on. To get an advantage you want to be a first mover and use features others aren't or use them in a new way. One critical feature involves 'shilling detection' as the detection of shilling would make some features irrelevant.
For classification (such as classifying a price increase as a pump or natural growth or classifying coins into 'winners' and 'losers') I would really like to see 'gradient boosted decision tree induction' as one of the available classification algorithms. It is really neat and has replaced random forests as my favorite. It deals with things like overfitting rather well though has a number of benefits which will help when classifying new unknowns. As for numeric time series prediction make sure you give due diligence to recurrent networks. They 'remember' thus can find patterns within long sequences of temporal data.
In DataTrading system we try our best to move away from the classical technical indicators, especially those in which you need to adjust the parameters. Instead, our team is using machine learning algorithms. On the one hand, we focus on neural networks, but we also know how to work with other machine learning algorithms. We plan to provide complete list of machine learning algorithms with simple and understandable interface for using them in the final product of DataTrading.
I see you address some of these things in the whitepaper. LSTM cells would be great! I do have a few concerns about the scope and feasibility of the project. As it is machine learning and AI is used extensively in algorithmic trading and adapts rather fast. You need to construct a platform which allows for quick adaptation.
We are developing a model, which is capable of self-learning. Currently our team is working on the criteria for the assessment of model's learning quality with possible settings for its optimization. Thus, the flexibility of the model and the adjustment to the trader's tasks will not require a deep understanding of the learning process and will also allow the trader to adapt model to his needs quickly and independently.
On a side note - it would be really neat to incentivize hand labeling and tagging of data points. One problems with machine learning is a lack of labeled training data. I like unsupervised and semi supervised learning but supervised learning is just so powerful. Tokens could be provided to people though you would want to establish some sort of expert ranking based on how accurate they are.
Yes, we agree, lack of labeled training data may be a problem in machine learning. We've setup automatic data labeling according to our profit function. We also make automatic clusterization that does not need labeling.