Ensembles of Text and Time-Series Models for Automatic Generation of Financial Trading Signals from Social Media Content

1.1 Motivation

With advancements in computation power, market investors adapt by relying heavily on empirical findings as opposed to classical economic theories. The fields of behavioral and mathematical finance emphasize a shift towards interpreting empirical findings and market realities.

Technological innovations have changed the way that news disseminates. Therefore, an emphasis is placed in this work on modern news mediums. An example of this transformation is Twitter, a popular social media platform, providing a means for posting unverified bits of information. To financial markets, any medium that breaks news first is relevant.

The focus of this paper is to use predictive analytics to gain insights on reactions to financial events. As a contribution, we address procedures to reduce the challenges of modern media.

1.2 Overview

Event studies in finance investigate the impact that events have on the value of traded firms. The impact is quantified by the change in stock prices immediately following an event. A principal study from economics provides the guideline on how best to structure an event study [11], with event analysis research considering the efficient-market hypothesis [5].

Figure 1:

Event Horizon.

1.3 Event Analysis

1.4 Natural Language Processing

A review paper in text mining for market prediction highlights that nearly all studies use the bag-of-words model [14]. The model captures word frequencies in a document-term matrix scheme. Once in this schema, any linear or non-linear classifier may be used for prediction. The frequent choices for bag-of-words are linear classifiers, namely, support vector machines (SVM) and naive Bayes. Since the task considered is document classification (more specifically, sentiment classification), one must generate a vector of positive/negative labels for each sample.

There are many variants of SVMs and naive Bayes. Thus, we look for answers in the field of sentiment classification. For sentiment in larger documents, SVMs outperform naive Bayes, whereas the reverse holds true for smaller documents [23]. Finally, a combination of naive Bayes and SVM called Naive Bayes Support Vector Machine (NBSVM) achieves strong and robust performance across standard datasets (both large and small documents) [23].

Simplicity and accuracy are the reasons why computational finance researchers select the bag-of-words model. It is easy to see that words from the model are treated completely as symbolic. There is no notion of word relationships, ordering, or context. For example, in trading, considering the terms “buy” and “sell”, the words “stocks”, “shares”, “options”, and “bonds” have a higher probability of being surrounding words as opposed to “laugh” or “cry”. Word frequencies in bag-of-words will not find the relationships above. Newer trends in textual analysis, using distributed representations of words, may capture such similarities and associations.

Word embeddings handle relational similarities by preserving aspects of similarity. In looking at the current state of financial event studies, no language model has used word embeddings. This work investigates this further as recent trends in sentiment classification move towards language models that use the full textual representation. The similarity on distributed representation of word embeddings can help find close word matches numerically. Word embeddings can be trained on all tweets.

1.5 Time-Series Analysis

For prediction tasks, modeling prices directly is not always necessary. Stock quotes have simply been used as features. However, in time-series classification it may be better to use both current and past prices entirely. A method is to use regression schemes to convert real-valued numbers into a class label. A conversion grants access to many regression algorithms for time-series predictions. As an example, we consider a regression model that predicts stock price movements in cents. What is needed for conversion to classification is a buy criterion. If the prediction is greater than x cents, then buy; contrarily, do nothing. The downside is that the training optimizer will only be concerned with minimizing prediction error on real-valued outputs, not buying decisions. The upside is that it allows usage of conventional methods such as linear regression, ridge regression, support vector regression (SVR), and neural networks. The strategy of leveraging regression systems for a classification task is considered in this work.

1.6 Objectives

2.1 Models

All systems are either language models or time-series models. Models predict using input from two representations, namely, text and financial time-series. Language models try to classify stock movement given the text within tweets. Language models are trained and tested on tweets. Language models enable the extraction of information from textual data, rather than numerical data. Language models are constructed for general natural language processing. For time-series, the task is not to compute long-term forecasts on a single stock. Thus, we do not consider traditional time-series forecasting designs such as autoregressive moving averages or stochastic models. Instead, the task compares machine learning techniques to mine data for patterns in events. Traditional models are applied: linear regression, kernel ridge regression (KRR), and SVR. Furthermore, with the recent trends of deep learning, we create systems using recurrent neural network (RNN) architectures.

2.2 Model Ensemble

Regression predicts real-valued outputs. Thus, a natural solution applies weighted averages between models. The approach scales with increasing predictors. Weights are computed with the goal to raise annualized Sharpe ratios. The validation set is used to approximate the ideal weights.

Models that individually perform better are given a higher weight in an ensemble prediction. Therefore, a weighted arithmetic mean of Sharpe ratios determines the influence of individuals in a combined model directly. As previously mentioned, the metrics of interest are accuracy, precision, recall, and financial statistics. These must all be calculated on trading decisions, not on regression outputs. Thus, even though regression models output a numerical value, they are then converted to a labeled class (BUY/NEUTRAL/SHORT). Classification models already predict those categories. Since classification tasks predict labels, it is simplest to take the most common predicted class when building an ensemble. This is the strategy used on language models. If multiple learning algorithms conflict in a tie, then it is safest to select NEUTRAL.

2.3 The Process

Assigning the financial security associated with a tweet is straightforward when the symbol is in the text. As such, we store tweets that single out companies using Twitter’s “$” character. The list of steps required to study price influence of events for companies mentioned in tweets include the following:

1. Track and store all tweets mentioning company-centric stock symbols.

2. Provide a method to define “new event” from a cluster of similar tweets.

3. Develop a procedure to calculate a numerical event matrix that aligns events with financial time-series.

4. Construct language models from tweets that best classify price spikes.

5. Construct a temporal model that classifies large price movements using share prices.

6. Measure the performance of language, temporal, and ensemble models on the validation and testing data.

7. Use classified outputs from prediction model as buy/sell trading signals.

8. Evaluate trading metrics (returns, Sharpe ratio, and profit per share).

A significant concern for predictive financial models is overfitting, i.e. the lack of ability to generalize. We alleviate the suspicion above to some extent by holding out large sections of our dataset for validation and testing. Supervised learning algorithms learn by seeing examples from the training set, then predict on unseen data. The dataset partition consists of training, validation, and testing set. Choosing two longer-term hold-out periods is necessary to avoid bias from short-term market conditions. The validation set, while being unseen by the learning model, allows parameter tuning. Intraday trading ensures a considerable volume of data, enabling to partition higher percentages. Unlike other applications, finance does not grant shuffling of data due to its sequential nature. Therefore, the sets are arranged in a preserved order of trading days. In addition, partitioning using trading days avoids having data from the same intraday period between two sets. The dates used for partitions are described next.

2.4 Data

There are two forms of collected data, namely, tweets and stock quotes. The choice of trading frequency determines the level of detail in constructing an event study. Tweets measure time stamps to the second. Therefore, we may choose a trading rate of either minutes or days. Since intraday trading provides a larger volume of data, we elect minutes. The Twitter API is used to retrieve tweets. MongoDB is then used to store tweets for analysis. This section provides a summary of curated data.

2.5 Event Study Horizon

The horizon is established to measure the effect Twitter events have on prices. The relevant definitions are the event time, pre-event region, and post-event region. As suggested, intraday impacts are of interest. Because of this, the event time can be any minute during market hours that produces a complete post-event region. To compute the financial metrics, the entry and exit of trades must be clear. A simple exit strategy of closing a position after a set amount of time works well with event studies. The approach is to examine exit strategies of 60 and 240 min. As such, given time of event T, the post-even regions will be (T, T+60], (T, T+240]. These intraday regions help differentiate how long the influence remains on stock prices.

Time-series models use the pre-event region for predicting movements. A larger time span grants more input data for the forecasts. However, it reduces the number of events possible due to missing data. Since the markets open at 9:30 am EST, if an input requires 60 min of financial quotes, then it will not allow a trade before 10:30 am EST. Accordingly, a 30-min selection defines an area of [T−30, T).

2.6 Evaluation Metrics

The selected evaluation metrics are price spike accuracy (with a focus on precision/recall), and trading metrics. Directional accuracy, representing a binary classification with UP/DOWN as classes, is a consistent choice amongst researchers because it displays the algorithms’ ability to predict price direction. It is less important by industry standards since trading only on price direction can lead to financial losses. If incurred transaction costs are higher than stock gains, then the overall result is negative. Incurring high transaction costs is a common pitfall in designing trading strategies. The binary classification of UP/DOWN does not follow trading logic – mostly it is best to make no trade at all.

Predicting price spikes can have considerable benefits. Let us define a price spike as stock movement greater than —x—% within time t. We can empirically find the value for x. It need not be symmetric, in such a case one can use two positive values x and y. The values must be large enough to offset transaction costs but small enough to find enough events. To view this as a classification problem, we define the three classes of POSITIVE: +x% movement in time t; NEUTRAL: [−y, x]% movement in time t; and NEGATIVE: −y% movement in time t. Given this setup, we can train a prediction classifier that works well with classical buy/sell trading logic: purchase the stock when the classifier predicts POSITIVE; short the stock when the prediction is NEGATIVE; and do nothing for a NEUTRAL prediction. Lastly, the simplest exit strategy is to close the trading position after time t.

Evaluation of the results uses the precision and recall [4] metrics. For the price spike classification task, we can consider a one versus all strategy. When measuring precision for the POSITIVE spike class, both NEUTRAL and NEGATIVE would be the incorrect categories. For this work, the reported values for precision and recall are computed using weighted averages from the POSITIVE and NEGATIVE classes. This stays consistent with providing market-neutral trading decisions. The emphasis on precision and recall stems from trading logic. Trading does not have a huge penalty for missing out on buying opportunities. Profit and loss are only affected by precision. It is best to tune model parameters to maximize precision. The only consideration needed is to increase recall if there are not enough trading opportunities. We expect to have low levels of recall since we do not want to trade on all news events.

The final evaluation criteria are achieved by computing trading metrics. The following industry standards are used as metrics: returns, Sharpe ratio, and profits per share. Sharpe ratio [19] is defined as Sharpe ratio = (r_p−R_f)/σ_p, where r_p is return of the portfolio, R_f is return of risk-free rate, and σ_p is standard deviation of portfolio. Sharpe ratio is a better metric than returns because it measures returns with respect to risk. As such, a strategy that produces 8% returns with very low volatility is much better than one with high fluctuations. The goal is to attain positive Sharpe ratios. In general, a strategy with a Sharpe ratio above 3 will be profitable daily [3]. Due to the low-risk levels, strategies with lower average returns but larger Sharpe ratios can be leveraged to maximize profit. An important task is to ensure that all values of risk and returns are annualized. With current economic conditions, the risk-free rate can be ignored.

Commission costs are often calculated on a per share basis. For example, a brokerage can charge a fee of 0.0075 cents per share on a trade. For this reason, we highlight the average profit per share in cents for our prediction algorithm, to provide portfolio managers with a better view of how to associate their fee structure. An emphasis on precision and recall can aid in optimizing prediction algorithms for finance.

Figure 2:

Flow Chart of System Overview.

2.7 Overview Diagram

Figure 2 presents the system design, visualizing the procedure for constructing the proposed event study. The overview flows through the full procedure, starting from storing tweets to the evaluation of trading decisions. A later section describes with detail all components from the system overview. The system overview highlights the following components: data retrieval, data storage, cluster formation, model building, and trading metrics. Prior to training, the data needed for retrieval are deduced. The two components of data are financial time-series and tweets. Tweets are retrieved using the Twitter API, while financial data are gathered using QuantQuote (financial data provider). The retrieved data are stored in MongoDB databases. Once all data are stored, clusters are formed as described. Clustered events and market data are converted to a format suitable for classification tasks. The outputs from learning algorithms are ensembled and used as trading decisions.

3.1 Model Type: Language Models

The purpose of the language model is to predict a price spike given a tweet. Since features are textual, there are no issues with missing values. Two approaches are attempted for model selection. First, staying consistent with past studies, a bag-of-words style approach where words are symbolic is used for features. Second, more recent models are tested using words converted to numerical vectors. For a symbolic word feature set, NBSVM is applied [23]. For distributed representations, the following models are considered: sentence/paragraph vector model [8] and a RNN language model (RNNLM) [13]. These model implementations were adapted from available code [6]. Most of the core implementation, such as preprocessing techniques to normalize text, was left exactly as is. The main design change was to go from a binary classification task to a multi-class classification with three labels. Two solutions are available: convert to a multi-class classification task or use a one-vs-all classification strategy.

A multi-class solution reduces the number of samples of each category in the training set. The one-vs-all method can perform a generative maneuvering where instead of one model we create two, namely, a positive for buy decisions and a negative for shorting decisions. In both models, the opposing class will be a neutral (do nothing) label. Table 2 lists all potential cases to combine predictions of the generative models. The risk on returns is reduced by diffusing the predictions of both “buy” and “short” to neutral. The text in such a scenario could have high volatility conditions and, therefore, is avoided.

3.2 Event Clusters

One tweet may not satisfy the requirements of being noteworthy. However, a burst of tweets on the same topic would assert likelihood. The following algorithm is put forward to cluster events.

1. For incoming tweets, tag text using part-of-speech (POS) tagger, where words are separated using a highly developed POS tagger [15].

2. Compare tagged common nouns, URLs, proper nouns, and hashtags with prior events. If no such event has occurred in the past T minutes, create event ID and map POS tags to a new event. If POS tags match with past events’ tags, then append tweet to closest old event. Do not use stock symbol in matching criteria.

3. Close an event if it does not have a new tweet appended to it within T minutes.

4. Save an event as noteworthy if the required number of tweets cluster within T minutes.

5. Once an event is created, save it for reference in merging future indistinguishable events.

6. Mark an event as lagged if it shares high similarity with an older saved event.

7. Convert clustered event time to a tradeable market time.

8. For each clustered event, send time stamp to market data database to connect financial data. Use the stock representation of the company mentioned using the “$”. Use Step 8 to label price spikes as BUY/NEUTRAL/SHORT.

The suggested algorithm for clustering events has parameters. During the training period, the parameters may be empirically adjusted to improve performance. However, the event clustering algorithm is the most computationally expensive algorithm in this work. An extensive amount of parameter tuning could further improve results. The clustering method needs to identify events early enough to counter the pre-news effect. In addition, merging handles lagged news by determining whether late tweets have an additional impact. Thus, the current parameter set is an initial solution for event clustering. Table 3 specifies crucial parameters.

Once the cluster reaches the accepted size, the time stamp of the last tweet is used. We convert this time stamp to an acceptable market time for trading. Since Twitter time stamps appear at a second frequency, we round up to the nearest minute. It is important not to round down, as it would have a look ahead bias of seconds. Rounding up to the next minute does not solve the scenario where exchange hours are not open. To deal with this, any event that comes during outside hours takes the next available market time. This strategy puts to use all newsworthy events leading up to the market opening at 9:30 am EST. The lagged news effect is dealt with by storing any noteworthy event for 4 h, purging any incoming event with a strong resemblance. A primary benefit of using the enumerated method is the ability to label tweets automatically. It is essential for supervised learning.

3.3 Event Matrix

An event matrix is designed to align the time stamp of an event with its stock symbol. Event matrices are utilized by both the time-series model and for trading metrics. Table 4 uses a made-up example to best describe an event matrix for a time-series model. The “E” inside the matrix identifies a clustered tweet event. The time-series model will use the information to know when and what stock to examine. One tweak is needed for trading metrics. The values inside the matrix now correspond to output labels, where 1 is BUY, 0 is NEUTRAL, and −1 is SHORT. The event matrix in Table 5 can be used to make buy/sell trading decisions. The hypothetical trades in the example are as follows: buy CVX at 11:31, short XOM at 11:32, and short SLB at 11:39.

3.4 Model Type: Time-Series Models

Regression systems use past 30 min of price movements to predict future shifts. We convert the price change prediction to trading logic. For example, if a linear regression system predicts a price increase of 50 cents, then buy if the buy criterion is any move larger than 30 cents. Essentially, while the learning algorithm is designed for regression, the output is classification.

Classical techniques such as linear regression, SVR, and KRR use default parameters. Neural networks, however, have infinite possibilities for architectures. To stay within reason, we have focused on one feedforward (FWD) neural network and three recurrent neural networks. The reason for multiple structures is to compare the gated recurrent unit (GRU) versus the long short-term memory (LSTM) unit. Model names for neural networks are labeled in a way to highlight architecture. The FWD neural network, FWD 0 RNN 6 Dense, has zero RNN layers and six fully connected hidden layers. RNN 1 GRU 1 Dense has one GRU layer and one fully connected hidden layer. RNN 4 LSTM 1 Dense has four LSTM layers and one fully connected hidden layer. Lastly, RNN7 LSTM 0 Dense has seven LSTM layers and zero fully connected hidden layers. Every neural network has a single neuron as its output layer. The output layer does not have an activation function since the neural networks learn to predict values for price shifts. Training enhances these predictions by minimizing the mean squared error function. Once complete, the projection is converted to class labels using BUY/SHORT criteria. Based on the overall strategy, the system attempts to answer the question “Is the stock primed for a large movement?”

4.1 Analysis of Twitter Dataset

Before profit results, we analyze the stored collection and highlight some helpful techniques found to reduce useless events. A filter is necessary because Twitter is full of casual conversations. The event clustering algorithm moves towards using Twitter in a manner similar to those of news headlines. We begin by not including retweets, and in doing so, all saved tweets are unique and independently produced. The number of retweets is unknown because the storing routine rejects them immediately. Using the clustering method, the total number of original tweets stored for the company-centric collection is 1,557,811. Since language has grammatical peculiarities, the focus is only on English tweets. Removing retweets aids event clustering by avoiding the arrival of identical texts. Unfortunately, a similar concern arises given the pervasiveness of automated bots on Twitter. Luckily, the Twitter API provides an origination of source for each tweet. A source tag registers if a tweet was formed using an iPhone app, Blackberry app, any third-party app, or the Twitter website itself. By examining sources, a filter is built to block automated tweets.

In viewing preliminary events, few sources appear to dominate as candidates for presenting repetitive texts. A blacklist containing sources that exhibit such behavior is designed to mitigate the problem. The list is by no means exhaustive; however, it is an essential step for the event clustering algorithm. Either blacklisting sources or accepting trusted sources is our recommendation to move towards identifying noteworthy news on Twitter. Table 6 states figures for filtered documents, i.e. English filter and blacklisted source filter. Approximately, half of the tweets are filtered from grouping into events. This high percentage shows the influence that bots have on Twitter.

4.2 Analysis of Event Collections

Presented here are the results after applying the event clustering algorithm on filtered documents. The numbers change as the exit strategy parameter is modified. Currently, a 60-min exit strategy is used. An examination of which exit strategy works best is presented later. Total counts on the number of events included 35,453 total events with training size of 19,788, validation size of 7352, and testing size of 8313. As mentioned, large percentages of events are prevalent in the validation and testing sets. This stays consistent in ensuring fairness to the concern of overfitting in finance.

If a cluster is identified outside of trading hours, we can either ignore it or trade it at market open. A large proportion of clustered events, 69%, is at market open. Thus, the decision is to use events at market open. There are no events after 3:00 pm EST, because we are using a 60-min holding period. All events that need to hold the stock for a longer period will be a subset of either collection. Events at market open are not available for time-series models since they require a 30-min pre-event region.

Similar logic applies to trading days. The strategy is to ignore all evening events on Friday while keeping all weekend (Saturday and Sunday) tweets. Including weekends increases the number of events on Mondays. The decisions described here are the same choices traders must consider when incorporating morning and weekend news. The effect of news from outside of trading hours may explain why the largest price movements in a trading day happen at market open. A significant number of events have been merged and classified as lagged events. Each event is saved for 4 h. If any new event has a high similarity to a past event, it is considered lagged. A total of 32,258 events are merged; meaning 47.6% of all company-centric events are merged.

With additional information provided by the Twitter API, analysis can go beyond just the text of tweets. Hashtags are used as topical indicators, as they give a quick look at top categories clustered by the algorithm. The top hashtags are considered relevant.

4.3 Performance of Event Collections

Annualizing Sharpe ratios is a matter of scaling. If we have monthly returns, then we would multiply the Sharpe ratio by the square root of 12. Daily returns would scale by the square root of 252 and not 365, because there are 252 trading days in a year. In using a 1-h holding period, we may be tempted to multiply 252 by the number of hours in a trading day. However, this should not be the case because the strategies do not provide opportunities to trade every single hour. Instead, the learning algorithms do present enough events to achieve these returns at least once a day. Thus, we scale returns daily to annual rather than hourly to annual. To allow scaling of Sharpe ratios, an underlying assumption is that the returns are independent and identically distributed. A study suggests an overestimation of Sharpe ratios when the assumption does not hold [10]. Overstated Sharpe ratios stem from the presence of serial correlation in returns [10].

If at 9:30 EST there are five events for a company where four are buys and one short, then select buy. Doing so removes the serial correlation that would arise from having same returns and removes the one differing trading decision. Now there is less likelihood for serial correlation in returns, and thus, Sharpe ratios will be annualized.

An issue occurs when multiple stock symbols show up in a tweet. If “$AAPL” and “$FB” are listed, then we must choose to accept either both symbols, only one, or none. Multiple events with identical text are generated if all symbols are used. As an initial solution, we only choose the first symbol alphabetically.

4.4 Exit Strategy for Language Models

A difficult decision for traders is to figure out when to exit a trade. The experiments measure performance on intraday holding periods of 60 and 240 min. Figures 3 and 4 show Sharpe ratios and the number of trades for each language model. Figures include ensembles that form by using the most common prediction from individual models. Given the doubt of serial correlation in the company-centric collection, a conversion is made to an annualized Sharpe ratio.

Figure 3:

Performance of Language Models on 60-Min Exit Strategy.

Figure 4:

Performance of Language Models on 240-Min Exit Strategy.

The company-centric collection does well and is mostly consistent. Two of the most promising models are language models: NBSVM and PARAGRAPH. The decisions used from these systems would present meaningful gains for day traders over the 2-month period. Sharpe ratios give a general feeling of trading strategies profitability.

The PARAGRAPH model is based on unsupervised learning of distributed vector representations for texts, varying from a sentence to an entire paragraph. The vector representation of model is trained to predict surrounding words in a paragraph. Based on empirical results, the paragraph model is a good technique for text representation [8].

With monthly or yearly holding periods, a Sharpe ratio of greater than 3 is considered excellent. Shorter-term and high-frequency traders prefer higher Sharpe ratios. Due to smaller returns, a larger amount of capital is needed to invest in high-frequency positions. Since a significant capital investment is required, lower risk opportunities are adequate.

Trade counts, greater than or close to 500, boost confidence in the results as it ensures that no single outcome dominates averages. The only ensembles that have a small number of trades are pairs that include RNNLM; this shows that there is a limited union of decisions between RNNLM and other language models. The displayed results fit the requirements of short-term traders noted above.

Figure 5:

Performance of Regression Models on 60-Min Exit Strategy.

Figure 6:

Performance of Regression Models on 240-Min Exit Strategy.

4.5 Exit Strategy for Regression Models

Regression performances across exit strategies are displayed in Figures 5 and 6, including the KRR. Sharpe ratios ranging from −2 to 3 fail to impress when compared to language models. A lower range may be partly due to being evaluated on a small subset of the data that language models trade on. Regression models forecast price shifts based on the past 30 min of movements. A significant portion of cases appear at market open. Therefore, less than 30% have the pre-event region required for projection. Even with a smaller training set, time-series models mostly learn to be profitable.

Time-series predictors, unlike language models, show clear differences among exit strategies. Longer periods of 240 min show negative Sharpe ratios and inconsistencies between the validation and testing set. The 60-min holding period seems like the best choice for regression models. It is reasonable to expect shorter term forecasts to be more accurate. On good decisions, allowing a trade more time aids the ability to accumulate greater returns. Since Sharpe ratios are a function of both risk and returns, a good balance helps. When comparing models, there is not one that outperforms all others across the given exit strategies. A closer examination of the performance of regression models is required. Both language models and regression models perform well on a 60-min holding period. For this reason, we continue beyond Sharpe ratios and later highlight financial metrics using the 60-min exit strategy.

Figure 7:

Percentage Returns Compared Using a 60-Min Exit Strategy on Language Models.

4.6 Evaluation Metrics

A Sharpe ratio acts as a sound indication to accept or reject a trading strategy. Still, a closer inspection of additional metrics is advised. This section considers the distribution of stock returns, profits per share, and machine learning metrics of precision, recall, and accuracy. Figure 7 helps in understanding the distribution of percentage returns; presented is a box plot of returns for language models. A box plot highlights the mean, quantiles, and outliers from all trades. The mean and majority of predictions are above 0% for NBSVM and PARAGRAPH. Furthermore, most outliers have large positive gains, and regardless of the high Sharpe ratios, it should be noted that most of these outliers are positive. Sharpe ratios are discouraged by risk in either direction. Investors, on the other hand, want notable positive gains. For this reason, some traders prefer to use Sortino ratios which only adjust for downside risk. The distribution of returns further persuades investors in using the event clustering algorithm and language models. Comparing the box plot of returns and Sharpe ratios infers more meaningful interpretations.

By using a histogram in the range of −5% to 5%, we can capture the distribution of where most movements appear. Figure 8 shows how a profitable model should look, while Figure 9 displays a model that would lead to losses. There is a natural skew to the right for both NBSVM and PARAGRAPH along with a large number of profitable trades. It is also clear that the majority of returns have a mean close to 0.6%. The average may seem small, but these are accumulated under just 60 min and not annualized. The advantage of a shorter holding period is the ability to free up capital. A 0.6% return can be absorbed and reinvested to give compounding effects. Tables 7–10 report evaluation metrics for language and regression models on the validation and testing sets. On the evaluation metrics alone, text models undoubtedly outperform time-series models. Still, it is important to compare them separately since both are trading on different subsets of the data.

Figure 8:

PARAGRAPH Histogram of Percentage Returns Focused on Range (−5%, 5%).

Figure 9:

RNNLM Histogram of Percentage Returns Focused on Range (−5%, 5%).

The variance in the price an asset trades at explains why some strategies have a negative profit per share but positive returns. For example, on the validation set, SVR loses an average of −0.04 cents per trade but has an average of 0.19% returns. SVR made a few bad predictions on stocks that trade at higher prices; this could be a large decline in cents but a small ratio according to percentage return. Nonetheless, the current regression techniques learn to be profitable with the selected representation. Only RNN 4 LSTM1 Dense model achieves a Sharpe ratio of over 2 across both validation and testing set.

4.7 Ensembles of Time-Series Models

Model combinations and weights are selected using the method described earlier. Listed in Table 11 are the top 10 performers based on Sharpe ratio on the testing set. As shown in Tables 12 and 13, all the recorded ensembles have Sharpe ratios greater than 2. Of the 10, seven use RNN 4 LSTM 1 Dense. Three out of the seven, 1, 2, and 4, perform better as a combination than the original.

4.8 Summary

The figures and tables provided in the paper enable further investigation of the use of the proposed models, as the performance of regression models is evaluated versus the performance of language models. The metrics of Sharpe ratio and trace count – typically used in this area – are utilized for the assessments. In addition to the models, the exit strategy is an important factor to be considered. The time-series predictors are dependent on exit strategies, and the data presented in figures and tables depict the evaluation of 60-min exit strategy versus 240-min exit strategy, with the 60-min exit strategy performing better for the regression models. As models are ensembled, the performance of certain ensembles is included for further analysis.

The event clustering algorithm does well in generating events. Both language and regression models have positive Sharpe ratios. Language models are better predictors and more profitable than regression models. Language models have a larger training set available for learning since they do not require financial data for predictions. Performance on regression models is strengthened by taking advantage of ensembles. Table 14 contains a list of recommended models along with their reported Sharpe ratios. Recommendations reflect evaluation metrics and the distribution of returns. Lastly, language models are favored. The preference is not only due to greater Sharpe ratios but also because performance is reported over a larger sample of trading decisions. Reporting performance on many trades lifts belief in the model’s ability to stay persistent.

5.1 Discussion

Journalists scour through dealings and stories to find noteworthy news headlines. The implemented event clustering algorithm sequentially assembles tweets with similar text, creating topical events as clusters. Only gathering related tweets under specific time restrictions helps identify developing stories. Furthermore, filters reduce junk by removing retweets and tweets from some automated bots. A keyword-based similarity clustering technique used on a total of 1,557,811 tweets forms 35,453 events. As such, the clustering technique is useful in generating Twitter events that resemble news headlines.

The majority of generated events have an associated time stamp of 9:30 am EST when financial markets open. Therefore, prices have not had a chance to incorporate any information found in events. By making decisions before trading hours, we avoid pre-news concerns.

To address lagged news, we have used an approach of merging events. We purge all matching clusters that arrive within a 4-h time frame of a saved event. For instance, in this study, there are 32,258 cases merged, which is important considering a total of only 35,453 events created. A key benefit of merging is to avoid repeated purchases on a single story. Multiple transactions would have caused serial correlations in returns, which were previously shown to overstate Sharpe ratios. Furthermore, if the merged event arrives a few hours late, the price influence may have already been exhausted.

Therefore, seven models have been recommended. Recommendations are listed in Table 14. Advised models have Sharpe ratios ranging from 2.1 to 7.6 for the validation set and 3.1 to 11.7 on the testing set. Text models are preferred over price-based models.

5.2 Contributions

In summary, the research generates events from social content, predicts trading decisions on those events, and evaluates learning algorithms to produce recommendations for trading strategies. New media of delivering information to the public continue to emerge, and in order to adapt, event analysis and computational finance need to promote the generation of events from unstructured data. Most questions answered previously looked at structured data for events, such as earnings announcements. Now techniques to identify entities and cluster-related events from casual conversation can move the research forward to readjust to current and future social media content. Furthermore, machine formed actionable trading decisions may improve as natural language processing and time-series classification techniques advance.