Analysis of U.S. Stock Market Trends using LSTM and Ensemble Algorithms

Navigating the complexities of the U.S. stock market requires a deep understanding of its shifting trends. To explore this, we combined insights from technical and fundamental analyses. By employing cutting-edge tools like LSTM and ensemble learning techniques, and gathering data from diverse sources, we aimed to explain the intricate patterns of market behavior.

Project Description:

Objective and Goals:

We embarked on a deep dive into understanding deep learning and the various challenges associated with stock price prediction. Our primary goal was to forecast the adjusted closing price of U.S.-listed companies for the day following our latest data point.

Methodology:

Our approach intertwined both technical and fundamental analyses, crucial paradigms in stock prediction. The former analyzes historical price trends, while the latter emphasizes the company's intrinsic factors affecting stock value. We ventured beyond the bounds of traditional models by using deep learning, specifically the LSTM algorithm, renowned for its proficiency in time-series analysis. This model was then enhanced using a Stacking meta-model utilizing XGBoost, optimized for precision through GridSearchCV.

For data acquisition, we employed a mix of open-source libraries and web scraping, gathering everything from stock prices to granular financial statements. Post-data collection, a selection process was adopted to choose variables with a correlation coefficient greater than 0.9 with the adjusted closing prices.

Outcomes:
Our technical analysis model showcased the recurring issue of overfitting, even though the training was well-executed. Despite this, the model still achieved respectable predictive accuracy. Our fundamental analysis model, on the other hand, demonstrated lower performance. However, the crowning achievement was our Stacking meta-model, which amalgamated outputs from both the LSTM models, showcasing a significantly improved prediction rate, albeit with minor overfitting.

Limitations:

While the predictive accuracy was commendable, our meta-model lacked the finesse to predict the directional movement (bullish or bearish) of the stocks for the following day. Overfitting remains a persistent challenge that we aim to address in future iterations.

In essence, our project melds traditional stock analysis with modern deep learning techniques to create a robust stock price prediction model for U.S. companies, providing insights and pushing the boundaries of conventional methodologies