Stock Sentiment Analysis
Natural Language Processing
In the Magic Investment Mix App blog, I talked about how to use API information about the stock market to calculate the optimum portfolio mix. In the Emotion Detection in Text blog, I went through sentiment analysis for text data. This post combines the two to analyze text data to get the sentiment on specific stocks. The below example is on Apple’s stock.
YouTube Tutorial
The video that goes through the above in streaming format is below.
The code for this use the requests, pandas, and textblob packages in Python. The requests package is used for API requests. The pandas package is used for data manipulation. The textblob package is used for performing sentiment analysis on the extracted text feeds from API.
Once the packages are imported, the below code identifies the api_key, information of interest that is held in the API, ticker of a stock, as well as the appropriate url based on these information.
The below code then uses the get request to fetch news data from the Alpha Vantage API and converts the fetched data to json format.
The below code extracts the news feed from the news_sentiment function in the API above and stores it in a data frame.
From the above data frame, title and summary of the news feeds are separately extracted with handling of any missing values.
With the above data frames ready, the below code defines the function to perform the sentiment analysis and apply it on the news feed summaries.
Finally, the summary and sentiment are displayed with the title of the news feeds.
You Look Like a Thing and I Love You
In 2017 Wired published an article whose authors analyzed 92 million comments on more than 7000 internet forums. They concluded that the place in the US with the most toxic commenters was, somewhat surprisingly, Vermont.
How could this have happened? The common thread seems to be that if data comes from humans, it will have bias in it.
Studies have shown that algorithms learn less pleasant associations for traditionally African American names than for traditionally European American names. They also learn from the internet that female words like she, her, woman, and daughter are more associated with arts-related words like poetry, dance, and literature than with math-related words like he, him, and son. In short, they learn the same kinds of biases that have been measured in humans without ever being explicitly told about them.
The AI that thought humans were rating Mexican restaurants badly had probably learned from internet articles and posts that associated the word Mexican with words like illegal.