Project Overview: Sentiment Analysis of Academic Publications

This project focuses on performing sentiment analysis on academic publications, using data from a Scopus dataset (from mendeley.com). The objective is to extract and visualize the sentiment trends (positive, neutral, or negative) across various types of documents, years, and research topics.

Steps Involved:

1. Preprocessing: First, cleaned the text data from the titles by converting it to lowercase, removing special characters, and eliminating stop words. This preprocessing step ensures that only meaningful words remain for sentiment analysis.
2. Sentiment Analysis: Using the TextBlob library, applied sentiment analysis to the titles. Each title was assigned a polarity score (ranging from -1 for negative sentiment to +1 for positive sentiment), and based on these scores, titles were classified as positive, neutral, or negative.
3. Visualization: Several graphs were created to visualize the sentiment trends and insights from the dataset:

Sentiment Distribution by Year: This graph reveals how sentiment in academic publications has changed over the years. For example, we can see that neutral sentiment has dominated most years, but there has been a consistent presence of positive and negative sentiment in recent years. This could indicate that the academic discourse has been relatively balanced, though neutral perspectives are still the majority. It’s also worth mentioning that positive sentiment increases steadily.

Word Cloud for Positive Titles: The word cloud highlights the most frequently occurring terms in titles with positive sentiment. Words like “data,” “analytics,” and “decision-making” are prominent, indicating that topics related to data science, decision processes, and business analytics are often discussed in a positive light.

Word Cloud for Negative Titles: In contrast, the negative sentiment word cloud displays terms like “artificial intelligence,” “security,” and “network.” These words suggest that there may be critical or challenging discussions around AI, data security, and network systems, possibly related to ethical or technical issues in these areas.

Sentiment Score Trends Over Time: This line graph shows the change in average sentiment scores across the years. There is a noticeable spike around 2014, after which the average sentiment dipped and remained relatively stable with slight fluctuations. This could suggest an initial surge in optimism or excitement in the field followed by a more balanced or critical perspective in later years.

Sentiment by Document Type: This graph illustrates how different types of academic documents (e.g., books, articles, conference reviews) vary in sentiment. For example, conference reviews tend to have higher positive sentiment, while books and short surveys display more neutral or even slightly negative sentiment on average. This could be due to the nature of the content reviewed in each document type.

Heatmap of Sentiment Concentration by Year: The heatmap provides an overview of how sentiment is distributed across different years. For instance, it shows that 2021 had a higher concentration of neutral and negative publications, while more recent years show an uptick in positive sentiment. This gives a comprehensive snapshot of how the sentiment of academic discussions evolves over time.

Conclusion :

Throughout this analysis, there is a notable increase in positive sentiment within academic publications over time, particularly in recent years. This rise in positivity suggests growing optimism and confidence in the role of data science in strategic decision-making. As data-driven methods become more sophisticated and widely applied across business contexts, the academic community appears increasingly hopeful about their potential impact. Despite fluctuations, this trend toward positivity highlights the expanding role of data science in shaping future business strategies and decision-making processes.