Discovering User Opinions on Automation in Social Media: The SPY Tool

Mr Nikos Dimakopoulos

Project Manager

The fast growth of social media has dramatically transformed the interaction and communication of individuals on the Web and throughout the real world, which raised the need for advance business intelligence (BI) and advance analytics techniques to quantify in a scalable way the information coming from social media posts.  

This evolution of social media has bloomed in a plethora of social media users and applications that produced innovative and efficient intelligent techniques for data processing in fields like pattern recognition, information fusion, knowledge discovery and data visualization, making it easier to exploit for research and business purposes.

Towards this direction and following the market needs for social media data understanding, by eliciting insights and trends on specific research topics, within the SHOW Project, which is co-funded by the EU under the H2020 Research and Innovation Programme (grant agreement No 875530), ITML launched the innovative prototype namely the SPY (another Social media Periscope for You) tool for social media monitoring and analysis in the context of defining the ecosystem around automated urban mobility, based on the use cases of the SHOW project. The aim of the tool is to:

  • Collect opinions on automated mobility and automated vehicles (AVs) from different Social Media platforms.
  • Analyse the collected opinions.
  • Define and depict the level of the Social Media user’s acceptance of automated vehicles and in extension, extract insights into the envisioned shared, connected, electrified fleets of AVs in coordinated public transport, DRT, MaaS and LaaS operational chains.

For this to be achieved, the SPY tool consists of 4 main subcomponents for data mining, sentiment analysis, data classification and clustering & visualization, including technologies such as Named Entity Recognition (NER) – for Data mining, Bidirectional Encoder Representations from Transformers model (BERT) – for Sentiment analysis, Latent Dirichlet Allocation (LDA) – for Data classification/clustering and ElasticSearch Kibana toolkit – for dashboards with charts, interactive controls, markdown, and more.

The results of this social media data mining, aggregation, and analysis are presented at  as a mini dashboard with a set of interactive visualisations per analysed topic (challenges, road safety, public transport and logistics), including a Word Cloud graph that depicts the Top 6 most frequent words that appear on Social Media in the context of AVs, a sentiments’ pie chart that depicts the ratio of the collected positive and negative Social Media posts and a posts timeline chart of collected posts for a given time period.

This project has received funding from the European Union’s Horizon 2020 research and innovation programme under grant agreement No 875530.