Spidering scripts for opinion monitoring

With the advent of Web 2.0 technologies, online forms of communication are rich sources of data to study socio-economic growth patterns and consumer behaviours. In this research field, the more robust development of data mining and opinion monitoring depends on fully automating data collection to monitor the evolution of customer opinions and preferences in real time. Although web crawlers or spiders can assist researchers in an innovative and effective way, this data collection approach could give rise to ethical concerns on the cost of web crawling processes and on data protection and privacy. With a focus on opinion monitoring, the chapter aims to discuss the ethical and legal issues of data mining in relation to spidering scripts. This contribution proposes a detailed analysis of the ethical and legal aspects of online data collection by comparing existing legislations. For illustrative purposes, a spidering software is presented to discuss its potential and explore ethical solutions in the data-mining sphere.