Updates on the SOCIAM Web Observatory

A core aspect in SOCIAM is observing and understanding social machines. In order to achieve this, the SOCIAM Web Observatory, which is part of a larger venture under the Web Science Trust network is developing the technical and analytical infrastructure in order to observe, analyse and understand the characteristics and evolution of social machines, at varying levels of granularity.
The last three months have been a very active time for the Web Observatory theme within SOCIAM, with a major ramp up in both technical infrastructure, available datasets, analytical frameworks and also visualisations tools. Starting at the foundations, the SOCIAM Web Observatory infrastructure has been developed and built from the bottom up; we are experimenting with the latest storage platforms and querying technologies in order to develop the most scalable, long term solution. We are currently experimenting with a variety of distributed platforms including distributed 4store, Hadoop, and Hbase, testing out their performance in terms of querying capabilities and scalability. We are currently tackling the issues and challenges that accompany ‘Big Data’ analytics, using large Web datasets (~9bn records) to test and benchmark our analytical capabilities.
Another line research currently underway is achieving interoperability between Web Observatories, with the ultimate goal of being able to link multiple installations together and query across different platforms and datasets, whilst being as technology agnostic as possible. As part of this, and in line with the current studies into the social machines of Zooniverse, the Southampton team has been working closely with the Oxford team to share and link between datasets. We are currently working on the technicalities of exposing datasets across sites, and discussing the governance and control of doing so, which is also an important component when building Web Observatories.
As part of our activities to engage with the wider research community, we have also been involved in a number of Web Observatory research exchanges including a recent trip to Shenzhen’s Tsinghua Graduate campus, where core members of SOCIAM lead a number of research challenges to develop analytical and visual applications on top of the datasets that we have currently made available. This was a great success, with three groups able to produce 9 different analyses and visualisations, which can be found at the Southampton Web Observatory. We are now going to be building upon this success and developing these further in future workshops.
In addition to this, we have also just had a paper accepted at the International World Wide Web conference (WWW2014) for our work describing a new methodology to analyse and understand the viewing behaviour of the Wikipedia social machine, which describes an approach to identify how page views of Wikipedia translated pages (i.e. English and Chinese) can be associated with users from different countries, and more interestingly, how these indicate cross-over of content and interests.
Keep checking back to find out how we are developing our techniques to observing social machines, and those that are interested in engaging with the Web Observatory community should think about submitting their work to the upcoming Building Web Observatories Workshop (B-WOW2014) at the Web Science 2014 conference. More details to come soon.