Media Hack Day: Developing social graph analysis tool

On March 22-23 Axel Springer hosted the second Media Hack Day in Berlin. Our team – me and software engineer Artem Pryakhin – won the Storyful API price after creating SocialViz, a social graph tool showing connections between Storyful users. Above you can see how it looks like (it’s obviously interactive in the real version). And here’s the code just to get a feeling of it:


So the first question is: why do we need that? Well, because data is the new currency. Combined with the social aspect, it becomes even more important for media companies. It’s about how ideas are spread and how user networks come to life. But media really don’t know who their users are and how they are connected. Every user is just a blank face for them. Data connected with the social aspect is something that media companies haven’t really harnessed yet. To get useful insights about your users, you need to be able to quantify and visualize flow of ideas, rate of engagement and connections between people in real time.

That’s where our social graph data analysis tool comes into play. We give users a face and put them into the context of their relationship with other users. It helps discover hidden connections.

[slideshare id=33379099&style=border:1px solid #CCC; border-width:1px 1px 0; margin-bottom:5px; max-width: 100%;&sc=no]

So we developed a small prototype based on the Storyful API. We wanted to give users a face. Our social graph tool analyses hidden connections between users whose mobile videos have been used in the reporting and highlight user networks depending on the topic and region.

After 2 days of hacking

After 2 days of hacking

You just click on the node, which stands for a Twitter user, and see his or her connections highlighted. Thus you can see how information is spread across users’ networks.

Here’s a short description of the project.

Basic concept

Social graph is a visual web and mobile tool based on the Storyful API which helps you analyze hidden connections between the users whose mobile videos have been used in the reporting. That will help to explore hidden connection, verify information and explore geographic, demographic and thematic information distribution patterns. It can be further expanded to be used by any media organization to analyze its users and identify how they are related and who the real opinion leaders are.


There are basically two problems we try to solve:

1) Storyful employs various verification techniques to find out if the users whose mobile videos are used, but never knows how users are related to each other and if they are just duplicating each other’s content.
2) Media don’t really know their users, what they are talking about, where they are located and how they move around, what inspires and upsets them and how they are connected to each other.


Social graph is an internal visual analysis tool which can be employed by the Storyful reporters to analyze their data. It can be also applied by other media outlets to analyze hidden connections between users who have logged in with their Twitter account.


Our social graph tool plugs into the Storyful API data and creates a social graph visualization based on the topic, region or separate user names. This happens by analyzing the Twitter friend lists, location data and keywords through the Twitter API. It provides analysis of hidden connections between the users (content providers): Storyful reporters can immediately discover which users follow each other (followers, followers of the followers, etc), how they are connected through their first-, second-, or third-degree followers and what their networks look like.

With Adam Thomas from Storyful - the API prize is ours!

With Adam Thomas (@datatheism) from Storyful – the API prize is ours!

Scalability and further development

Visualization of the users’ social graphs will help media understand who actually uses their websites and identify opinion leaders who can then be addressed directly. A further step would be sentiment analysis of the users’ tweets to gain deeper insights into opinion distribution patterns.

There’re other positive effects for Storyful reporters using the tool in the further iteration process:

1. Visualization of video distribution patterns
2. Dashboard that shows which regions and topics are undercovered (according to the distribution of sources).
3. Build clusters of interconnected users and create statistics according to the regional, demographic and thematic distribution of its users.
4. A positive side effect is the creation of a more complete user profile for the internal use.
5. Scaling out the framework horizontally to be able to process terabytes of data.

Any media company can use the tool to analyze its users who have logged in with Twitter. By doing that, they will be able to finally learn what its audience is like: where it lives, what it is really interested in and how different users are interconnected. By doing that, media companies could finally identify opinion leaders with most relevant connections and address these key users. This would help establish the missing link between the newsrooms and the audience.


Leave a Reply

Your email address will not be published. Required fields are marked *