Recently I have been playing with Mike Bostock’s D3.js and continue to be impressed with different examples of how people are visualising data. D3 provides an incredibly rich library, although with a relatively steep learning curve, the results can be fantastic.
I decided to mix the real-time data source of DataSift data with a volume/time graph example from the D3 library. I added a simple UI and real-time rendering using Node.js, Socket.io and Express to allow the user to enter their stream credentials. The finished example is as follows:
DataSift data can be exported from both recordings and historics in either JSON or CSV formats. Depending upon your filter definition, the fields within the resulting data will vary, therefore making importing the data in to a database such as MySQL a little time consuming.
The below PHP script automatically generates the CREATE TABLE command, along with the LOAD DATA INFILE command, allowing you to create a new table, and import the data quickly and easily.
Working at DataSift, I speak to many different clients and discuss their use cases regarding what type of data they would like to filter on, and include within their real-time data streams. I thought it would be useful to start capturing many of the common use cases in a single post and explain how these translate in to Curated Stream Definition Language (CSDL). If you are new to DataSift and or CSDL, the DataSift documentation is great and growing all the time, and this post should give some useful examples.
I will be adding to this post as and when I capture new examples. Please feel free to submit your own if you feel they will be beneficial to others.