Consuming tweet stream:
Need to consume public streams from twitter using their streaming API.
Twitter has provided an http client to listen to the streaming API.
The SampleStreamExample is good enough for the first cut. I just added a properties file instead of giving the credentials on the commandline.
Now we have to parse the JSON streamed by the above code. Or it turns out we can index json docs directly in elasticsearch.
Indexing tweets in elasticsearch:
Preliminary tests in elasticsearch.
Java client for elasticsearch:
There seem to be a number of java clients available to talk to elastic search. Though the native client is highly recommended, on first look, I like the Jest client.