Sunday, January 27, 2013

[TUTORIAL] Getting tweets into Cinder


    Hmm...the notion of "getting something into Cinder" may be a bit of a misnomer, but then, you are pulling data into a framework/environment, so maybe it is.  Ah well, point being we're moving on from the previous installment wherein we walked through the steps required to build the twitcurl library so we could tweet in C++.  Now we need to actually use the darn thing, yeah?  So let's get to...err...Tweendering? (Cindweeting?  Cindereeting?  Sure, ok).  I'm assuming we all know how to use Tinderbox to setup a Cinder project, if not, just hit up <your cinder root>\tools\TinderBox.exe, it's pretty self-explanatory after that.    Here we gooo...

1.a) Once we've got an initial project tree, let's move some files and folders around to make setting up dependencies a bit simpler. Starting from <your cinder project root>, let's make our project tree look something like this (completely optional):

assets/
include/
  twitcurl.h
  oauthlib.h
  curl/
    (all the curl headers)
lib/ <-- add this folder manually
  libcurl.lib
  twitcurl.lib
resources/
src/
vc10/
  afont.ttf

Given this tree, setting up the rest of the dependencies for the project should be pretty straightforward.  I should point out that putting the font file directly into the vc10 folder is a bit of a hack and not at all the proper way to set up a Cinder resource, but for now I just want to get something functional.  Much respect to the Cinder team for their solution to cross-platform resource management though, I'll probably cover that once we start getting into building the final project.  Feel free to do some independent study, though, and check out the documentation on Assets & Resources in Cinder (and send me a pull request if you do!). 

1.b) So...let's code (and test out the new style sheet I wrote for syntax-highlighting)!  If you're interested in taking a peek at what the finished result might look like, check out the web version of Jer Thorp's tutorial, and if you're reading this Jer, no disrespect, I'm totally not meaning to steal your work for profit or some nefarious purpose, it's just a great, simple example that's super straightforward and easy to understand.  Had to get that off my chest, all credit where it's due.  If you haven't checked out the original tutorial, it goes (a little something) like this:

1) Do a twitter search for some term, we'll use "perceptualcomputing"
2) Split all the tweets up into individual words
3) Draw a word on screen every so often at a random location
4) Fade out a bit, rinse, repeat steps 3 and 4

1.c) Easy-peasy!  'Right, so first we need to get some credentials from twitter so we can access the API.  Not a hard process, just login to Twitter Developers, go to My Applications by hovering over your account icon on the upper-right, then click the Create a new application button, also on the upper-right.  Fill out all the info, then we'll need to grab a few values once the application page has been created.  The Consumer Key and Consumer Secret at the top of the page are the first two values we'll need, then we'll scroll down to the bottom of the page, click the Create Access Token button, and grab the Access Token and Access Token Secret values.  For now we'll just stick these in a text file somewhere for future reference.

1.d) Finally the moment we've all been waiting for, getting down with cpp (yeah you know m...ok, ok that's enough of that).  As with most C++ projects, we'll start with some includes and using directives:

#include <iostream>
#include "cinder/app/AppBasic.h"
#include "cinder/gl/gl.h"
#include "cinder/gl/TextureFont.h"
#include "cinder/Rand.h"
#include "cinder/Utilities.h"
#include "json/json.h"
#include "twitcurl.h"

using namespace ci;
using namespace ci::app;
using namespace std;

Outside of the normal Cinder includes, we'll be using Rand and TextureFont to draw our list of tweet words on screen, and we'll be using Utilities, twitcurl, and json to fetch, parse, and set up our twitter content for drawing.

1.e) Let's set up our app class next, should be no surprises here:

class TwitCurlTestApp : public AppBasic
{
public:
    //Optional for setting app size
    void prepareSettings(Settings* settings);

    void setup();
    void update();
    void draw();

    //We'll parse our twitter content into these
    vector<string> temp;
    vector<string> words;

    //For drawing our text
    gl::TextureFont::DrawOptions fontOpts;
    gl::TextureFontRef font;

    //One of dad's pet names for us
    twitCurl twit;
};

Ok, so I may have lied just a tiny bit.  If you're coming from the processing or openFrameworks lands, notice we need to do a little bit of setup before drawing text, but it's nothing daunting.  We'll see this a bit with Cinder as we get into more projects, there's a little more setup and it does require a little bit more C++ knowledge to grok completely, but it's nothing that should throw anyone with even just a little scripting experience.  That said, a little bit of C++ learnings can never hurt.

1.f) Time to implement functions!  If we're choosing to implement a prepareSettings() method, let's go ahead and clear that first.  For this tutorial, I'm going with a resolution of 1280x720, so:

void TwitCurlTestApp::prepareSettings(Settings* settings)
{
    settings->setWindowSize(1280, 720);
}

1.g) Onward!  Let's populate our setup() method now.  The first thing we'll want to do is setup our canvas and drawing resources, which means loading our font and setting some GL settings so our effect looks cool-ish:

gl::clear(Color(0, 0, 0));
gl::enableAlphaBlending(false);
font = gl::TextureFont::create(Font(loadFile("acmesa.TTF"), 16));

1.h) Now it's time to warm up the core, or I guess you could we could call it setting up our twitCurl object, so let's get out those Consumer and Access tokens and do something with them:

//Optional, i'm locked behind a corporate firewall, send help!
twit.setProxyServerIp(std::string("ip.ip.ip.ip"));
twit.setProxyServerPort(std::string("port"));

//Obviously we'll replace these strings
twit.getOAuth().setConsumerKey(std::string("Consumer Key"));
twit.getOAuth().setConsumerSecret(std::string("Consumer Secret"));
twit.getOAuth().setOAuthTokenKey(std::string("Token Key"));
twit.getOAuth().setOAuthTokenSecret(std::string("Token Secret"));

//We like Json, he's a cool guy, but we could've used XML too, FYI.
twit.setTwitterApiType(twitCurlTypes::eTwitCurlApiFormatJson);

Hopefully this all makes sense and goes over without a hitch.  Never a bad idea to scroll through everything and look for the telltale red squiggles, or if you're lazy like me, just hit the build button and wait for errors.



    Since we're only going to be polling twitter once in this demo, we'll do all of our twitter queries in the setup() method as well.  Let's take a look at the main block of code first, then we'll go through the major points:

if(twit.accountVerifyCredGet())
{
    twit.getLastWebResponse(resp);
    console() << resp << std::endl;
    if(twit.search(string("perceptualcomputing")))
    {
        twit.getLastWebResponse(resp);

        Json::Value root;
        Json::Reader json;
        bool parsed = json.parse(resp, root, false);

        if(!parsed)
        {
            console() << json.getFormattedErrorMessages() << endl;
        }
        else
        {
            const Json::Value results = root["results"];
            for(int i=0;i<results.size();++i)
            {
                temp.clear();
                const string content = results[i]["text"].asString();
                temp = split(content, ' ');
                words.insert(words.end(), temp.begin(), temp.end());
            }
        }
    }
}
else
{
    twit.getLastCurlError(resp);
    console() << resp << endl;
}

    This code should read pretty straightforward, there are really just a few ideas we need to be comfortable with to make sense of things:

1) Both Jsoncpp and twitcurl follow a similar paradigm (which pops up in a lot of places, truth be told) wherein we get a bool value back depending on the success or failure of the call.

2) The pattern for using twitcurl is a) make a twitter api call b) if successful, .getLastWebResponse(), if not .getLastCurlError().

3) There are a few different constructors for Json::Value, but for our purposes the default is sufficient.

4) Json members can be accessed with the .get() method or via the [] operator, es.g. jsonvalue.get("member",default), jsonvalue["member"].  I'm just using the [] operator, but either one seems to work.

That all in mind, let's walk through that last block a chunk at a time.

2.a) First, we need to make sure we can successfully connect to the twitter API, and here we see the twitcurl pattern in action.  .accountVerifyCredGet() "logs us in" and verifies our consumer and access keys, then returns some info about our account.  If all went according to plan (unlike the latest reincarnation), we should see the string representation of our jsonified twitter account info in the debug console:

if(twit.accountVerifyCredGet())
{
    twit.getLastWebResponse(resp);
    console() << resp << endl;

console() returns a reference to an output stream, provided for cross-platform friendliness.  Just think of it as Cinder's cout.

2.b) Now the fun stuff, let's get some usable data from twitter.  We'll do a quick twitter search, then get a json object from the result, provided everything goes well (from here on out, let's just assume that happens, if something goes horribly awry, email me and we'll work it out):

    if(twit.search(string("perceptualcomputing")))
    {
        twit.getLastWebResponse(resp);

        Json::Value root;
        Json::Reader json;
        bool parsed = json.parse(resp, root, false);

        if(!parsed)
        {
            console() << json.getFormattedErrorMessages() << endl;
        }

Hopefully nothing too hairy here, there's that twitcurl pattern again.  We do our search with our term of choice (note this could be a hashtag or an @name too), catch the result into a string, then call our json reader's parse() method.  The false argument for parse() just tells our reader to toss any comments it comes across while parsing the source string.  In this case, since we know what keys we're looking for, it's probably not a big deal, but if we were ever in a situation where we were going to have to query all the members to find something specific, having less noise might be a good thing.

2.c) Ok, since for the duration of this tutorial we're living in a perfect world, everything went according to plan, there were no oauth or parsing errors, and now we have a nice, pretty json egg ready to be cracked open and scrambled.  Let's get our tweets, split them up, and stash them in our string vector, then we'll be ready to make some art.

        else
        {
            const Json::Value results = root["results"];
            for(int i=0;i<results.size();++i)
            {
                temp.clear();
                const string content = results[i]["text"].asString();
                temp = split(content, ' ');
                words.insert(words.end(), temp.begin(), temp.end());
            }
        }
    }
}

Again, nothing crazy here, in fact I'm sorta starting to feel bad for making people read this since i'm not doing any crazy 3d, shadery, lighting, particle, meshy awesomeness, it's just simple parsing operations...Ah well, the sexy bullshit (as the good Josh Nimoy calls it) is coming, I promise.  One of the things to be aware of here is that Json::Value is really good about parsing data into the proper types for us.  As I mentioned earlier, the docs present a few different constructors, but we're not using any of those here.  Querying the "results" key (which contains all of our search results) gives us back a list we can iterate through in fairly simple order.  So all we do is parse that, then for every element in our array, we get its "text" key, which contains the actual body of a tweet.  Lastly, we take that text and use Cinder's built-in string splitter, which should be quite familiar to you if you've ever split a string in a different language.



    Looks like all we have left is to make some stuff happen on-screen, so same as we did with the setup() method, let's take a glance at the code first, then we'll break it down, although if you're already familiar with Cinder, there probably won't be anything new here...

void TwitCurlTestApp::draw()
{
    gl::color(0, 0, 0, 0.015f);
    gl::drawSolidRect(Rectf(0, 0, getWindowWidth(), getWindowHeight()));

    int numFrames = getElapsedFrames();
    if(numFrames%15==0)
    {
        if(words.size()>0)
        {
            int i = numFrames%words.size();

            gl::color(1, 1, 1, Rand::randFloat(0.25f, 0.75f));
            fontOpts.scale(randFloat(0.3f, 3.0f));
            font->drawString(words[i],
                Vec2f(Rand::randFloat(getWindowWidth()),
                    Rand::randFloat(getWindowHeight())),
                fontOpts );
        }
    }
}

3.a) No messing around, let's get right to it.  If you've ever done anything in processing, you're probably familiar with the technique we're implementing with these two lines of code to fade the foreground a bit between frames, i.e. set the fill color to black with some amount of transparency and draw a rectangle the size of the screen.

    gl::color(0, 0, 0, 0.015f);
    gl::drawSolidRect(Rectf(0, 0, getWindowWidth(), getWindowHeight()));

3.b) The last step then, is to draw some words to the screen.  We'll grab a new word every 15 frames, set the fill color to white (also with some amount of transparency), scale the font by a random amount, and draw the word to a random location in the window. 

    int numFrames = getElapsedFrames();
    if(numFrames%15==0)
    {
        if(words.size()>0)
        {
            int i = numFrames%words.size();

            gl::color(1, 1, 1, Rand::randFloat(0.25f, 0.75f));
            fontOpts.scale(Rand::randFloat(0.3f, 3.0f));
            font->drawString(words[i],
                Vec2f(Rand::randFloat(getWindowWidth()),
                    Rand::randFloat(getWindowHeight())),
                fontOpts );
        }
    }
}

At this point, we should be able to build/run the project and hopefully see something similar to this:


If something has gone horribly awry, send me an e-mail or hit me up on github.  I've put the project up on github as well, but be advised you may have to change some of the project settings to reflect your own build environment.  For the scope of my project, I've got quite a bit more twitter to learn, including how to manage tweets, maybe how to deal with the streaming API, and a few other things, but that's all down the road.  Next up: