post reality filter

        You know, for as much as i love music, i never really considered myself much of a music snob, or at least, not a genre snob anyway.  The whole idea of having to create millions of sub-genres in some feeble attempt to differentiate yourself from the next guy other than just having cool music always boggled my mind. Even worse, the music critics and journos who would always be coming up with new genres just to, i dunno, safeguard their idea of what a particular genre was or some such nonsense...For as much as i rail on game journalists, it's probably true of most media journalists.  The genre qualifier "post" always bothered me the most, what you can't think of a new genre name, so you just take the laziest path?  "Oh, this is what comes AFTER <genre>", in which case it might just be arrogance, i.e. really, wow, you think your music is that much different that you alone are going to define what comes next?  Wow alright then...

        Anyway, here's a fun little video experiment i knocked together in processing yesterday.  You can copy and paste  the code below into a sketch and run it, should be that simple.  Ohhh yes, you'll also need GSVideo, as I'm not using processing 1.5's native video.  After reading a few pages on what was required to get it working based on software that may or may not really exist anymore and also reading somewhere that p2 was going to move to GSVideo anyway, figured i might as well take the plunge.  It's a pretty friendly library overall...on the subject of laziness, gotta admit i probably could've captured a video of this, but didn't.  That's probably ok though, you should really run this yourself and see the effect, maybe play around with it a bit, see what you come up with...


Hello from videoland...

import codeanticode.gsvideo.*;

PVector gridStep = new PVector(16,8);
int gridX;
int gridY;
GSCapture vStream;

void setup()
{
  size(640, 480, P2D);
  frameRate(30);
  gridX = int(width / gridStep.x);
  gridY = int(height / gridStep.y);
  vStream = new GSCapture(this, width, height);
  vStream.start();
  background(0);
  noStroke();
}

void draw()
{ 
  if(vStream.available())
  {
    vStream.read();
    vStream.loadPixels();
    
    //Thanks, RoHS
    fill(0,0,0,8);           
    rectMode(CORNER);        
    rect(0,0,width,height);  
    
    for (int i = 0; i < gridX; i++)
    {
      for (int j = 0; j < gridY; j++)
      {
        int x = i*int(gridStep.x);
        int y = j*int(gridStep.y);
        int loc = (vStream.width - x - 1) + y*vStream.width;
      
        color col = color(red(vStream.pixels[loc]),green(vStream.pixels[loc]), blue(vStream.pixels[loc]), 32);
        float vradius = brightness(col)*0.1;
        
        fill(col);
        if(vradius<12.8)
        {
          ellipseMode(CENTER);
          ellipse(x, y, vradius,vradius);
        }
        else
        {
          rectMode(CENTER);
          rect(x,y,vradius,vradius);
        }
      }
    }
  }
}

First Light

        When i was working in SoCal, i worked with a rather brilliant tools programmer who used to refer to MVPs as the "First Light" version of something.  I like that term, MVP sounds so stuffy and business-y.
     
        Probably the most impactful realization I've had since coming to Intel is that I want to be a creative coder when I grow up.  My absolute dreamest job would be to build interactive digital art pieces/installations, and I know it's possible, because there are people out there doing it.  The downside is that it's an exteremely niche market.  It's like Tech Art or Concept Art on steroids, that is, people want to do it, moreso than the number of positions available.

This does not deter me.
        As daunting a challenge is it may seem, I have a particular set of skills, skills that will only partially help me achieve my new position through poise and audacity.  To this, I must now add resolve (guess both of those movies and I'll...i dunno buy you a shot next time i see you).  My resolve is that I need to get the rest of the skills required for this sort of work, and in the process maybe build some simple starting out pieces.  As i mentioned in my previous post, the big skill I feel like I'm lacking is not a technical one, but an artistic one.  Taking a step back from visualizing tweets, I decided to go even simpler and visualize some simple data-over-time type sets.  So, armed with pygame, I settled on visualizing periodic functions (sine and cosine in this case), and decided to see what I could come up with.

I started simple, just so I'd have a template to work with...

..but things got pretty crazy pretty quickly.
        Overall this was a fun exercise and really helped me get my feet wet.  The thing I realized over the course of doing this is that really the possibilities are endless.  I mean, this is just a simple 2d library with no interactivity.  I haven't even started getting crazy with processing, cinder, vvvv, or any of the other insanely cool toolkits out there, to say nothing of something like Unity or other game engines.  I've already started working on some interactive stuff, mayhap I'll post some of that next once it gets a little more presentable.

        I uploaded all the source from this to my Github, or you can just check out the results on my Vimeo channel.  I challenge you to see what you can come up with yourself, you might be surprised how addicting it is...

"Fun" is part of "Functional"

        It's been a really interesting...week?  I dunno, to be honest, time has completely disappeared, which is a good thing.  I've always had some interesting thoughts about time, but that's for another blog post, or maybe a good smokeout.  Given the kind of work I'm doing now, a little bit of the ol' green might not be a bad idea, but we'll save that as a last resort.

        As I think i've probably mentioned to some folks, one of the things I'm working on now, or at least was supposed to be working on, is "new" interaction ideas, natural/new user interfaces, and all kinds of other buzzwords that add up to "do what nintendo and apple did, but better".  No pressure.  I'm going to be honest with everyone, i'm NOT an incredibly brilliant, innovative, or creative person, but i think i know what i like, which i feel like could be an asset in this space.  This all sort of hit home a little while ago while i was prototyping some tech for an interaction demo to showcase a...pretty freakin cool bit of technology my co-workers Stan and Sterling have been grinding on for a bit.  Here's a short demo:


        Apologies for the lame-o compression, it's been a while since i've been a video producing guy man, something i intend to change, and by that i mean change that i haven't been producing videos, not that i'm a guy man.  Aaaaaanyway...So here's the thing, I found myself just playing with this demo for...minutes at a time, seriously.  And by playing i mean, putting my mouse on one of the fireflies and watching it light up.  How crazy is that??  It's funny though, i'm reminded of a story Mom used to tell me about how Dad would take me to the arcade when i was a wee bern and sit me on the pinball glass while he would play.  Apparently, i would paw at the glass trying to get to all the shiny-movey things.  While i have no memory of this, i feel like it probably affected me pretty deeply (and relevantly!).

        Fast forward to about a week ago, my co-worker Chris showed me this video from The Creators Project (which I'm so going to next year), which really got me thinking.  Unlike me, Memo Akten actually is a brilliant, innovative, creative individual, and i could watch his stuff forever.  Seriously, TRY and shut this off once you start watching it, you won't be able to...


        This got me thinking a few things, in no particular order or connectivity:

• Complexity from simplicity is beauty
• Chronological doesn't mean linear
• I need to start small
• "Fun" is part of functional
• Graphics programming can make things that aren't games
• In digital world, data is just numbers, and numbers make pictures

        Now, the last two points, i imagine you're all going "well duh", but here's the thing.  I'm VERY new to all this.  For all i've been around graphics programming and interaction design, i have to retrain myself to think of things not as game tools or game concepts.  I understand that the delta between "game" and "interactive experience" from a high level isn't that great, but at the same time, there are things you wouldn't do in a game, presentation-wise, that i feel like is probably permissible in other spaces.

        So i decided to start small and visualize an obvious and relevant known: Tweets!  More specifically, mentions.  I wanted to especially get away from linear representations and just think of mentions as a big data cloud.  How would i display and navigate said cloud?  I grabbed Tweepy and pygame and started messing around with some different ideas.  Everything here is WIP, not even anywhere close to being a solid idea, for now i'm just playing around.



Iter 0: Branching off from the most recent mention, age denoted by line color.


Iter 1: Main mentions represented by orbits, age denoted by distance from center. Related tweets branch off of each mention.


Iter 2: Same as 1, related tweets represented radially.
        This is fun, but you know, I'm not nearly as drawn into this as I am from the fireflies for some reason, even after running it on some touch-enabled hardware.

        But that's the thing, I can figure that out.  Like i said, in this space, i'm a BABY.  I need to really just immerse myself in all this stuff, the way I did when I was a game developer.  Taking a bit of my own advice, I'm going to go back to square one and start small.  Taking a page from Memo Akten, I'm going to visualize some simple data-over-time sets just to get my head more into that space.  Taking another page from Memo Akten, i'm going to start with simple functions.  Maybe I'll use a cosine though, just to be slightly different, hehe...Stay Tuned.  I feel like i'm finally hitting a bit of a stride here, like I finally gave myself permission to work on this stuff.  It's funny, it's been such a big mandate for our group, but i've felt like...it was too much not like work to be working on.  Ah well...at least i'm changing before it's too late.

i can see forever

        It's funny how sometimes just being around a term or concept so much skews your perspective on it.  Case in point, at work, one of the hot topics is CV (computer vision), like, you're immersed in it here.  Sometimes it's hard to not feel like everyone in the world is researching/investigating/developing products against a certain technology and you're being left behind.  Of course, in the case of CV, that's probably absolutely true, i'm sure everyone is trying to figure out how to use it to drive the next wave of great interactive experiences, but then, i guess that's the real hot topic.  Anyway, i just thought it was funny how over the last month i've felt like CV is the new hotness and everyone in the world is playing with it...


OpenCV...one of the few things that doesn't break when it sees my ugly mug...
        That said, the thing that caught my eye most at Maker Faire was SimpleCV, which is a wrapper around OpenCV (ya think) and a few other super useful libraries.  They had a few really cool demos running, but it was more being able to look at the code and see how simple it was that really got my attention.  Not that OpenCV by itself doesn't have a ton of functionality already, SimpleCV just has a few more convenience functions, not to mention some slick blob finding stuff (among other things).  The big (and very specific) hurdle I ran into is that SimpleCV uses libfreenect, specifically the python bindings, for kinect access, and from everything I read online, getting the freenect python bindings to play nice with Windows is a bit of a chore.  I must say, in the last week or so, I've been amazed at how many tasks i've undertaken that turned out to be...not so simple or well documented.  That means i'm either truly blazing the trail or I'm doing it horribly wrong and missing the obvious solution.  Let's assume the latter.


         Anyway, so after a day of trying to build python freenect and only being somewhat succesful, i decided to see if I could get SimpleCV and pykinect to play nice together.  Thanks to some hard work done by another Codeplex-er who goes by the handle bunkus, i got pykinect and the OpenCV python bindings talking.  From there it was a pretty simple step to get pykinect and SimpleCV talking, since SimpleCV talks straight through OpenCV.  Well, it would've been if i'd remembered to install PIL...As much time as i'd spend digging through the SimpleCV ImageClass source, you'd think the dependencies would've been burned into my brain.


         But you know, at the end of the day, I've got pykinect streaming into SimpleCV, which is a good thing.  There's been a ton of interesting work done in open source kinect space, but having had my fill of trying to deal with OpenNI, i'm more than happy to use a supported, iterated on SDK with the most current features.  Who knows what I might want to do next?  So here's some code, yes i know, it's horrible and un-pythonic and there are globals all over the place, but it works...cleanup is next. Actually, next I'm going to work my way through the SimpleCV book and see what happens.  This + multiple kinects?  If performance doesn't murder me, could be pretty cool...

import array
import thread

import cv
import SimpleCV
import pykinect.nui

def frame_ready(frame):
    global disp, screen_lock, img_address, img_bytes, cv_img, cv_img_3, alpha_img
    with screen_lock:
        frame.image.copy_bits(img_address)
        cv.SetData(cv_img, img_bytes.tostring())
        cv.MixChannels([cv_img],[cv_img_3,alpha_img], [(0,0),(1,1),(2,2),(3,3)])
        scv_img = SimpleCV.Image(cv_img_3)
        scv_img.save(disp)

if __name__=='__main__':
    screen_lock = thread.allocate()
    cv_img = cv.CreateImage((640,480), cv.IPL_DEPTH_8U,4)
    cv_img_3 = cv.CreateImage((640,480), cv.IPL_DEPTH_8U,3)
    alpha_img = cv.CreateImage((640,480), cv.IPL_DEPTH_8U,1)
    img_bytes = array.array('c','0'*640*480*4)
    img_address = img_bytes.buffer_info()[0]
    disp = SimpleCV.Display()

    kinect = pykinect.nui.Runtime()
    kinect.video_frame_ready += frame_ready
    kinect.video_stream.open(pykinect.nui.ImageStreamType.Video, 2, pykinect.nui.ImageResolution.Resolution640x480, pykinect.nui.ImageType.Color)

in defense of the generalist

        A while back (years almost, come to think about it), I made a post on a friend's facebook on why I thought generalist as it related to Tech Art was probably a bad goal to pursue in the long term. I'll admit at the time I was definitely not in the best of headspaces and not at all thinking straight.  I'd also been through a few interesting work stints that had me questioning the value of Tech Art in general, or in other words I was feeling sorry for myself and projecting onto other people, trying to "save" them from a professional fate worse than death.  Or something to that effect.  But as I alluded to in my post the other day and as I remarked (or maybe admitted) to the same individual who i had previously publicly lambasted for being a generalist, if it were not for my own background as a generalist, there's no way I could be doing the job I'm currently attempting to bungle my way through...



As with many things, happiness is found somewhere in between...
        It's true, at every stage of my career I've picked up more skills and honestly, the only reason I ended up where I was doing what I was doing was because I felt it necessary to fill a niche in production.  My first job was actually as a VFX intern, but I did some facial animation, rigging, cinematics, modeling, hell, I even have a sound credit from my first job, of all things.  So I, of all people, definitely appreciate the value of the generalist developer.

        Now, I'm probably not telling anybody anything that they're not already thinking, or more correctly, haven't already realized.  At the same time, I feel like I've been one of the more vocal proponents of the Tech Artist as Tools Programmer, or at least the notion that Tech Artists should specialize in something at some point.  And yet, here I am, finding myself not really specializing in anything, or even more ironically, specializing at being a generalist...talk about taking the red pill.



Just take both of the pills, seriously, the purple pill is f'ing wicked...

        I guess what I'm really saying is not to specialize or generalize, but more like don't pass up an opportunity to learn new things.  One of the most disheartening statements I'd ever heard uttered was something to the effect of "Well, i'm working in Max right now, I don't need to learn python".  Yeah, but you could!  It's not about learning python, or C#, or C++, or Lisp, or MSP, it's about LEARNING.  When I was at Bungie, sure, I was a Maya guy through and through, but that didn't stop me from taking advantage of the fact that I was working with some great C# developers and learning C#, a skill that's serving me really well now.  And once you're comfortable with that new skillset, Apply It!  Take some time and come up with some creative solutions, in the process you'll gain a deeper understanding of said skillset.  I started learning Django just for fun, but in the process, I realized what a great asset management system Django would make, or to be more specific, what a great asset management you could write on top of Django.  Being able to apply technologies in new ways is a great way to keep your interests up, especially if it's technology that already exists and is just waiting to be put to a creative use.  As Grandmaster Harper says "Take What's In Front Of You".



Tech Art...The Kajukenbo of Game Development
        The industry being what it is, you never know what your job sitch is going to look like in the next few years, heck even in the next year.  The skills you learn now are going to protect you for your entire career.  Who knows?  If it ever comes down to it and you have to jump from content creation, the fact that you used Python to do more than just write Maya scripts may be your saving grace.

        So in the end, expand your horizons, learn all you can, and embrace your generalist self.  I'm definitely not sorry I did, given the dividends it's paying now.

prototyper's toolbox: pythonic edition

        I had a really good convo with a co-worker today and I realized I miss working in Python just a wee bit.  For prototyping especially, why wouldn't you use Python?   I'm surprised the thought hasn't really hit me until just now, but you know, the fast turnaround provided by working in an interpreted environment is fairly ideal for rapid prototyping, no?

        As much as I like Unity, it's definitely getting to the point where i'm having to do enough custom implementation that it's putting a damper in my relevant iteration time.  Don't get me wrong, i'm enjoying getting to dip my toes into other languages and features (learning P/Invoke has been pretty cool), but i feel like a Python based prototyping environment might get me a little further a little faster.  More often than not, Python modules work in whatever your Python environment is, and I understand that's not a 1-to-1 comparison as Unity is not a pure .NET environment, but still...

        So in my wanderings last night, I dug up a few different tools that seem like things I'll want to be diving into next.  I went to bed super early, so I haven't spent a ton of time with alot of these resources, but enough to feel like these will all be useful in the future.  Once i get done with this current demo cycle, I'm going to get out of Unityland and start writing standalone apps, just to get more practice writing big software projects.  I feel like I'll have more flexibility in a pure python environment and I'll probably get more done faster. Props to my co-worker Chris for inspiring my search...

---

Modules/Libraries
        Most of these provide the full suite of lower-level functionality you need for prototyping/building media applications (event loop processing, input management, etc), just import whatever else you need alongside them and you're good to go.  Fair warning, some of these are NUI/MT specific, but then, that's really what I'm into these days.  I gotta say it's also made me realize how much of a gadgetwhore I am, but man, I hate spending money on them, which is probably why i don't have one.  That and it also makes me realize how much of a touch interface freak I am.  I can't say I'm huge on gestures, but touch is cool.  I actually found a fairly extensive list of game engines that would probably be make great generic event loop/input/rendering managers too, that's for another blogpost...

Python for Android
I'm not 100% sure why i put this here but it seemed like a cool little aside. I only glanced over the docs a bit, so I can't make any recommendations, but I will say it's definitely more of a power tool than a ready to go development resource. More and more i keep finding reasons to want to get that Android device i keep threatening to get, but then i think, ugh, do i really want to spend money on another gadget? "Another", like i own a bunch of them already...

pygame and PyKinect
I lumped both of these together because most of the PyKinect samples run on pygame anyway, altho nothing precludes you from using your event loop manager (heretoforth referred to as ELMs) of choice. While this isn't necessarily MT, if you want to get started with NUI prototyping, this is honestly the way I'd recommend going. The Kinect SDK is super easy to write code against, you'll be doing crazy things in no time.

VPython
Alright kids, gather around, and let me tell you a tale of POVRay...yes, I just dated myself horribly, but then, i'm constantly getting told that I'm too old for...certain people anyway, so i guess it is what it is. But really, that's about what we have here, even to the point where VPython lets you export to POVRay renderable files. How cool is that? Also includes its own version of IDLE, but I'm going to have to frown on that...

kivy
From what I understand, this is the way to go for Python MT development. I found a few other options (PyMT, etc), but this seems to be the one everyone recommends. If you're an experienced pythonista or have an established development environment, you can skip some of the weird setup requirements and dive right in. Get started now and whip some cool stuff up for their next contest!

pyTUIO
No discussion of MT development would be complete without including some TUIO bindings, so here you go! There are a few dependencies you might want to be aware of too (most notably reacTIVision), but once you've get everything down, this is another great way to get up and running with minimal hardware investment and setup. Make your own Reactable, actually, i've just hit on a plan...

---

Art IDEs
        Sure you could sit in a coffee shop and write a book, but let's be honest, if any chick looks over your shoulder and sees you rockin some cool interactive art in one of these apps, there's an icebreaker. Show her your camera or other input device, get her to play around with it a bit and it's all over except for the part where you embarrass yourself horribly trying to ask her out...Probably better to just get a bunch of buddies together and rock some interactive art jams on your laptops, takes me back to the days of laptop jamming with Live.  Or at least I think it would, i'll know more tomorrow, actually going to a group Processing session of sorts, should be fun. May even be the beginning of my social life...

Nodebox 2
If you're a Maya or Houdini Tech Artist, or just an ICE freak, this one's for you.  I actually came across Nodebox a while back, but got a sad panda because it was MacOS only, altho I suppose it would have been a good use for my still not very used Macbook Pro. But yeah, this is basically Processing with a hypergraph attached to it, how cool is that? And Python to boot, so it's like Maya, but light and stable...ish.

Shoebot
Shoebot is the best-ish of both worlds in that you can run it from its own IDE or you can import it into existing Python projects. It's also a...well, let's say derivative, for lack of a better term, of Nodebox, so most of the docs you find for one apply to the other. Not to down on the work of the Shoebot guys, it's a "derivative" like pyprocessing or pycessing are derivatives of Processing.

---

Additional Reading
        A few last little tidbits to keep your head in the MT game, skip this if you're not interested in this sort of thing.

Txzone
All sorts of really fun musings here, not much of a learning resource, but great for inspiration and keeping up with what all's out there for the MT pythonista...


Getting Started With MT Dev In Python
Some dated and slightly specific tutorials, but definitely great for a quick dip of your toes into the whole world of Python MT/NUI development.


Part 1 | Part 2 | Part 3 | Part 4

---

        I've got some pretty cool interactive art projects in the pipe right now, no ETA on any of them yet, but hopefully I'll be able to post some stuff up in the near future.  Got some longer term "milestones", altho I'm not sure milestone is the proper term since they tend to equate to public releases or showings of some sort...Nothing like a little pressure.

with trees

        Updated to include Teck's suggestions so it looks less like machine and more like man(god?).  Now we get something that looks like this:


Yes, this looks much better methinks.  Only slight changes, just dropping in new terms really:

void Start() { terr = GetComponent<Terrain>(); terr.terrainData.treeInstances = new TreeInstance[]{}; float mapHeight = terr.terrainData.heightmapHeight; float mapWidth = terr.terrainData.heightmapWidth; for(int j=0;j<mapHeight;j+=4) { int treeLine = Random.Range(50,100); for(int i=0;i<mapWidth;i+=4) { TreeInstance t = new TreeInstance(); if(terr.terrainData.GetHeight(i,j)>treeLine) { Vector2 noise = Random.insideUnitCircle*0.01f; t.prototypeIndex = 0; t.position = new Vector3(((float)i/mapWidth)+noise.x,0,((float)j/mapHeight)+noise.y); t.color = Color.white; t.lightmapColor = Color.white; t.heightScale = 1; t.widthScale = 1; terr.AddTreeInstance(t); } } Debug.Log (j); } //terr.terrainData.treeInstances.SetValue (t,terr.terrainData.treeInstances.Length-1); terr.Flush(); }