Arranging By Color And Objects With t-SNE

For an upcoming Drift Station project, we’ve been considering how to curatorially sort a massive number of images (about 100k) for presentation. Chronological? Random order? Some other logical scheme? But a more computational approach seemed to make sense: some way of parsing the images that took into account a variety of visual factors in each image, something that would be impossible to do manually.

Neural networks are the obvious answer here, and so I found some very helpful sample code from Gene Kogan and Kyle McDonald, and wrote some Python and Processing code that loads up a folder of images and extracts a vector representation from them. Then, using t-SNE and Rasterfairy, the images were organized into a 2D grid.

I’ve spent the last few days playing with settings in the code, and found there is an interesting balance to be struck between locally preserving color similarity and object similarity. (Note: this post is more of a quick note than a deep-dive analysis.)

Above: a version with blurred images, showing a pretty clear separation by color with fairly smooth transitions. Click on images for a higher-res version. Continue reading “Arranging By Color And Objects With t-SNE”

Facebook Message

Weird religious spam message on Facebook this weekend (from someone I don’t know), including an over-compressed image of some scripty text, a poem (I guess), and a text-to-speech voice message, which is transcribed below.

“Rice plant this is for you. Read till the end, it’s adorable. Post box I sent an angel to watch over you last night, but it came back and I asked ‘why?’ The angel said, ‘the angels don’t watch over angels.’ 20 angels are in your world, 10 of them are sleeping, 9 of them are playing, and one is reading this message. God has seen you struggling with some things, and God says it’s over. A blessing is coming your way. Tulip if you believe in God, send this to 14 friends, including me. If I don’t get it back, I guess I’m not one of them. As soon as you get five replies, someone you love will quietly surprise you. Not joking maple leaf. Pass this message on. Please don’t ignore it. You are being tested, and God is going to fix two big things tonight in your favor. If you believe in God, drop everything and pass it on. Tomorrow will be the best day of your life. Don’t break this. Send this to 14 friends in 10 minutes, it’s not that hard. However sent this to you must care about you. You are an angel strawberry smile with a strawberry.”