Introducing Cryptstagram

[This is the long version of a post I wrote for The Barbarian Group’s tumblr.  We also got written up in Wired’s design blog, Laughing Squid, and PSFK, my first time getting such large coverage!]

Periodically at The Barbarian Group, a small team of new employees and veterans with varying skill-sets are brought together to work on Project Popcorn, a side project that embodies the inventiveness, technical skills, and values of the agency.

Within my first week as a developer for the agency, I was assigned to Project Popcorn, along with a senior art director, a senior copywriter, a creative director, and a technical director.

Cryptstagram is the resultant product of our teamwork. Cryptstagram is a web site that lets you steganographically encrypt a hidden message into any image, unlockable only with a specific password, and then apply glitchy Instagram-like filters to that image.

The National Security Agency whistleblowing story had just broken, and as a result we quickly became fixated on doing something involving cryptography and, to fulfill our artistic ambitions, on doing something involving the glitch aesthetic.

Cryptstagram-5

The glitch aesthetic is popular amongst our creatives and developers because it requires technical knowledge of formats and methods underpinning systems in order to then manipulate, pixel-push, and distort those systems towards the artist’s ambitions.

As a team we spent a lot of time discussing what tone we wanted to convey with Cryptstagram. How secure should it be end-to-end to pay respect to citizens’ cryptographic requirements and expectations of privacy? Should Cryptstagram’s filters create aesthetically-pleasing images or should they reflect entropy of input and interaction from the user? To what degree did we want Cryptstagram to be a statement about the NSA and privacy in general or just a reaction to the state of our relationship with technology?

In the end we decided on a reflective tone for Cryptstagram, emphasizing a common public desire to share stylized parts of ourselves in public but other parts of ourselves in private or semi-private. We should empower ourselves with stealth and style.

Cryptstagram-2

Cryptstagram-4

The Project Popcorn team also wanted to keep Cryptstagram open-ended, taking advantage of the fact that JavaScript is pervasive in users’ browsers and that most computation, SHA-1 encryption and decryption, and glitching can now be done with open source libraries, entirely on the client’s side, and using HTML5 Canvas. A JavaScript stack (Node.js/Express, MongoDB, and good ol’ jQuery) cached and served up via cloud services allows for rapid app development and easy deployment for a small creative team.

So, easily, a citizen could choose to encrypt her image with a message and save it without it ever touching an external server, then send that image via private means to intended recipients with a password that could be distributed in any number of ways. Or the citizen could share the image on Cryptstagram’s wall for others to see and even try to decrypt!

A perk of this open-ended design is that we can extend its utility: we (or you, via jsfiddle sandbox!) can add seasonally- or topically- themed filters for specific clients, a celebrity can release images with a password known only to fans, or people can use the Cryptstagram wall to create puzzles or scavenger hunts or just post encrypted messages publicly without drawing suspicion.

At The Barbarian Group, we’re invested in making things that are gonna be awesome, and in order to do so, we need to be just as able to wade through the technical limitations of CORS and CSRF as to create something people love to use and share online.

So, what creative uses can you come up with for Cryptstagram?

Glitch: The Flip-Flop Technique

For our final Glitch class, our assignment was to employ the flip-flop technique, as described by Robin Sloan.  Technically, our job was to glitch something by flipping it between the analog and digital x times, or as Sloan says, “the process of pushing a work of art or craft from the physical world to the digital world and back again—maybe more than once”.

I decided to make a puzzle of it, to see if my idea would work.  First I had to put a file online, shorten it via bit.ly (to reduce encoding complexity in the QR code), and then generate a QR code for it.  Here’s the summation of that:

In order to do my flip-flop, one would need to graph out the x,y grid coordinates from a txt file onto graph paper. (ANALOG) Check out the text file here.  0,0 is the top-left of the 25×25 grid.

You can see it’s a 25×25 grid.  Once you draw out the pixel coordinates, you’ll get what appears to be a QR code.  Here’s my work, partially through it:

I wasn’t sure if this part would work, though in theory it should have.  When I first filled in the QR code on my graphing paper, neither Google Goggles nor QR Droid could detect the QR code.  Google Goggles found the closest match to be photo images of bathroom tile patterns.

Then, figuring my drawn QR code wasn’t precise enough, or perhaps dark enough, I erased my axis labels and filled in the square pixels with a black pen.  To my amazement, the QR code then worked in QR Droid! (DIGITAL)

I was surprised that it could pick up the code, because I started getting sloppy while filling in the squares, figuring I could just use the final product as some artistic hand-drawn rendering of a QR code.  Apparently though, QR codes, depending on encoding, can handle quite a lot of error, and correct for it.  Here’s some information about how its error correction works (thanks Neil).

One interesting project handling data error and error correction was our professor Kyle McDonald’s Future Fragments project, where he had other classmates encode messages into colored grid squares, then keep the printouts in their wallets for a summer, then decoding the blocks back into a message after the pieces of paper had been worn down a bit.

The QR code takes you to a bit.ly link, which forwards you to an image on my site, a reproduction of Caravaggio’s The Card Sharps, one of my favorite paintings. (DIGITAL)

 

The next task would then be to hand-draw the painting. (ANALOG)

Then you take a photo of it to auto-save to Google+ or upload to the cloud. (DIGITAL)

Then the final task would be to color in the drawing using Photoshop or another digital tool. (DIGITAL)

Here was my final product:

As a final test, I thought I’d see if I could run the drawings through glassgiant.com’s ASCII converter.  The results for my drawing were not too good, probably mainly because I did not make bold enough lines and outlines to make the ASCII conversion stark enough.  I also ASCII’d up the QR code (ink) and the original Caravaggio. (DIGITAL)

Note: along the way, I found this awesome site that has a QR code stencil generator, for making stencils in hobo code!

Anyway, that concludes my work for Glitch class.  Thank you Kyle McDonald and Jeremiah Johnson!  This ITP class was fucking kickass.

Glitching in Processing

Some more stuff from my Glitch class, which ends next week. :/  My impression of the class is that, though most people (including, perhaps, my professors) may believe glitch-alike and glitch aesthetic are passing fads that move way too fast in the transition to some future styles of art, that Glitch is some of the most fascinating artwork I’ve seen in a long, long time.  I never really want to see any more old paintings of Christian-themed stuff, nor the post-modern Christian-themed shock art, and painting and drawing and sculpture are sort of done with, with hipsterish 8-bit and old video game-style art being too nostalgic from people who are still too young.  In terms of artists pushing boundaries, criticizing the system, and making a commentary on society, it’s hard to think of a better place for art than a world where people are learning how things work, then reprogramming them to either subvert the original intent or to distort it in a way which undermines the rigidity and order of large, interconnected, imposing data/network/bureaucratic systems.

Anyway.  In our last assignment, we all had to create an algorithm in Processing which could be used to distort an image, so that we could all put our algorithms together and output the results next to each other for comparison.

For my algo, I took Kyle McDonald’s algo which broke out an image into a grid of pixels.  Then I figured I could weave in another image by using modulus (%) arithmetic, an easy programming trick, to intersperse pixels at specific places into the canvas.  Then I also decided to take the original image’s pixels and map it on upside-down onto the image.  The original:

The result:

The algo:

PImage vt520_1(PImage img) {
  PImage img2 = loadImage("flag.jpg");
  img2.loadPixels();
  int n = img.width * img.height;
  for (int j = 0; j < n; j++) {
    color cur = img.pixels[j];
    color cur2 = img2.pixels[j];
    int r = int(red(cur)), g = int(green(cur)), b = int(blue(cur));
    int off = j*10&(j>>20)|j*3&(j*5>>15); // miiro
    off %= 256; // don't let the offset overflow
    if (j%2==0) {
    img.pixels[j] = color(0, 0, g^off, 0);
    }
    else if (j%3==0 || j%7==0) {
      img.pixels[j] = cur2;
    }
    else  {
      img.pixels[n-j-1] = color(r^off, 0, 0);
    }
  }

  return img;
}

Interestingly, because I was using modulus arithmetic, this resulted in the image changing in some applications because of the way they scale images down to certain sizes.  Some other views of the image:

Should have an image of all the algos working together at some point soon…  Classmates came up with some pretty amazing stuff!

Epler’s:

Guilherme’s:

Then our in-class assignment was to go out on the cloud and try to break services or get banned.  I liked this trick on Facebook by Hanna Kang-Brown, just using extended characters:

Breakout Glitch

Our task in Glitch this week was to glitch Steph Thirion‘s Breakout code written in Processing, using an hour to see what we could come up with.

Here’s a vid of Super-Breakout:

My result:

I didn’t feel like it was destructive or additive; mainly I was just tweaking values with randomization, and then rotated it a bit, which would need further tweaking to have more of it on-screen.

Github gist: https://gist.github.com/3835687

Though I did find this pretty cool glitchiness written in Processing, by Amnon:

We had another assignment, which was to post screenshots of our crash reports.  Here’s mine, from doing my Java homework in Eclipse (I think I had an infinite loop going or something):

And finally, my girlfriend tried to take a photo using her phone’s front-facing camera, but this kept happening:

Glitch Work

This is stuff from my ITP Glitch class.

One-Line-Code Music

We were introduced to one-line code that could generate music, as we studied Demoscene, which, among other things, tries to compact whole videos into ridiculously few lines of code by using generative algorithms and mathematics.   This can be said to create computationally minimal art.  A few users began to dabble in making music in this way (read this and this).

Videos of the captured music plus the code that generated them:

Here’s what I came up with, entitled, “Invaders”, via a link to the site.  Or, below, an MP3 w/ HTML5:

Your browser does not support the audio element.

The examples from the blogs are much richer, more refined, and more varied than mine is.  Mine sounds great, but it’s simplistic, and sounds mathematically generated as opposed to crafted.

Here’s the visual representation of the formulae, from the site:

But hey, try your hand at making this music!  The web site for it is super easy to use!

Number Generator

I might have overthought this part of the assignment.  I liked Epler’s random number generator which counts the number of chewing gum stains on squares of sidewalk.  But I couldn’t think of something in my daily life that’s actually random.

In NYC, not only is it a grid-like city full of regular systems (crosswalk signals, daily work day routines, structured lanes and processes), but that oppressiveness of systems (if you want to see it that way) is relentless against things which do not go with the flow.  Side note: perhaps this makes it a reliable medium for different industries and cultures to interact with each other in a standardized way.

Anyway, I also had a problem with this task because computers do randomization better than nature does.  Nature is full of systems as well.  It can be highly regular and predictable, even if it has long seemed like chaos to us.  Computers, however, can give you a reliable distribution of truly randomized numbers, if they are given enough input noise to counteract any regularity in computers’ signals.

So, why do this task if code can do it any which way you like?

That said, I talked with my girlfriend and we came up with something suitable: counting the number of fallen leaves per hour in a day.  While wind and time of year may affect day-to-day numbers, within the hour or smaller segments the variance is large enough to come up with reasonable frequent and random data.

It surprised me how regular any “random” data is.  You could come up with any predictive range for many “random” numbers, such as counting people carrying coffee cups, women in skirts or jeans, number of people entering a store (which, I learned recently, big department stores have very accurate software motion-tracking to calculate for them).

We live in a world of systems which drastically affects our lives.  We’re used to it, of course, but it reminds me of a book we had to read for my globalization class at Georgetown, which talked about the Meiji Restoration in Japan, putting Japan onto the western calendar and causing massive cultural upheaval to a culture which did not rely on regularized numerical calendar dates.

Glitch-Aesthetic

ITP Glitch: Editing Civilization Save Files

So far in my Glitch class, we’ve experimented with databending and transcoding images, audio, and video, using a variety of different techniques, but mainly trying to de-abstract how software handles file formats so that we can edit the actual bytes of files to alter the expected output — e.g. shifting the bytes by one in order to cause the image’s dimensions to warp or change color.

Here is our homework assignment for this week:

  • Find and write about a file format / specification. For example, ISO/IEC 10918 (JPEG),  ISO/IEC 1172-3 (MP3), ISO/IEC 14496-3 (AAC) GIF89a, ZIP, MIDI, etc. See List of Codecs for more. You should provide an analysis of any features of the format that may be interesting to glitch artists working with it, or provide a history that explains the various biases that are reified by the format, and the advantages and disadvantages of those biases in actual use.
  • Create your own file format / specification. A complete specification will allow someone else to both encode media into an intermediate representation, and then decode it back to an approximation (lossy) or exact replica (lossless) of the original. As far as “types” of media: audio, images and video are popular; dance, haptic sensations, taste, legal requirements, written language, etc. all count as media. You must run at least one piece of media through your encoding / decoding process, but you may do this by hand — it does not need to be executed by a computer.

 

The second assignment seems pretty difficult at this point.  I looked into making “extrafiles” and found this OS X app, extrafile, which introduces a series of unique file formats such as one called “Block Ascii” which is only 1bit color:

So you can save a file into one of those formats using the app.

Coming up with my own file format specification would take a while, and I’m not sure how I would transpose data through media processing without losing everything in a jumble of noise.  However, I did do an assignment last semester dealing with genetic inheritance which involved passing a database record of peoples’ stats (1-10 sliding scales of peoples’ stats, like creativity or strength or determination) into a Wolfram cellular automata algorithm, via a node.js server passing JSON, so that the output looked like this:

But that’s not really a file format per se.  It might be interesting to be able to encode someone’s physical and life characteristics into a photo file of them, so that in, say, 50 years, someone could open up the photo image and see what the person’s biometrics and other statistics were at the time the photo was taken.  Hmm, possible idea!

So in terms of describing another existing file format, it made me think about when I first played Civilization as a kid.  Back then, there was no real internet to speak of, and I was using Prodigy* via a dialup modem so that I could (slowly) browse the Civ forums there.  Some people who knew their shit were hex editing their save files so that they could test the limits of the game (e.g. I think cities couldn’t get any higher than 99 population or something, because of memory issues from going to 3 digits) and cheat to win.  That stuff was over my head so I never played with it, but I remember it to my day as a way that some people were able to change settings for games, registration verification, etc.

Anyway, even though it’s hard to look up info on the first couple Civ games (because most Google results now are for Civ 4 and 5), I did find pages with a pretty good write-up of the Civilization 2 savefile format and of the Civilization 3 BIC file format.

On data types:

"...there are three data types: The null-terminated string is a variable length type for text. It always ends with a byte value of zero, meaning a string of 9 bytes only has place for at most 8 characters. The signedshort integer is a numeric data type taking up 2 bytes. It can take values from -32768 to +32767. The unsigned byteshould be self-explanatory. It takes values from 0 to 255."

For handling states of engineer units:

An engineer in CIV II generates 2 points per turn. A settler generates 1 point. These numbers remain constant when the “Base time for engineers to transform terrain (x2)” is changed in the Rules file. With the “Base time…….” set to 20 (vanilla CIV II), twenty points are needed to transform Grassland to Hills.
Consequently, if the Grassland to Hills transformation command is entered on turn 1, a single Engineer will complete the transformation on turn 10. Its byte 15 is changed to 2 on turn 1, is incremented by 2 on each of the following 9 turns, and reaches 20 on turn 10.
If 2 engineers (E1 and E2) are assigned to the task, their byte 15’s change as follows:
UNIT -- E1 E2
 TURN 1 04 00
 TURN 2 08 00
 TURN 3 12 00
 TURN 4 16 00
 TURN 5 00 00 Task completed

Some culture variables in the savefile:

CULT SECTION (Culture)
  4	char		"CULT"
  4	long		??? (33)
  4	long		??? (5)
  4	long		number of culture opinions
  For each culture opinion:
    4	long		length of culture data (88)
    64	string		culture opinion name
    4	long		chance of successful propaganda
    4	long		culture ratio percentage (3:1 = 300, 3:4 = 75)
    4	long		culture ratio denominator (e.g. the 1 in 3:1)
    4	long		culture ratio numerator (e.g. the 3 in 3:1)
    4	long		initial resistance chance
    4	long		continued resistance chance

It might be interesting, given the exact formats and memory sizes of the variables, to make a random savefile generator, and then open it up within the game to see what you get.  Would the game be playable, or does it require a certain logic within the savefile where, say, you couldn’t have some variables if other variables were of a certain value?

Tool-assisted speedruns have become big as a result of savefiles.  Basically, if you can save the exact state of a game in a file, you can restore from that point without limit.  I played through Zelda: Ocarina of Time via an emulator on my computer back when I was in college, so it was easy to quickly restore at difficult points in the game.  Apparently all this is called save scumming.  This is how you get those weird videos every once in a while that show some Japanese player doing a perfect speedrun through a custom-made annoying-as-holy-fuck Mario Brothers-family game, e.g.:

That video introduces some followup questions.  Why do people subject themselves to such torture?  How did that person not kill everyone in his building?  Is there a good reason he never uploaded past part 3 since 2007?  Did he go on a murderous rampage?

In terms of savefile formats, I know that later Civ games used XML for settings files and maybe savefile formats?  What if you could translate the format of one game’s savefiles to that of another, so you could play your character in a different world?  More importantly, what if there were a standardized player file format that you could re-use between different games through a secure authenticator? (say, you used it on Steam)  This has been done with some games and some networks (such as Mass Effect on Xbox Live), but how about an open player file format?  And how about if it included your real-world Galapag.us characteristics and history?