User Zero

I came up with the idea in 2006.  An email I sent to my Army buddy in April, 2006:

I sort of had an idea but it seems like it’ll be difficult to build out.  My idea would be for something similar to Xbox Live’s ranking system.  Except it’s for your life.  Privacy issues aside, people would volunteer to put in as much personal info as they want.  At first it might seem cumbersome putting in so much info but I think as myspace and other services have shown, people are willing to do it if it means it cultivates their identity.

So for instance you put in your income and number of kids and connect your accounts for online game rankings (like in Halo or Battlefield 2) and your exercise plan and your birthdate and your finances and investments and how many web sites you’re on (like myspace, digg, yahoo, etc.) and from all that data, the company would generate statistics that break down your life and give you info about how much time you spend on certain tasks, how efficient you are with your money, what your online reputation is.  Stuff like that.  The core would be statistics…anonymous statistics I think so people won’t have any incentive to forge their results.  The point would be to turn peoples’ lives into a numeric game where they can see how they rate in certain aspects of their lives.  Think of all those online quizzes people take about what kind of lover they are or what their personality is.  That could be tabulated into the statistics, which could be searchable or broke down any way the person wanted.

At the end of a year, we could look internally at our statistics and go visit the top overall people in person to go verify their data and videotape their lives, interview them.  Then a winner would be announced…like the best person award.  Heh…there’d be so much controversy and whining and competition if it caught on.  Then we could write a book about our experiences going out and discovering what makes someone “the best” compared to everyone else.

So…that’s my idea so far.  Sort of like a real-life RPG.  Perhaps we could offer points for real-world scavenger hunts or traveling to different countries around the globe.  What about having life coaches for certain segments, if someone was weak in an area like professional development?  I was thinking we could also offer points for accomplishing certain tasks like humanitarian work.

A lot of stuff happened in the meantime: I got out of the Army, went to study foreign policy in DC, worked for Homeland Security, moved to NYC for school.  And so now I’m wrapping up the thesis, which allowed me more than a semester to work just about full-time (including any waking moment) on trying to make Galapag.us a reality before I can either A) get funding or B) get an engineer/developer job after school.

So I present Galapag.us for thesis on May 15 at NYU.  I have two weeks still to work on it before then.  I think I’ve gotten it to a point where I can start letting alpha testers in to explore, and think about it.  My work log has been tracked on the thesis blog.

Beginning Alpha Testing with Thesis

Want to help alpha test or just look around?  Give it a try at https://galapag.us/login and see what you think!

Here’s the slide deck I’ll be presenting at thesis:

Stack

• Amazon EC2 small instance with ubuntu
• node.js/express.js: So easy to build a site using this framework.
• varnish/nginx+ngx_pagespeed: Caching, run-time optimizations for faster page loads/downloads.  Routes to https and socket.io server too.
• python scripts for maintaining server default state
• celery for queuing
• redis for temporary data dumps and lookups
• mongodb for permanent data storage
• angularjs for the comment system
• mocha, unittest, qunit for unit testing in python and javascript

I know the site’s confusing — like an airplane pilot dashboard.  It’ll become more cohesive over time.  A lot of things aren’t quite working yet, or they have filler data to get them going.  Apologies for that.  For more familiarization, try the welcome demo.

But here are some features that are worth checking out:

Comment System

Comments will be available for tribe forums, formula critiques, peoples’ profiles.  I decided to use angularjs so I could learn how to build SPAs with it!

Tribes

By tracking individual data, one can also track internal company metrics and state-level happiness metrics too!

The Islands

Each island has its own weather, environment, and bonuses/penalties for certain user behavior, so it benefits you to live on the island that incorporates your style best.

Professions and Skills

What does it mean to be “good” at something? Are companies hiring the most qualified candidates? How do we standardize that?

Your Genome

A profile for your data.  You get reputation scores in different areas.  Those scores are determined by which formulas you choose to use.  You can also see your internet of things (devices, pets, objects) is on the bottom right, while you’ll also be able to create gaming characters using your own data.

Quests

You can complete quests within Galapag.us to gain experience.  Some tasks will be data-gardening for other people, some will be to introduce gaming elements, others will be to visit lesser-seen parts of the site.  But mostly the quests should be geared towards helping others.

Status Bar

I love github’s command bar. I want users to be able to do most everything through the search bar.

Tribes and Their Genomes

Tribes are important to our identities. formulas serve as their DNA.

Universal Reputation Lookup

The left-side widget can be opened on most pages to see what reputations the people named on them have.  I intend to allow people to look up reputations from just about anywhere.

API/Sandbox

Galapag.us will have an API to access one’s data, as well as common stats such as state population census results, zodiac signs, and global stats.  Plus a place to test the routes, within the sandbox:

So with all that, please go help alpha test at https://galapag.us/login to begin your exploration of identity and reputation.  Thanks, and come to the ITP Spring Show if you can!

[FAKE] The Economist: A (Fake) Review of Galapag.us

Reputation and identity

The internet was a house of nil repute, until Galapag.us

An NYC startup is providing an ecosystem for people to build reputation metrics using any of their data

Forays into the online world have been fraught with peril, uncertainty, hucksters, intrigue, and irresistible indulgence of curiosities. Despite a need for truth-telling and fact-checking, social networking sites have been unable to establish reputation as a measure of quantifiable value to provide the unicorn of positive user experience.

However, a New York City-based startup would like to change that. Galapag.us, so named after the chain of islands made famous as Charles Darwin’s Eureka! moment for his theory of evolution, as well as for its American heritage and social emphasis (.us), has built up a loyal tribe of Galapag.users who promote its cause.

The tribe promotes openness of data and identity as a way to create measures for reputation, under the idea that being radically public will create more value for individuals and for society than governments, businesses, and nefarious individuals can take away through violations of privacy.

Surely other companies have attempted to reëvaluate reputation metrics, but as Ben Turner, the “discoverer” of Galapag.us says, “Companies either focus on low-hanging fruit from Foursquare and Facebook APIs, or they can only measure data in small spheres of peoples’ identities such as diet, exercise, or desktop PC productivity.” The strength of Galapag.us, Turner says, is in its ecosystem of data, variables, and formulae cutting across the spectrum of peoples’ identities, such as their professional progress, their hobbies, what they spend the fixed-limit twenty-four hours of the day doing, etc.

The other strength is indubitably its passionate tribe, made up of early adopters, quantified selfers, data science geeks, academic researchers (who have access to large-scale anonymized organization- or “tribe-” level data), and even recovering patients whose lives have been made quantified in order to survive their maladies.

For now, Galapag.us is closed only to members who vow to be open with their data, even though granular privacy controls with an “opt-in” only mentality are available. But Turner envisions a future where Galapag.us will provide reputations and identities to everyone — and every thing — on the planet, as a way to combat trafficking, bullying, undervaluation of good behavior, and other social problems.

The company can walk the walk, with an organic, thriving user base, as well as with its discoverer being a Texan and a former Army sergeant with an intelligence background, technical capability, and New York City Silicon Alley DNA. In an age where Internet users feel as though everything is being taken away from them, it is a breath of fresh air that Galapag.us is trying to return more value to people — and to society — than they need to provide on their own.

The number in parentheses is my 1-10 rating.  Books that are rated 10 are definite must-reads, but I think that anything above an 8 on this list this year was very, very interesting.  Having 7 out of 30 books rated “10″ means to me that I did a good job of picking books worthy of my time!

Past years here: google spreadsheet

2012 Goal: 30 BOOKS
TOTAL: 30 BOOKS

• (10) Washington: A Life – Ron Chernow
• (10) Reckless Road: Guns N’ Roses and the Making of Appetite for Destruction – Marc Canter
• (10) Growing Artificial Societies: Social Science from the Bottom Up – Joshua Epstein
• (8) The Plundered Planet: Why We Must–And How We Can–Manage Nature for Global Prosperity – Paul Collier
• (8) Brave Dragons: A Chinese Basketball Team, an American Coach, and Two Cultures Clashing – Jim Yardley
• (8) The Hunger Games – Suzanne Collins
• (7) Catching Fire – Suzanne Collins
• (6) Mockingjay – Suzanne Collins
• (6) Empires of Light: Edison, Tesla, Westinghouse, and the Race to Electrify the World – Jill Jonnes
• (7) The Giver – Lois Lowry
• (5) Gathering Blue – Lois Lowry
• (7) Messenger – Lois Lowry
• (9) The Fortune Cookie Chronicles: Adventures in the World of Chinese Food – Jennifer 8 Lee
• (7) Developing Backbone.js Applications – Addy Osmani
• (8) Drawing on the Right Side of the Brain – Betty Edwards
• (7) Creation: Life and How to Make It – Steve Grand
• (6) Interop: The Promise and Perils of Highly Interconnected Systems – John Palfrey and Urs Gasser
• (9) It’s All About the Bike: The Pursuit of Happiness on Two Wheels – Robert Penn
• (10) Born to Run: A Hidden Tribe, Superathletes, and the Greatest Race the World Has Never Seen – Christopher McDougall
• (9) True Colors: The Real Life of the Art World – Anthony Haden-Guest
• (10) Kill Decision – Daniel Suarez
• (7) The Glitch Moment(um) - Rosa Menkman
• (7) The Double Helix: A Personal Account of the Discovery of the Structure of DNA – James Watson
• (6) Biopunk: Solving Biotech’s Biggest Problems in Kitchens and Garages – Marcus Wohlsen
• (10) Goddess of the Market: Ayn Rand and the American Right – Jennifer Burns
• (10) Total Recall: My Unbelievably True Life Story – Arnold Schwarzenegger
• (8) Introduction to Java Programming – Y. Daniel Liang
• (7) The ARRL Ham Radio License Manual: Technician Level – H. Ward Silver
• (9) JavaScript: The Good Parts – Douglas Crockford
• (8) Stealth of Nations: The Global Rise of the Informal Economy – Robert Neuwirth

Gathering the Datasets

The data is unfortunately aggregated only at the state level, but it at least is updated regularly, so I ended up pulling 51 .csv (comma-separated values) files from the site with the publish date of October 2012.  The categories found in the data:

d_first_name, d_mid_name, d_last_name, d_suffix, d_birth_date, d_death_date, section_id, row_num, site_num, cem_name, cem_addr_one, cem_addr_two, city, state, zip, cem_url, cem_phone, relationship, v_first_name, v_mid_name, v_last_name, v_suffix, branch, rank, war

Since we would have to apply what we’d learned working with the R language to our dataset, what I hoped was that I could use the gravesite data, which goes back to the 1800′s or even earlier, to see how where veterans end up being buried correlates with national population trends over time.  In other words, if many Americans are buried in California, does this mean more veterans are also going to be buried there?

I figured, since there were categories for branch, rank, and war, that I’d be able to find some logical correlations: many privates and junior-enlisted sergeants would have died, while fewer senior-enlisted and officers would have, in past wars.  I figured the d_death_year might correlate with dates for the US’s multiple wars, with deaths elevated during those time periods.

So I guessed that with 51 data sets, this would begin to fill up my system RAM (4GB on this MacBook Air).  Look at my memory usage!

I skipped the files for the US-owned territories and “foreign addresses” since I wouldn’t be able to find normalized population data for those.  I also cleaned up the dataset so it would return only the veteran, not his/her beneficiaries who may also have been listed in the data as being deceased.

Loading the Data into R

Given that I’m not very comfortable with R, I started out just loading the Washington DC dataset since it only has 986 entries in it.  Problem?  The download for the file didn’t work.  “ngl_washington%20dc.csv” was not found.  %20 is a URL-encoded representation for a blank space.  Luckily, getting rid of the %20 revealed the proper filename, “ngl_washingtondc.csv”.  I also found that the .csv files were not importing into RStudio immediately.  I’d get an error.  I would get something similar to an uneven rows error.  What I had to do for each of the states’ .csv files was to open them up first in Excel and then save them in Excel.  Excel would properly format the files so they could be imported into RStudio.

I tried to write my R code in such a way that it could easily handle the 51 states + DC via functions, but I ended up having 51 calls for each state.  I wish the datasets were integrated into one national file.  I also wish I knew in R how to make a variable variable.  So that if I wanted to pass “Texas” to a row name, I could do “t <- “Texas” and then “state_data${t}” or something similar to convert on the fly to “state_data$Texas”.  In PHP, you might do this with $stateData{$t} (I think) and in JavaScript or Python you’d use eval().  Not pretty but I didn’t know how to do it properly.

The next step was to break up d_death_date (which was in a variety of formats such as “1993″, “9/3/98″, and “07/11/1864″) so that I could extract the year.  I had to check for the number of characters in the string, then figure out if the year was 2 digits or 4.  If it was 4 digits, I knew the year for sure (e.g. “2008″).  If 2 digits, I figured that if it were less than “15″, then it was probably referring to the year 2000 and higher (lazy data entry).  If higher than “15″, it probably was assumed to be the 20th century.  Finally, I had to convert this result from a string result to a numeric so I could do math on it.

More below the jump…

Alright, that was done.  Next I had to go pull up state-by-state population data going back to the 1800′s.  I figured this would be easy, but it was not.  I needed something like US census data going back to before 1800.  And it needed to be in a tabular format or something I could import into R easily.  The only thing I could find was a fixed-width text file version of census data from 1790 to 1990 on the New Jersey Department of Labor and Workforce Development web site.  How crazy is that?  For the year 2000 (and I also found 2010), I pulled in data from other sources for all the states.

To use this data, I had to paste it specially into Excel, selecting fixed-width instead of comma-separated.  I had to manually set the column widths so that the data would go into the appropriate columns.  Not very clean, but it worked!

Data Analysis

Okay, now I could begin comparing some data.  Very frustrating up till this point, and I wondered if I had bitten off more than I could chew in the time given (a few weeks).

I created a stats() function to run on each state, compiling different stats for each 50-year block.  I calculated the number of deaths per period but found that would not be very useful unless I did a per capita (dividing it by that year’s state population), so I did that too.  I also calculated the number of certain ranks per time period.  This was fraught with error too because my grepping for “CPL” (Corporal) would also count “LCPL” or Lance Corporal, a Marine rank.  But I figured the numbers didn’t require too much specificity unless I did deeper work on these datasets or found interesting conclusions from the results.

I also calculated the number of Army, Air Force, Marine Corps, and Navy deaths in 50-year blocks, as well as deaths for the periods of war.  The results of all this are below:

Total entries in mass dataset (all 51 files combined): 6,427,126

States sorted by most deaths in 2000 (using: rev(sort(table(bs$state[bs$death.year==2000]))) ):

Top 5: TX (18560), CA (17318), NY (15771), OH (13208), FL (11581)

TX CA NY OH FL PA IL MA MI NC MO TN MN GA VA AL
18560 17318 15771 13208 11581 11447 10202 9422 9271 8670 7996 7854 6592 6556 6366 5984

NJ KY IN WI OK LA MD SC AR MS CT OR WA WV IA AZ
5842 5776 5619 5426 5390 4730 4635 4453 4377 4060 4057 4021 3831 3795 3472 3377

CO KS ME NM NE NV NH RI SD ND MT HI ID UT DE VT
3206 2977 2238 1975 1686 1669 1576 1355 1270 1217 1217 1134 1086 1039 909 806

WY AK DC
416 269 30

States sorted by most deaths in 1950 (using: rev(sort(table(bs$state[bs$death.year==1950]))) ):

Top 5: CA, NY, MN, MO, NJ

CA NY MN MO NJ TX MD TN KY KS HI IL VA WI OH FL AR LA GA
3495 2846 710 679 674 642 583 564 331 289 267 253 245 226 223 195 188 158 152
IN NC AL OK MS OR NM SD PA WV CO WA MA SC IA MT AZ MI NE
130 110 92 90 90 85 72 69 65 60 60 44 42 38 38 37 36 29 24
ME AK CT VT UT NH WY RI ND ID DE DC
10 10 7 5 5 4 2 2 1 1 1 1

States sorted by most deaths in 1900 (using: rev(sort(table(bs$state[bs$death.year==1900]))) ):

Top 5: CA, OH, NY, VA, KS

CA OH NY VA KS WI GA IN IL AL TN TX ME KY PA MO MD NC AR
1123 320 303 237 198 152 138 132 104 100 94 93 89 81 73 72 67 66 62
LA SC MS NM FL MI WV IA CO NJ OK MN OR WA RI MT SD AK UT
51 46 46 41 29 25 21 17 12 11 10 10 9 8 8 5 4 4 3
NE MA DC VT
3 3 2 1

States sorted by most deaths in 1850 (using: rev(sort(table(bs$state[bs$death.year==1850]))) ):

Top 5: NY, PA, GA, OH, TN

NY PA GA OH TN DC IN WV VA KY AL WI MO IL NJ AR TX NC MS MN MI LA KS FL
26 14 12 11 6 6 5 4 4 4 4 3 3 3 2 2 1 1 1 1 1 1 1 1

States sorted by most deaths in 1800 (using: rev(sort(table(bs$state[bs$death.year==1800]))) ):

Top 5: NY, PA, NC, VA, OH

NY PA NC VA OH NJ MA TX SC NH KY GA DC CT
5 3 3 2 2 2 2 1 1 1 1 1 1 1

Veteran deaths in given year / US population in given year:

• 2000: 265734 / 281421906 = 0.0009443
• 1950: 13980 / 150,697,361 = 0.00009277
• 1900: 3873 / 76,212,168 = 0.00005082
• 1850: 117 / 23,191,876 = 0.0000050449
• 1800: 26 / 5,308,483 = 0.000004898

Total Deaths by Rank:

• SGT: 957556
• PFC: 813700 (using: length(grep(“PFC”, bs$rank)) )
• PVT: 783822
• CPL: 519318
• SSG: 153950
• MSG: 97652
• SFC: 66966
• COL: 60584
• MAJ: 33219
• LTC: 19106
• LCPL: 16882
• 1LT: 15591
• PV2: 14454
• CPT: 8071
• SPC: 7998
• 1SG: 7898
• 2LT: 7013
• GYSGT: 5354
• SGM: 4875
• CSM: 3231
• MGYSGT: 1012

Deaths during periods of war (but not necessarily as a result of the war):

• Civil War,  nrow(bs[bs$death.year >= 1861 & bs$death.year <= 1865,]):  143480
• World War I, nrow(bs[bs$death.year >= 1917 & bs$death.year <= 1920,]): 4023
• World War II, nrow(bs[bs$death.year >= 1940 & bs$death.year <= 1945,]): 97366
• Korean War, nrow(bs[bs$death.year >= 1950 & bs$death.year <= 1954,]): 76204
• Vietnam War, nrow(bs[bs$death.year >= 1964 & bs$death.year <= 1975,]): 375148
• Persian Gulf War, nrow(bs[bs$death.year >= 1990 & bs$death.year <= 1991,]): 104683
• OIF, nrow(bs[bs$death.year >= 2003 & bs$death.year <= 2011,]): 2412103
• OEF, nrow(bs[bs$death.year >= 2001 & bs$death.year <= 2013,]): 3104646

Deaths per Capita by State, 2000

Top: WV, ND, ME, SD, AR

West.Virginia North.Dakota Maine South.Dakota Arkansas
deaths.per.capita.2000 0.002040526 0.001800549 0.001678827 0.001549232 0.001495782
Massachusetts Oklahoma Mississippi Missouri Kentucky
deaths.per.capita.2000 0.001436358 0.001431652 0.001363221 0.001330122 0.001327631
Rhode.Island Vermont Alabama Minnesota Tennessee
deaths.per.capita.2000 0.00128406 0.001278681 0.001245893 0.001240292 0.001231917
Montana New.Hampshire Ohio Iowa Connecticut
deaths.per.capita.2000 0.001223834 0.001192634 0.001141722 0.001136949 0.001132725
Kansas Louisiana Delaware South.Carolina New.Mexico
deaths.per.capita.2000 0.001039523 0.001038656 0.001009017 0.000958464 0.0009553649
Wisconsin Michigan Nebraska North.Carolina Pennsylvania
deaths.per.capita.2000 0.0009522255 0.0009353662 0.0009203936 0.0009063557 0.0008988681
Indiana Hawaii New.York Maryland Illinois
deaths.per.capita.2000 0.0008642512 0.0008296375 0.0008120568 0.000800528 0.0007930425
Virginia Texas Wyoming Idaho Georgia
deaths.per.capita.2000 0.0007920141 0.0007345137 0.0007320077 0.0006901816 0.000673961
New.Jersey Colorado Nevada Florida Washington
deaths.per.capita.2000 0.0006632982 0.0006354895 0.0006159963 0.0006127263 0.0005672724
Arizona California Utah Alaska
deaths.per.capita.2000 0.0005266113 0.0004637675 0.0003749867 0.0003728225
District.of.Columbia
deaths.per.capita.2000 4.985683e-05

Deaths per Capita by State, 1950

Top: HI, NY, MN, MO, MD

Hawaii New.York Minnesota Missouri Maryland
deaths.per.capita.1950 0.0002203812 0.0001499753 0.0001443242 0.0001213538 0.000110073
Kansas California Tennessee South.Dakota Kentucky
deaths.per.capita.1950 0.0001074982 0.0001031836 9.913376e-05 9.140962e-05 8.189483e-05
New.Jersey Arkansas Wisconsin Montana New.Mexico
deaths.per.capita.1950 8.010126e-05 7.032244e-05 4.213529e-05 4.101109e-05 3.958119e-05
Louisiana Virginia West.Virginia Mississippi Texas
deaths.per.capita.1950 3.535486e-05 3.461178e-05 3.317953e-05 3.163825e-05 3.078868e-05
Oklahoma Indiana Alabama Illinois Ohio
deaths.per.capita.1950 2.608201e-05 2.137987e-05 2.068764e-05 2.037153e-05 1.964214e-05
Georgia Alaska Nebraska Colorado North.Carolina
deaths.per.capita.1950 1.856726e-05 1.595069e-05 1.402473e-05 1.39494e-05 1.366576e-05
Iowa Florida South.Carolina Vermont Maine
deaths.per.capita.1950 1.298558e-05 1.220094e-05 9.471557e-06 8.212514e-06 7.843611e-06
Washington Arizona Massachusetts Pennsylvania Wyoming
deaths.per.capita.1950 7.465066e-06 7.016679e-06 6.615114e-06 5.292705e-06 4.05037e-06
New.Hampshire Michigan Utah Connecticut Rhode.Island
deaths.per.capita.1950 3.236806e-06 2.917962e-06 2.238971e-06 2.055459e-06 1.907816e-06
District.of.Columbia North.Dakota Delaware Idaho Nevada
deaths.per.capita.1950 1.748071e-06 1.557147e-06 1.276161e-06 7.728256e-07 0

Deaths per Capita by State, 1900

Top: KS, ME, VA, CA, WI

Kansas Maine Virginia California Wisconsin
deaths.per.capita.1900 7.991689e-05 7.247982e-05 3.830391e-05 3.773519e-05 3.10726e-05
Ohio New.Mexico Arkansas Alabama Indiana
deaths.per.capita.1900 2.950093e-05 2.706147e-05 2.637484e-05 2.474888e-05 2.380884e-05
Kentucky Georgia Tennessee Mississippi New.York
deaths.per.capita.1900 2.197924e-05 2.130216e-05 1.927341e-05 1.787646e-05 1.684226e-05
Missouri Maryland South.Carolina Louisiana West.Virginia
deaths.per.capita.1900 1.407054e-05 1.401243e-05 1.319298e-05 1.208539e-05 1.17091e-05
North.Carolina Illinois Rhode.Island Alaska Montana
deaths.per.capita.1900 9.956798e-06 9.098383e-06 7.972384e-06 7.272159e-06 6.257313e-06
Pennsylvania Iowa South.Dakota Texas Colorado
deaths.per.capita.1900 6.143931e-06 6.122254e-06 5.747093e-06 5.474933e-06 3.642552e-06
District.of.Columbia Oklahoma Michigan Minnesota
deaths.per.capita.1900 3.295436e-06 3.179059e-06 2.689532e-06 2.285663e-06
Florida Nebraska Vermont Utah Washington
deaths.per.capita.1900 2.241472e-06 1.900677e-06 1.776963e-06 1.741301e-06 1.643827e-06
New.Jersey Massachusetts Wyoming North.Dakota New.Hampshire Nevada
deaths.per.capita.1900 1.422993e-06 4.98635e-07 0 0 0 0

Deaths per Capita by State, 1850

Top: DC, GA, WV, TN, NY

District.of.Columbia Georgia West.Virginia Tennessee
deaths.per.capita.1850 7.479637e-06 3.483736e-06 1.994463e-06 1.822756e-06
New.York Ohio Kentucky Pennsylvania Alabama
deaths.per.capita.1850 1.75318e-06 1.384235e-06 1.358324e-06 1.333586e-06 1.306445e-06
Indiana Virginia Arkansas Wisconsin Missouri
deaths.per.capita.1850 1.270899e-06 1.205298e-06 1.047389e-06 8.734705e-07 7.586001e-07
Kansas Mississippi New.Jersey Louisiana Florida
deaths.per.capita.1850 5.24852e-07 4.589442e-07 4.136223e-07 3.726454e-07 3.608408e-07
Illinois Minnesota North.Carolina Michigan Texas
deaths.per.capita.1850 3.443457e-07 3.352911e-07 2.461884e-07 1.569424e-07 1.296816e-07

Deaths per Capita by State, 1800

Top: DC, NH, NC, CT, VA

District.of.Columbia New.Hampshire North.Carolina Connecticut
deaths.per.capita.1800 3.587856e-06 2.429614e-06 1.584108e-06 1.100812e-06
Virginia New.Jersey South.Carolina Massachusetts New.York
deaths.per.capita.1800 1.078642e-06 1.061758e-06 7.460927e-07 7.129245e-07 6.878626e-07
Ohio Pennsylvania Kentucky Georgia Texas
deaths.per.capita.1800 4.810531e-07 4.760307e-07 4.657284e-07 4.511961e-07 3.280076e-07

Army Deaths by State, 2000

Top: TX, NY, CA, OH, PA

Texas New.York California Ohio Pennsylvania Illinois Florida Michigan
pop.army.2000 11537 10412 9296 8756 7655 6844 6575 6239
North.Carolina Massachusetts Missouri Tennessee Georgia Minnesota Kentucky
pop.army.2000 5720 5330 5231 5218 4276 4189 4132
Virginia Alabama New.Jersey Indiana Wisconsin Oklahoma Maryland Louisiana
pop.army.2000 4070 4017 3743 3674 3615 3485 3072 3011
South.Carolina Arkansas Mississippi Connecticut West.Virginia Iowa Washington
pop.army.2000 2878 2822 2631 2597 2569 2237 2064
Kansas Arizona Colorado Maine New.Mexico Nebraska New.Hampshire North.Dakota
pop.army.2000 1918 1852 1809 1371 1224 1055 901 860
South.Dakota Nevada Hawaii Rhode.Island Montana Idaho Utah Delaware Vermont
pop.army.2000 858 856 793 756 739 650 590 553 526
Wyoming Alaska District.of.Columbia X1.36
pop.army.2000 256 166 19 15

Army Deaths by State, 1950

Top: CA, NY, NJ, MO, TX

California New.York New.Jersey Missouri Texas Minnesota Tennessee Kentucky
pop.army.1950 2395 2185 568 557 533 480 444 267
Illinois Kansas Ohio Wisconsin Hawaii Arkansas Maryland Virginia Louisiana
pop.army.1950 218 214 197 196 184 150 147 145 140
Florida Indiana Georgia North.Carolina Alabama Oklahoma New.Mexico Mississippi
pop.army.1950 139 112 83 79 71 60 53 44
Colorado South.Dakota Iowa Pennsylvania Massachusetts South.Carolina
pop.army.1950 43 41 34 27 26 25
West.Virginia Michigan Nebraska Arizona X1.36 Washington Maine Montana
pop.army.1950 24 24 17 17 16 12 9 7
Connecticut Vermont Alaska Utah New.Hampshire Wyoming Rhode.Island Idaho
pop.army.1950 6 5 4 3 3 1 1 1
Delaware District.of.Columbia North.Dakota Nevada
pop.army.1950 1 0 0 0

Army Deaths by State, 1900

Top: CA, VA, NY, GA, WI

California Virginia New.York Georgia Wisconsin Kansas Indiana Alabama Texas
pop.army.1900 971 187 148 138 130 115 100 96 90
Maine Kentucky Missouri North.Carolina Tennessee Arkansas Pennsylvania
pop.army.1900 79 71 64 62 61 56 54
Illinois South.Carolina New.Mexico Ohio Mississippi Florida Louisiana Michigan
pop.army.1900 45 39 38 34 32 29 27 23
West.Virginia X1.36 Iowa Colorado New.Jersey Minnesota Rhode.Island Oklahoma
pop.army.1900 19 17 16 12 10 10 8 7
Washington Massachusetts Maryland Alaska Nebraska Vermont Utah South.Dakota
pop.army.1900 4 3 3 3 2 1 1 1

Army Deaths by State, 1850

Top: OH, GA, NY, WV, TN

Ohio Georgia New.York West.Virginia Tennessee Kentucky Alabama Wisconsin
pop.army.1850 11 9 8 4 4 4 4 3
Missouri Illinois Virginia Pennsylvania Indiana Arkansas Texas North.Carolina
pop.army.1850 3 3 2 2 2 2 1 1
New.Jersey Mississippi Louisiana Wyoming District.of.Columbia Washington
pop.army.1850 1 1 1 0 0 0

Army Deaths by State, 1800

Top: NY, NJ, MA, TX, SC

New.York New.Jersey Massachusetts Texas South.Carolina North.Carolina
pop.army.1800 2 2 2 1 1 1
Kentucky Georgia Connecticut Wyoming Wisconsin West.Virginia
pop.army.1800 1 1 1 0 0 0

Air Force Deaths by State, 2000

Top: TX, CA, FL, NY, OH

Texas California Florida New.York Ohio Pennsylvania Illinois Massachusetts
pop.airforce.2000 3557 2899 2104 2098 1749 1502 1288 1215
Missouri Michigan Tennessee North.Carolina Georgia Minnesota Oklahoma
pop.airforce.2000 1143 1122 1028 993 967 934 896
Alabama Wisconsin Virginia New.Jersey Colorado Arizona Louisiana Indiana
pop.airforce.2000 842 774 759 740 706 694 692 668
South.Carolina Kentucky Washington Arkansas Maryland Mississippi
pop.airforce.2000 624 624 606 576 561 546
Connecticut Kansas Iowa West.Virginia Nevada New.Mexico Maine Nebraska
pop.airforce.2000 474 460 432 406 355 346 292 262
New.Hampshire Idaho Montana Rhode.Island Utah South.Dakota Hawaii Delaware
pop.airforce.2000 243 187 180 174 172 167 152 148
North.Dakota Vermont Wyoming Alaska X1.36 District.of.Columbia
pop.airforce.2000 137 110 73 46 20 4

Air Force Deaths by State, 1950

Top: CA, TX, MO, NY, MN

California Texas Missouri New.York Minnesota X1.36 Tennessee Kentucky
pop.airforce.1950 151 58 58 41 26 21 20 19
New.Jersey Illinois Virginia New.Mexico Kansas North.Carolina Indiana
pop.airforce.1950 17 9 8 7 7 6 6
Florida Alabama South.Dakota Oklahoma Louisiana Wisconsin Pennsylvania
pop.airforce.1950 6 6 5 5 5 4 4
Georgia Arkansas Michigan Hawaii Ohio Mississippi Massachusetts Colorado
pop.airforce.1950 4 4 3 3 2 2 2 2
West.Virginia Maryland Iowa Wyoming District.of.Columbia Washington
pop.airforce.1950 1 1 1 0 0 0

Air Force Deaths by State, 1900

Texas California
pop.airforce.1900 1 1

Navy Deaths by State, 2000

Top: CA, TX, NY, FL, MA

California Texas New.York Florida Massachusetts Ohio Pennsylvania Illinois
pop.navy.2000 4798 3646 3332 2893 2745 2690 2345 2123
Michigan North.Carolina Missouri Minnesota Tennessee Virginia New.Jersey
pop.navy.2000 1875 1740 1660 1604 1571 1382 1299
Georgia Indiana Alabama Oklahoma Wisconsin Washington Arkansas Kentucky
pop.navy.2000 1256 1206 1168 1115 1114 1084 976 974
Connecticut Louisiana South.Carolina Maryland Iowa Mississippi West.Virginia
pop.navy.2000 962 956 920 899 811 804 789
Arizona Colorado Kansas Maine New.Hampshire New.Mexico Rhode.Island Nevada
pop.navy.2000 761 737 639 520 416 414 412 411
Nebraska Montana Utah Idaho South.Dakota North.Dakota Delaware Vermont Hawaii
pop.navy.2000 362 291 264 255 253 210 193 177 152
Wyoming Alaska X1.36 District.of.Columbia
pop.navy.2000 87 52 25 7

Navy Deaths by State, 1950

Top: CA, NY, NJ, MN, MO

California New.York New.Jersey Minnesota Missouri Tennessee Florida Texas
pop.navy.1950 653 395 78 68 53 33 31 28
Kentucky Maryland Wisconsin Hawaii Kansas Pennsylvania Illinois Virginia
pop.navy.1950 25 20 19 19 16 15 14 12
New.Mexico Ohio Alabama Massachusetts Indiana North.Carolina South.Dakota
pop.navy.1950 12 11 10 9 9 8 7
South.Carolina Arkansas Georgia Oklahoma Louisiana Colorado Arizona
pop.navy.1950 7 7 6 5 5 5 5
West.Virginia Iowa Nebraska Mississippi Alaska Washington Rhode.Island
pop.navy.1950 3 3 2 2 2 1 1
North.Dakota Montana Michigan Maine Connecticut Wyoming District.of.Columbia
pop.navy.1950 1 1 1 1 1 0 0

Navy Deaths by State, 1900

Top: CA, NY, MD, VA, PA

California New.York Maryland Virginia Pennsylvania Maine Ohio Kansas
pop.navy.1900 34 31 12 9 9 8 7 6
Wisconsin Indiana Illinois South.Carolina North.Carolina Michigan New.Jersey
pop.navy.1900 5 5 4 3 2 2 1
Missouri Mississippi Iowa Alabama Wyoming West.Virginia District.of.Columbia
pop.navy.1900 1 1 1 1 0 0 0

Navy Deaths by State, 1850

Virginia Florida
pop.navy.1850 1 1

Marine Corps Deaths by State, 2000

Top: CA, TX, NY, OH, FL

California Texas New.York Ohio Florida Pennsylvania Illinois Massachusetts
pop.marines.2000 1346 1109 949 782 731 666 606 571
North.Carolina Michigan Missouri Virginia Tennessee Minnesota New.Jersey
pop.marines.2000 548 539 476 394 391 367 364
Indiana Georgia Wisconsin Louisiana Kentucky Maryland Arizona Oklahoma
pop.marines.2000 336 331 324 282 282 273 269 265
Connecticut Alabama South.Carolina Arkansas Washington Mississippi Iowa
pop.marines.2000 253 252 237 220 218 215 197
West.Virginia Colorado Kansas Maine New.Mexico Nevada Nebraska
pop.marines.2000 193 191 157 132 117 114 105
New.Hampshire Montana Idaho Utah Rhode.Island North.Dakota South.Dakota
pop.marines.2000 99 82 81 75 70 66 61
Hawaii Delaware Vermont X1.36 Wyoming Alaska District.of.Columbia
pop.marines.2000 58 48 39 30 29 16 2

Marine Corps Deaths by State, 1950

Top: CA, NY, HI, TX, MN

California New.York Hawaii Texas Minnesota X1.36 Missouri Tennessee
pop.marines.1950 172 70 54 34 33 31 27 22
New.Jersey Illinois Florida Virginia Ohio Louisiana Kentucky South.Carolina
pop.marines.1950 21 14 10 6 6 6 6 5
Pennsylvania Maryland Colorado South.Dakota Kansas Arkansas Alabama
pop.marines.1950 5 5 5 4 4 4 4
Wisconsin Oklahoma Indiana Massachusetts Georgia West.Virginia New.Mexico
pop.marines.1950 3 3 3 2 2 1 1
New.Hampshire Nebraska Mississippi Wyoming District.of.Columbia Washington
pop.marines.1950 1 1 1 0 0 0

Marine Corps Deaths by State, 1900

Top: CA, NY, WI, DC, VA

California New.York Wisconsin District.of.Columbia Virginia
pop.marines.1900 17 4 3 2 2
South.Dakota South.Carolina Ohio Alaska Wyoming West.Virginia Washington
pop.marines.1900 1 1 1 1 0 0 0

Wars served in by veterans deceased:

• WORLD WAR II: 2873227
• KOREA: 768669
• VIETNAM: 634830
• WORLD WAR I: 350223
• WORLD WAR II, KOREA: 175399
• CIVIL WAR: 163817
• KOREA, VIETNAM: 94051
• WORLD WAR II, KOREA, VIETNAM: 88833
• SPANISH AMERICAN WAR: 28906
• PERSIAN GULF: 25501
• WORLD WAR II, WORLD WAR I: 9497
• PERSIAN GULF, VIETNAM: 8755
• CONFEDERATE STATES: 5192
• IRAQ: 4826
• WORLD WAR II, VIETNAM: 3630
• REVOLUTIONARY WAR: 3205
• WAR OF 1812: 1723
• IRAQ, PERSIAN GULF: 1687
• MEXICAN BORDER, WORLD WAR I: 1488
• AFGHANISTAN: 1144
• WORLD WAR II, KOREA, WORLD WAR I: 1023
• INDIAN WARS: 882
• MEXICAN WAR: 844
• MEXICAN BORDER: 706
• SPANISH AMERICAN WAR, WORLD WAR I: 550
• AFGHANISTAN, IRAQ: 497

Whew!  Okay, so that’s most of the results I gathered.

Conclusions

I was not happy with my results.  First of all, the results are confusing because my data is a poor proxy; are these veterans being buried in certain places because they were stationed in that state (hence, Marines and Navy would more likely be buried in coastal states — according to the stats, California has the most deaths for both the Marines and Navy) or because they grew old and died, or what?

Doing time periods wasn’t reliable for tracking results from war.  Some veterans would live longer than others, after all.  Besides, from looking at the basic data, it appeared as though the statistical reporting of the data was weak up until even the 90′s.  So you can imagine the deceased population on a graph from 1800 to 2010 as being a graph that spikes from low bar levels to very high ones in the 90′s (I assume statistical reporting greatly improved at around the time of World War II, so these boomers’ deaths in the latter 20th century were captured on record) and not a smooth curve or at least a normal population growth curve.

I thought it was interesting that deaths by capita in 1950 had New York in the top 5 states.  Usually death by capita is dominated by states with small populations.  But NY was in the top 5 in 1950?  It’s one of the most populous states now and has been throughout the US’s history.  Was this a reflection of NY’s economic success after WW2, or maybe its dedication to the American cause?

Deaths by rank would take some analysis.  Naturally PFCs and SGTs (my rank) died more than anyone else.  Enlisted ranks tended to have higher numbers.  But colonels and majors died a lot more than 1st Lieutenants and 2nd Lieutenants.  This was because officers have a higher life expectancy and are (probably) less likely to die in war and someone is not very likely to die young as an officer before he’s promoted at least to Captain or Major.  We junior enlisted had very short life expectancy.

To be honest, I was most interested in just state population numbers over time.  They remind me just how young the US is!  It’s crazy that California went from like 95k people in 1850 to 3 million by 2000!  North Dakota’s population has not changed that much since its inception.

But without further and deeper analysis, a lot of what I wrote above is not statistically proven.  The data is messy and my handling of it was sloppy (grep’ing for CPL vs. LCPL, “ARMY” capturing “ARMY AIR CORPS” (the Air Force used to be part of the Army), etc.).  No numbers really stood out to me in my analysis.  But this was excellent practice at writing code in R — so at least something productive came out of this!  I put all my R code and collected data and other scraps in my github repo for this final, available here.

I think there are some incorrectly geolocated keywords because I increased the browser size to 5k pixels across 3 screens and maybe the tag locations weren’t updated.

The Trendsmap blog also has some cool viz’s of world cities:

Easy:

And less obvious:

Here’s some shit that I’ve seen while at ITP that I thought was fucking awesome, not necessarily because it’s never been done before, but primarily because it’s so easy for any regular person to play with now.

Interactive Coding

From Dan Shiffman in his Nature of Code class, he passed along this link from Bret Victor‘s talk, “Inventing on Principle”.  In the lengthy vid (all of it worth watching), he shows real-time feedback for coding.  That is, if you change the logic in your code, you see how the variables change, in real-time:

Click here to view the embedded video.

He extended this to showing a circuit diagram timeline with sync’d waveforms and how the electrons and flow through the circuit as you change resistance and parts:

Click here to view the embedded video.

Then, he showed iPad swiping to change elements on an animation timeline, thus creating a sort of tangible real-time animation experimentation, seen below:

Click here to view the embedded video.

The benefit of this immediate feedback is that one can begin to play.  Instead of usual software development, which consists of planning ahead of time exactly how something should behave, and developing contingencies for when things go unexpectedly, this sort of format allows for someone to play with variables, such as the size of a character’s head, or the physics of a world, with immediate results.  This allows someone to fine-tune a world, or to test the bounds to see if unexpectedly fun behavior emerges from it.

This is more consistent with how an artist might try multiple things in order to fully flesh out a concept, instead of hoping to get lucky with it, or spending ages understanding the mechanics so well that the result is contrived.

Here’s the full version of Victor’s talk:

Click here to view the embedded video.

You can see an early application of this kind of coding mentality in, as an example, this indie game, Under the Ocean:

Click here to view the embedded video.

Websockets

One of my classmates, now an ITP resident, Craig Protzel, was showing me some code he was working on with a professor, linking up heartbeat monitor-type data with a data viz timeseries graphing web site, showing real-time streaming of the data onto a line graph, with a backend of node.js and socket.io.

The best demo I could find of something similar to this is this Arduino board with two potentiometers streaming output via websockets and a Python script up to the web:

Click here to view the embedded video.

Here’s the implication.  The web has been fairly static since its inception.  Even when AJAX came and ushered in web 2.0, you were still doing with an active getter-type web.  Databases, bandwidth, and client browsers just couldn’t handle unrequested data coming in.  But now they can, what with the cloud, key-value stores like redis or AMQP, faster and bigger bandwidth pipes, etc.  The web is going to start looking more like a stream and less like a restaurant menu.

In my Redial class last semester, a lot of our final projects involved setting up an open-source Asterisk telephony server with a cheap phone number routed to an Ubuntu server instance in the cloud — that stack was all the same, but our applications were different: one team (Phil and Robbie) made a super-easy conference call service, another dude (Tony) made a multiplayer sequencer controlled by people dialing in and punching numbers on their keypads:

Click here to view the embedded video.

Server stacks are flattening in a sense — you can use any language you want to set up a server (Ruby Sinatra/Rails, JavaScript node.js, Python Flask), and then plug in extra services you need (database, key-store, admin tools, task and load balancers).  HTML5 and some degree of normalization on the browser side is allowing JavaScript to mature so that we have all these kickass visualization and interface libraries for making better user interfaces now, too, which can easily handle the structured data being thrown at it by all the stuff going on on the backend.

The last note is that you can now easily, with a little help from an Arduino and a network shield, control the digital world with analog sensors, or vice versa.

Drones

I read Daniel Suarez’s latest book “Kill Decision” (review here), in which a scientist is targeted because of her research into weaver ants, the most warlike species on Earth, being inspiration for algorithms for killer drone swarming behavior.  We’re living in an age of the dawn of drones, where the US has found it can cheaply deploy drones to kill and monitor the enemy, broken down into a bureaucracy of various levels of kill and targeting authorization.  We’ve all seen the videos of quadcopters acting together using simple rules.  It won’t be too long until law enforcement and federal agencies will be able to use drones domestically.  Drones are far more versatile, expendable, and cost-effective as overhead imagery.  Look at the quality on this RC with a GoPro camera attached:

Click here to view the embedded video.

Here was a “robokopter” sent up to monitor the police in Warsaw as they kept two hostile groups of marchers apart:

Click here to view the embedded video.

I can’t see how drones won’t be banned soon, but how will they enforce it?  Shoot down rogue drones?  Jam them?

Somewhat corny, but here’s FPSRussia using what seems to be a terribly unsafe quadcopter with a machine gun attached:

Click here to view the embedded video.

Suarez’s book takes its name from the idea that an unknown actor has built drones that target individuals, assassinate them, and self-destruct, without any instructions coming in after they’re released.  They destroy their own fingerprint and are mostly impervious to being jammed or tracked back to their makers.

All with fairly cheap parts.  It’s not the same thing as weaponizing, say, biological weapons, or building a big enough EMP, which I would imagine are two of security apparatuses’ biggest fears.  I’m kind of sad more people haven’t read Suarez’s books because they’re addressing pretty near-term implications of emerging tech.

FaceShift

I didn’t get to take this class, but Professor Kyle McDonald (whom I took for Glitch) had his students play with this new kickass software, FaceShift.  Check the demo:

Click here to view the embedded video.

Basically, you plug in a Kinect to your computer and spend half an hour mapping your facial expressions to the software.  Then it renders a model of your face, which you can then use to map any skin you want (say, another person’s face) onto it.  So quickly and easily you can do this.  Previously this sort of work was the domain of special effects studios and game development teams.  Now it’s downloadable, and you can buy a Kinect (or other similar cameras) to capture yourself.

Holy Grails That are Still Missing

Self-Recording

Here’s what I wrote my buddy Chris after he showed me this upcoming product, the Autographer:

Problem with it is threefold:

It also means that photos of people and recording peoples’ lives has been primarily a solo adventure at this point.  Hence the phenomenon of mirror photos, the forward-facing camera (so you can see yourself and the person you’re with while you take the photo), etc.  Not so many people are lucky enough to A) want photos and B) have someone else who loves to take photos along with them at that time.  I have no awesome photos of Iraq as a result (and those that I took, as you know, got me in a shitload of photos…can’t even claim to have returned with anything beautiful from that hard-knocks lesson).  Maybe this also explains why photos of animals have done so well.  They are ignorant of us taking photos of them, and do their crazy animalistic shit with reckless abandon, and thus make excellent photo subjects.

Extend that to personal data collection (which is what Galapag.us will start off as) and it’s a somewhat isolating experience.  Who is going to follow you around and collect data on you?  You have to do it for yourself, or participate in activities that can be tracked automatically (marathons, online social networking sites, Fuelband, etc.).  Maybe social media whores (like me) became that way because that’s the cutting edge for living a quantified, recorded life.

Anyway, I think it’s pretty fascinating that among friends who are social users of online stuff, Facebook and Instagram are the key players (which is why Facebook paid $1bil for Instagram).  I love Twitter but very few casual users use it.  Pinterest is primarily women, fantasy leagues are men, etc.  But photos are HUGE.  Facebook knows it.  But we’re not even a third of the way towards reaching the full potential of capturing the human experience through a camera in my opinion.  The tech is not there yet.

Decentralization

The web is not very distributed or decentralized.  There is a myth of digital democracy.  When Amazon AWS has a hiccup, usually in its Virginia availability zone, half the American internet’s most popular sites go down.  The NSA and other countries’ intelligence agencies are up the telcos’ asses with eavesdropping, and sites are being shut down by ICE and the FBI.  Virtually the only site that has remained impervious to government attacks has been The Pirate Bay, which is constantly coming up with new ways to thwart efforts to shut it down, primarily through redundancy and distribution.  Torrenting has almost become a political act, even though it’s a resilient model for an unevenly distributed modern-day internet.

Social networks are walled gardens.  Twitter, a darling, was caught by the more traditional walled garden peloton, and is now locking down its data, after having, at one point, a role model API.

At some point we will have IPv6, which, thankfully after Windows has gone through some lengths to secure its OS, has been slow to roll out so far, but which will eventually allow any sensor, device, appliance, whatever to have its own internet-available unique ID, for better or for worse.

In Closing

I decided to make a puzzle of it, to see if my idea would work.  First I had to put a file online, shorten it via bit.ly (to reduce encoding complexity in the QR code), and then generate a QR code for it.  Here’s the summation of that:

In order to do my flip-flop, one would need to graph out the x,y grid coordinates from a txt file onto graph paper. (ANALOG) Check out the text file here.  0,0 is the top-left of the 25×25 grid.

You can see it’s a 25×25 grid.  Once you draw out the pixel coordinates, you’ll get what appears to be a QR code.  Here’s my work, partially through it:

I wasn’t sure if this part would work, though in theory it should have.  When I first filled in the QR code on my graphing paper, neither Google Goggles nor QR Droid could detect the QR code.  Google Goggles found the closest match to be photo images of bathroom tile patterns.

Then, figuring my drawn QR code wasn’t precise enough, or perhaps dark enough, I erased my axis labels and filled in the square pixels with a black pen.  To my amazement, the QR code then worked in QR Droid! (DIGITAL)

I was surprised that it could pick up the code, because I started getting sloppy while filling in the squares, figuring I could just use the final product as some artistic hand-drawn rendering of a QR code.  Apparently though, QR codes, depending on encoding, can handle quite a lot of error, and correct for it.  Here’s some information about how its error correction works (thanks Neil).

One interesting project handling data error and error correction was our professor Kyle McDonald’s Future Fragments project, where he had other classmates encode messages into colored grid squares, then keep the printouts in their wallets for a summer, then decoding the blocks back into a message after the pieces of paper had been worn down a bit.

The QR code takes you to a bit.ly link, which forwards you to an image on my site, a reproduction of Caravaggio’s The Card Sharps, one of my favorite paintings. (DIGITAL)

• bitly link: http://bit.ly/R5smVW
• Caravaggio’s “The Card Sharps”, benturner.com

The next task would then be to hand-draw the painting. (ANALOG)

Then you take a photo of it to auto-save to Google+ or upload to the cloud. (DIGITAL)

Then the final task would be to color in the drawing using Photoshop or another digital tool. (DIGITAL)

Here was my final product:

As a final test, I thought I’d see if I could run the drawings through glassgiant.com’s ASCII converter.  The results for my drawing were not too good, probably mainly because I did not make bold enough lines and outlines to make the ASCII conversion stark enough.  I also ASCII’d up the QR code (ink) and the original Caravaggio. (DIGITAL)

Note: along the way, I found this awesome site that has a QR code stencil generator, for making stencils in hobo code!

Anyway, that concludes my work for Glitch class.  Thank you Kyle McDonald and Jeremiah Johnson!  This ITP class was fucking kickass.

So, the first step is to set up a way to access the JSON object.  Right after the spring semester finished up, I had some free time before work started and so I began to port my PHP+MySQL Galapag.us code over to node.js+Express+MongoDB.  I still have a lot of work to do on that, but I can’t really work on that till next semester (AKA thesis semester).  Now, of course, because we use it at work, and there’s a class at ITP teaching it this semester, I’m interested in switching over to Python Flask+MongoDB, but I’m worried I might lose some time figuring out the quirks of that, versus just doing the quick-and-dirty with JavaScript.

Anyway, I’ve already set up some basics for a Galapag.user’s profile database model.  And for Nature of Code class last semester, I built a quick node.js Express server for sending a JSON object of a person’s characteristics (1-10 scales for creativity, strength, charisma, etc.) to a Processing sketch for my genetic crossings project.

So I combined the two models to create an expanded Galapag.user profile, which looks something like this below:
{ "status": "OK", "JSONtitle": "Profile Summary", "profile": [{ "_id": "xxx", "adjectives": "", "appearance": 10, "armSize": "38R", "bio": "test me", "bodyType": "average", "broken": "femur", "caffeine": "coffee", "charisma": 8, "chewNails": "never", "children": 0, "chineseSign": 2, "city": "Des Moines, IA", "coatSize": "38L", "creativity": 3, "discipline": 6, "dressSize": "N/A", "drink": 1, "drugs": 0, "education": 6, "entrepreneurialism": 1, "eyeColor": "brown", "eyewear": 0, "facebook": "", "facialHair": "none", "fillings": 1, "flickr": "", "formulaEducation": "", "foursquare": "", "freckles": "back", "gender": 1, "google": "", "gracefulness": 8, "hairColor": "blond", "hairLength": "short", "hairStyle": "shaved", "health": 6, "height": 66, "homeTown": "Des Moines, IA", "honesty": 3, "humor": 6, "income": 120000, "injuries": "", "instagram": "", "intelligence": 5, "legSize": "30", "linkedin": "", "liquidityInteraction": "", "loggedOnTotal": "", "luck": 8, "maritalStatus": "single", "money": 7, "nationality": "USA", "neckSize": "16 1/4", "occupation": "analyst", "personalityType": "ENTP", "pid": 99, "piercings": 0, "politics": "Republican", "prosthetics": "none", "race": 1, "religion": "Christian", "religiosity": 6, "responsibility": 6, "scars": 1, "sexuality": 0, "shoeSize": "9", "sign": 1, "smoke": 1, "stamina": 5, "strength": 6, "strengthFriend": "", "stress": 6, "suggestedBy": "", "surgeries": "", "talent_art": 2, "talent_math": 6, "talent_sports": 8, "tattoos": 1, "totalBooks": "", "totalCostWardrobePerson": "", "totalInteractions": "", "totalTransactions": "", "totalWardrobe": "", "trustBusiness": "", "trustFriendship": "", "twitter": "", "waistSize": "32", "website": "", "weight": 175, "wisdom": 3, "wit": 5, "xbl": "", "youtube": "", "zipCode": 85083, "claimed": true, "audited": false, "firstTime": false, "active": true, "joined": "xxx" }] }

To visualize the data, there are a couple things I’d like to focus on.  One, I don’t want it to be just another widget you put on your site (besides, does anyone have sites anymore?), and two, I want to emphasize the non-financial benefits of reputation, which is to say, Galapag.us would be great for dating or hiring, but it should really stick to a core of providing an alternative model of judging trust, reputation, and worth through non-financial actions such as being a good friend, a good citizen, self-sacrificing (or not, depending on your opinion of altruism), whatever.

The point is that everyone has his own opinion of what makes someone else valuable, and Galapag.us needs to be a system that does not favor one system over another.  Granted, I think Galapag.us will have its own core values, which I’m hoping will be built upon the company’s DNA, of its founders, but it will also allow for alternate models created by, for example, the average expectations of the entire Galapag.userbase, or hopefully, the aggregate beliefs of different regions, countries, cultures, etc.

Anyway, back to the assignment.  DNA transcription, translation, and replication are essentially protocols for the secure passing of information.  Biology has created a highly reliable process for this.  Humans have developed less reliable processes for the passing along of cultural information, less reliable because the processes rely on generations of humans adhering to the cultural norms and traditions and taking the time to teach it to the next generation.  But look what beautiful things humans have done to pass along generations of information quickly:

Maori facial tattoos (ta moko) were often used to denote one’s place in society, based on positional rules.  From this blog post:

 For men, the Moko showed their rank, their status and their ferocity. The which is generally divided into eight sections :
1. Ngakaipikirau (rank). The center forehead area
2. Ngunga (position). Around the brows
3. Uirere (hapu rank). The eyes and nose area
4. Uma (first or second marriage). The temples
5. Raurau (signature). The area under the nose
6. Taiohou (work). The cheek area
7. Wairua (mana). The chin
8. Taitoto (birth status). The jaw

Something I’m more familiar with, American military uniforms, which allow soldiers who have never seen each other to immediately know someone else’s rank, achievements, stature, etc.  Here’s General Petraeus:

American children are primed for the military uniform (which is now full of symbology invisible to most Americans because of the small percentage of people who actually serve in it) through their Boy/Girl Scout uniforms:

These are individual displays of conformity into cultural systems and achievement within them.

But we also have ways of remembering those we’ve lost and loved.  Take, for example, the American quilt, which has traditionally been used to record family histories or American history, …

…but which not too long ago was used to create a massive remembrance of HIV/AIDS victims on the National Mall:

So how can I encode profile information in such a way that it’s visually appealing, culturally useful, and statistically informative?  To be honest, I have no clue.  It’s not something I’m strong at.  What I’m hoping is to have a few different options available, but to also just put up the API for proper designers to build their own interpretations, something that’s more in keeping with an open-ended system.  Yes, it is likely that no one at all would ever use Galapag.us, particularly any developers, but I think what’s important is to build a simple API for people (including internally at Galapag.us) to interact with.

Here are some visualizations I already made in the past:

Click here to view the embedded video.

So if I were going to come up with a new visualization that could be displayed in various mediums (on clothing, on sites, badges, business cards, etc.), there doesn’t necessarily need to be any order or logic if I am looking to create something cool aesthetically with the data, but it could help, with minimal cost.

In thinking about Galapag.us and a holistic reputation system, I thought that one would need to find universal constraints.  The first constraint I thought applied to all things, and people, rich and poor, good and bad, was time.  As Arnold Schwarzenegger and other motivators would say, there are only so many hours in a day in order to be great.  No matter how important or unimportant we are, we all only have 24 hours a day to do things.  How do we use that time while we’re alive?  How much time have we invested into different aspects of our lives?

At the same time, there are qualities and characteristics about people and things that require no time to develop or improve or grow into.  It doesn’t take you time to develop kindness, or honesty, or whatever.  You either are at any particular point in time, or you’re not.  These are binary instances, which may vary over time (most people go through periods of both).  If you took a snapshot at any point in time, you would be either one or the other.  Contrast this (and all its inherent inconsistencies) with, say, having gone through college.  It takes 4 years, usually, to get a degree.  Being a college graduate is something that took time to develop.

So time is one dimension. The other one, for better and for worse, is money.  What is your opportunity cost of spending x hours doing something?  If you become a surgeon, you have to spend maybe 6 years and a ton of money for med school.  But afterwards you end up healing sick, weak people.  If you take out loans now, you are going into debt in the present in order to earn far more money and reputation in the future.  What if you wanted to start a family in the meantime, but had to wait?  What if you had to leave your country in order to go to med school?  What if you couldn’t leave your country because of warfare or poverty?  So time versus money is a construct that we’re all in some form or another a part of.

Note that setting up time versus money does not imply that richer or more productive people are better.  It just places events on a chart.  So, I could still say that you could be a productive person even if you had no money, or you could be completely lazy and rich, but it doesn’t condemn you to being “good” or “bad”.

What if I set up this chart?

This, above, is perhaps a chart that shows the objectivist vs. altruist debate.  Should you be the best person you can be, but only for yourself (“selfishness”, in Ayn Rand’s use of the word), or is that utility also dependent on how much you’re useful to people outside of yourself? (if we added a time vs. money vs. health dimension here, we could map altruism vs. selfishness better)

Are the charts biased by going down and to the right?  Or up and to the right?  Does that imply, because of the way we learn mathematics (positive x, y graphs) that heading further out up/down to the right is “success”?

So I’m already seeing that even in displaying data in order, there are a lot of issues with bias, implication, and categorization.  Would it be better to have a visualization where a core self starts in the middle, and branches head outwards to denote connections with people outside oneself?

Let’s try to visualize the above JSON object.  Oh, and did you know that a Pokemon’s “personality value” is captured into a 32-bit unsigned integer in computer memory? (e.g. a binary 2^32 number)

Here’s a simple grid, with transparency based on the 1-10 scale of each characteristic.

The next is a line of squares, separated by category (mind, heart, body), with opacity again representing the 1-10 scale:

A background repeated pattern of the first grid:

The last one charts outwardness and utility for two separate people.  The opacity represents the outward importance of the characteristic (i.e. intelligence is not very outwardly useful to other people) while the length of the bar represents the utility to the individual.  The two peoples’ bars are next to each other (i.e. the first bar is person 1′s intelligence, the second bar is person 2′s intelligence, the third bar is person 1′s strength, etc.).

What I like about the chart below (obviously it’s still not very clear) is that you can find the average utility vs. outwardness (the middle white line) and chop off the baseline, to find the spikiness of the person’s outward utility to the world.  If the data were better represented, you could also get a better sense of who is more “visible” in the world based on his characteristics, and who tends to be overshadowed, literally.

I didn’t have enough time to do more complicated visualizations, plus Processing is not the best thing for working with hashmapped/databased data.  I would rather be using Python or JavaScript to compare two different users’ data across different dimensions. Here’s a gist of my Processing sketch:

Anyway.  In our last assignment, we all had to create an algorithm in Processing which could be used to distort an image, so that we could all put our algorithms together and output the results next to each other for comparison.

For my algo, I took Kyle McDonald’s algo which broke out an image into a grid of pixels.  Then I figured I could weave in another image by using modulus (%) arithmetic, an easy programming trick, to intersperse pixels at specific places into the canvas.  Then I also decided to take the original image’s pixels and map it on upside-down onto the image.  The original:

The result:

The algo:

PImage vt520_1(PImage img) {
  PImage img2 = loadImage("flag.jpg");
  img2.loadPixels();
  int n = img.width * img.height;
  for (int j = 0; j < n; j++) {
    color cur = img.pixels[j];
    color cur2 = img2.pixels[j];
    int r = int(red(cur)), g = int(green(cur)), b = int(blue(cur));
    int off = j*10&(j>>20)|j*3&(j*5>>15); // miiro
    off %= 256; // don't let the offset overflow
    if (j%2==0) {
    img.pixels[j] = color(0, 0, g^off, 0);
    }
    else if (j%3==0 || j%7==0) {
      img.pixels[j] = cur2;
    }
    else  {
      img.pixels[n-j-1] = color(r^off, 0, 0);
    }
  }

  return img;
}

Interestingly, because I was using modulus arithmetic, this resulted in the image changing in some applications because of the way they scale images down to certain sizes.  Some other views of the image:

Should have an image of all the algos working together at some point soon…  Classmates came up with some pretty amazing stuff!

Epler’s:

Guilherme’s:

Then our in-class assignment was to go out on the cloud and try to break services or get banned.  I liked this trick on Facebook by Hanna Kang-Brown, just using extended characters:

Here’s a vid of Super-Breakout:

Click here to view the embedded video.

My result:

Click here to view the embedded video.

I didn’t feel like it was destructive or additive; mainly I was just tweaking values with randomization, and then rotated it a bit, which would need further tweaking to have more of it on-screen.

Github gist: https://gist.github.com/3835687

Though I did find this pretty cool glitchiness written in Processing, by Amnon:

Click here to view the embedded video.

We had another assignment, which was to post screenshots of our crash reports.  Here’s mine, from doing my Java homework in Eclipse (I think I had an infinite loop going or something):

And finally, my girlfriend tried to take a photo using her phone’s front-facing camera, but this kept happening: