At PayPal’s Battlehack NYC a few weekends ago, my friend Brian and I hacked together an app called DeadDropAR — it’s an app that lets you put digital money on real objects in real locations — just like how you’d do a dead drop in real life, but now, via your smartphone.
The flow is simple — as shown above:
- List of Drops: Load a list of “Drops” nearby, select and then scan for it!
- Create Drop: Create your own “Drop” by going to the location to scan (right photo, map) and specifying the object to scan (left photo, augmented image to track) and specifying the amount.
- Scanning: When the object is recognized, choose to pick up the amount deposited there (addendum due to popular demand: and see the equivalent virtual 3D dollars and coins in augmented reality!)
DeadDrop AR started as a boring old hackneyed augmented reality scavenger hunt: computer-vision scan objects to get money via paypal. But then, someone had already pitched that idea, and we were mostly just toying with our skills to see if we could make it in limited time, with intentions to go home early without actually pitching or entering the contest.
As usual, I worked with the hackathon junkie handicap of starting everything from scratch. As for hack idea — with me being still stuck in my AR phase, even on hiatus in NYC, the hack had to be something relating to augmented reality. Here are some ingredients from the final recipe that we worked on during the hack before we finally figured out what our hack was actually going to be:
- an iOS-like GUI framework for Unity, including bounce-scroll!
- (I wanted to show this off) computer vision-based object recognition with dynamic marker-object creation
- Google maps and geofencing based on lat lng
- straight-forward MySQL database on the serverside for logging each drop transaction
- PHP wrappers for the above transactions
- Basic ImageMagick image cropping and such for thumbs vs high resolution image recognition bread crumbs
- CC payments via PayPal sandbox via their new API
Several iterations later, in the dead of night, when such ideas are prone to occur, we pivoted from a marketing platform to a free economy money-trade implementation — what if, instead of just searching for objects, the user could openly leave money on real objects in real locations. Effectively, do a dead drop completely virtually, by “placing” digital currency in the real world.
Basically, give digital currency a real world body.
During the quick QA session we had for our Battlehack pitch, I think someone asked about the element of privacy and anonymity that real cash has that virtual currency inherently cannot have due to the necessary client-server transaction record in depositing the payment. After attending defcon, and taking a deeper look at the workings of distributed systems such as Tor or torrents, I’m convinced that digital transactions could be as anonymous as cash payments. (Also, a quick extension of our hack would be to simply use this system as something better than scan-a-QR-code-to-fetch-your-bitcoin.)
After showing the app to other participants, it seems people wanted to see a gimmicky version where 3D coins and bills are augmented — the third frame above shows the hack.
Funny caveat on digital money – In optimizing some currency textures from Wikipedia for hacking out the virtual version of this, I stumbled upon an odd Photoshop caveat:
I ended up just using Preview to crop and resize these images to powers of 2. Import, orient to pose tensor on recognized surface and presto – 3D augmented reality money sits on that surface.
Being handed battleaxes to represent NYC and the US against hackers from other countries in the Global finals, I suppose we did okay with our app for Battlehack — for this app’s next step, we only have to make a few breakthroughs to change how people perceive digital money as real as real world cash can be.
And as for the next round of Battlehack – we’re planning to make something totally new. PayPal is offering to fly us to San Francisco, but did not actually specify from where… I wonder if we’d be able to fly from space!