The Project

A wand instead of a device

Magic Wand was commissioned by Warner Bros. for the Wizarding World. Near Future Laboratory served as the Creative R&D function, and Google ATAP’s Direct Objects team, led by Alex Kauffmann, collaborated on the technology. The brief was a wand for the Wizarding World that could cast spells. The result was a wand with no buttons, no screen, no app, and no visible radio, that read close to 300 distinct spells from the motion of a person’s hand.

The wand is a good example of what Alex calls a direct object: Google’s smarts inside an everyday thing, without the thing becoming a device. Wands are usually made of found organic material. They act on the world at a distance. Nothing about a wand tells you how it works. Those constraints shaped every technical decision in the project.

Near Future Laboratory’s role

Warner Bros. was the client. Near Future Laboratory worked for them as the Creative R&D function on the wand, in collaboration with Google ATAP’s Direct Objects team, which carried the sensing, firmware, and recognition work. Near Future Laboratory fashioned the user experience of the wand and the business case for it, working alongside key partners at Warner Bros. and the other participants in the Wizarding World wand effort.

That meant shaping what casting a spell should feel like in the hand: how a spell begins and ends without a button, what the light at the tip tells a person, how a first-time fan and a practiced one both get a satisfying response, and how the wand behaves at home, in a Wizarding World location, or at a fan meet-up. It also meant putting the wand in front of the people who would have to fund, license, and ship it, with a case for repeat use, community, and a device that could keep gaining spells and interactions over time. The SmartWand brief below is the record of that side of the work.

The SmartWand brief

The Near Future Laboratory archive for this work holds the SmartWand Goals and Requirements document that framed it. The brief set out four goals for the wand:

• Bring the Wizarding World brand belief to life, that imagination has the power to change the world, with an experience that is genuinely imaginative, innovative, and category-leading.
• Generate repeat use and meaningful interaction between fans and the franchise, between fans and the wider Wizarding World community, and directly between fans.
• Give fans wow moments at many touchpoints, in owned and operated locations, at home, and at unofficial fan meet-ups, with the experience personalizing as a person engages more.
• Make a living device that evolves over time, with new features, new challenges, and new interactions.

The rest of this page follows Alex Kauffmann’s own account of how the wand came together on the ATAP side. His write-up tells the story from inside Direct Objects and does not describe the Warner Bros. commission, which is his to frame as he chooses. He wrote it up as a worked example of a healthy invention process, in five steps, and it is retold here with his structure kept intact.

1. A new challenge

Alex started Direct Objects at ATAP in 2018 with a theme, Google’s smarts in everyday objects, and no defined projects. His boss had told a friend in the gaming industry that ATAP’s Jacquard tags could easily power a magic wand game accessory. The Jacquard team was busy with other work, so making good on that promise fell to Direct Objects.

2. A coherent story

The more the team thought about wands, the more a wand looked like the purest version of the not-a-device idea. Wands have no buttons, screens, apps, or Bluetooth. They tend to be made of found organic materials. They still act at a distance on the world. Magic, in this case, would be indistinguishable from sufficiently advanced technology. That gave Direct Objects a narrative and a conceptual starting point.

3. A technical insight

A magic wand is, in practice, a laser pointer that writes strange letters on a faraway surface. Two-dimensional stroke recognition is a largely solved, low-latency problem.

The Jacquard tag carries only a gyro and an accelerometer. No microphone, no camera, no other motion sensors. So the wand’s powers had to come entirely from wand motion, which meant reconstructing that motion from drifting IMU data. Early experiments showed that a heuristic approach to spell recognition would not give consistent results across different people, and that a machine-learning model trained on 3D IMU data would struggle to tell apart more than a handful of gestures without noticeable lag.

The observation that unlocked the project came from handing people a wand and asking them to cast a spell, which Alex did a lot. Almost everyone mutters something and swishes the wand around, or flicks it back and forth like a conductor. Both of those motions stay mostly within a plane. They are not really three-dimensional at all.

That recalled the LG gyro-mouse remote control, and a colleague pointed out that the short-lived Daydream View controller worked on the same principle: raycasting a pointer from the end of a remote onto a distant screen.

4. A prototype to play with

The team built a proof-of-concept wand in about a week, wrote a phone app that projected the IMU data onto a plane, and connected it to an off-the-shelf stroke recognizer. It worked the first time.

The prototype became an experimental platform, and playing with it produced several further hardware and software findings:

• Stroke recognition needs a crisp on and off signal, and wands do not have buttons. A wand with a button is just a remote. The team found a way to detect the beginning and end of a spell from the wand user’s natural behavior, using an interesting inductive squeeze sensor.
• Projecting an imaginary ray from the wand tip onto a flat plane produced scaling problems and discontinuities as the tip swung parallel to the plane. The math was reworked to raycast onto the inner surface of a sphere instead.
• LEDs can serve as both light emitters and light detectors, so in some cases a single LED replaced a bulkier emitter and receiver pair.

The finished wand felt magical. It detected nearly 300 distinct spells with almost no false positives, negligible variation between users, and no perceptible latency.

5. Applying discoveries to other projects

The final demo folded in everything the team had learned and impressed a number of external partners, including the friend whose conversation had started it. It gave Direct Objects a tangible example of how a direct object looked and felt, and several sensing techniques that moved straight into other problems. To a casual observer the wand might have looked frivolous. To the team it smelled promising from the outset.

Alex’s takeaway, in his words: “answering a question no one had asked before led to novel and useful solutions.”

Files and archive

The Near Future Laboratory side of the archive holds the SmartWand Goals and Requirements brief, filed under the 2019 Warner Brothers client folder in Dropbox. The films and images on this page are hosted in Photarium under the cf-atap namespace.

Attribution

The narrative above, this demonstration film, the stroke-recognizer capture, and the patent-drawing detail come from Alex Kauffmann’s write-up of the project, The magic of invention, on his portfolio site. I paraphrased this with the help of Codex to add it to my portfolio of projects and for complete coverage of a project for which I didn’t take very many notes for some reason. Alex started Direct Objects at Google ATAP and led the wand work. The phrase about learning to smell promising ideas, which frames his account, belongs to his friend Chris Luomanen.