XR Interview : Savannah Niles (US) - Magic Leap

Pour la version française, cliquez ici.

Interview conducted by Greg J. Smith for MUTEK and Québec/Canada XR during MUTEK Forum 2020.

Savannah Niles is a multi-disciplinary designer and technologist working in XR. She's currently Principal Designer at Magic Leap, and a graduate of the MIT Media Lab. Niles is at the forefront of spatial computing, but beyond her enviable vantage point within the industry, she exudes an infectious optimism about XR’s potential to stoke the imagination, tell important stories, and create meaningful social ties. We chatted over Zoom recently, and our conversation spanned her roots and work at Magic Leap, her commentary on the state of XR in both consumer and enterprise contexts, and the people and organizations she recommends tuning in to, to keep tabs on new developments.

GJS: So it’s been approximately a month since your keynote “Place and Possibility—Experience Design for Mixed Reality,” delivered at MUTEK Forum 2020. How did it go?

SN: It was great! It was my first time doing a fully virtual talk at a conference that large, so at the time I was wondering if people were engaged—that was a challenge. But at the end I realized that there was a live audience and quite a lot of people tuned in. So it was fun to take audience questions and engage with folks on Twitter afterwards. I was so impressed with the programming this year and I hope I can be there in person in the future.

MUTEK also marked the kickoff of the The Montréal/Miami New Narratives Lab , which I know you were involved in, in a leadership role. What is its mandate, and where is that project headed, exactly?

I'm thrilled about this program, it's been in the works for over a year. From my perspective, it’s trying to catalyze a change in representation in the XR space that is about depth, not breadth. What I love is empowering a group of people tip-to-tail through the creative process and equipping them with resources, connections and insights from the NFB, Filmgate Miami, and MUTEK. I'm really excited about this, and the fact that this program is longer term, and that we're going to be able to support these creators with developing their projects. I'm also thrilled to build more of a connection between Montreal and Miami and to be a model perhaps for other kinds of cultural exchanges, between different cities and organizations.

Honestly, I think I'm ready to be a little surprised as well, because this field is so new. And I think that this program has attracted creators, who while they have an established practice–animation, dance, writing, etc.–they are looking for a way to incorporate XR platforms and technologies into it. It's, that's what the field needs in order to grow: pipelines for development. This kind of infrastructure has to be mature enough for a creator without much background in XR, to be able to quickly take these tools, add them to their toolbox and continue doing their work and telling their stories. So I'm excited to see how we can try to build those toolboxes in the context of this program, both for these creators selected as a part of the grant program but also for the organizations involved—to offer them to their broader creative communities. I'm excited about all of this.

About ten years ago, your BFA graduating project was a 3D drawing machine that scanned space and rendered it in Rorschach blot like point clouds. Could you talk a little about how you got started on that research within a sculpture program, and how its central concepts of "digital twins, computer-perception, and the relationship between the solid world and a softer one" that you describe in your folio shaped your thinking about XR?

Absolutely. When I was in undergrad there were no, or very few programs, dedicated to human-computer interaction or human-centered design or more multidisciplinary design technology courses, whereas now there's many. So I had this interest in the intersection of technology and human experience, and design, and products, and art, and I felt that sculpture was a program where I would have the most freedom to be able to explore those interests. Luckily, the SMU program I was in was very open ended and, and we did all kinds of stuff that one might not qualify as sculpture. Like one semester we built a house, and another we did 3D scanning, and so on.

It was in the context of that program that I got interested in technology, and began working on installation work for bigger brands and more artistic interactive installations. I think for a lot of new media artists, that's their gateway drug into the field. After I had some awesome projects under my belt, I ended up working at Walt Disney Imagineering, which is a great place to explore the intersection of technology and experience. And, it was at that point that I felt I was at a little bit of a crossroads, and wondered: do I want to work in more of a technology R&D space, or do I want to be an artist focused on installations?

At the MIT Media lab, they have a saying that ‘when you feel like you have nowhere else to go, you end up here’ as it is such an interdisciplinary place. So that was definitely my story. And that's kind of what brought me there, and it was really transformative; I just had the best time at the Media Lab. I was in a group called Viral Spaces, led by Andrew Lippman, and we were working on technologies around media and virality. I explored how you could artfully excerpt news, media, and video in the form of moving images that would be more appropriate for ambient displays or wearable displays—that was my research focus.

Much of your work at Magic Leap has remained shrouded in secrecy, but some of it is public facing. Could you talk about some of the key challenges you faced and learnings you derived from your team's work in rich touch interactions for spatial computing interfaces?

Definitely. So, the XR field has grown so quickly! When I joined Magic Leap in 2015, it was difficult to explain what we did. This was the era that Pokémon GO became a sensation and for many consumers, that was their introduction to augmented reality is. And still to this day, when I describe what augmented reality is and I mention Pokémon GO everybody kind of gets it. It was really interesting to watch that unfold. Since then, just from my perspective, there’s been an incredible exponential rate of adoption of a broad range of augmented reality technology on mobile platforms for consumers.

And that's just one end of the spectrum, on the other end there are the capabilities of standalone, high performance heads up displays—the Magic Leap, like the HoloLens 2, and others. It's been amazing to see how those platforms, in these very early days, are just trailblazing, due to the folks developing these platforms in our solutions ecosystem. And it takes trailblazing to locate the use cases for this inevitable technology that's in everyone's heads from science fiction, and to understand how computing can be more gentle and amplifying. That's why I think where we are at Magic Leap is sort of identifying potential partners in our ecosystem, enterprises that are ready to buckle up and say “let's figure out where the exact niches are where we can apply this technology and make a real impact, and see a real return.”

I think everybody sees the eventual transformative power of this technology. And in the next couple of years, it's less about demoing it and more about just getting it into the world—deploying it— and seeing it make an impact. It's been an amazing journey at Magic Leap, we started with more of a consumer entertainment focus. And now, like many startups that go through rapid growth we’re saying: “okay, it's time to identify areas where we can achieve product market fit, and really nail it.”

At Magic Leap I lead interaction, design, research, and prototyping, and over the last few months, I moved from a product strategy role into a purely design role, which is more in line with my background. It’s been extremely fulfilling, we’ve been working on an amazing set of challenges. I'm excited for the next couple of years in this space—we're really going to see an accelerating pace of adoption and innovation.

Are there any particular experiences you've had in XR (or just period!) that you would point at as the types of experiences that developers and storytellers should be striving to create in immersive media?

For sure. First, I think it's important to distinguish between enterprise and consumer experiences. For enterprise, even in the pre-COVID world, we saw an opportunity for more human sort of experiences—when connecting across distance. So, you know, exploring the capabilities of this technology to really bring a sense of co-presence and collaboration as the killer problem space, the killer use case of XR technology. Obviously training and visualization are areas where we're seeing lots of advances and innovation, and there's a real opportunity to apply this technology in those areas, but in a post-COVID world where travel is impossible, or less desirable due to environmental concerns, co-presence is a major challenge. By this point I think we’ve all experienced the fatigue of eight hours of Zoom calls, and the pros and cons of being remote workers. I think there's a real opportunity to see broader adoption of XR tech, once we start to nail these co-presence use cases on these devices, and can really bring a greater sense of presence—better collaboration, richer creative sessions for teams—and make a massive impact across knowledge work in general.

And as far as consumer use cases go, I think that there's a great opportunity in eCommerce. I absolutely see the future of buying things as holding up your phone and virtualizing the item in front of you on your table, in your home, in your hands. That process being a more immersive experience with shopping as an opportunity for so much innovation and creativity, and it can eventually extend to brick and mortar locations. Imagine being able to walk into a store and to virtualize and access endless inventory, or to have a more engaging experience with that brand, as well as the merchandise and the store—as an augmented space. To me, that's super exciting. And I know that those kinds of use cases are going to happen on the phone, as it is already equipped with the sensor stack needed to support these experiences.

Very much a sufferer of Zoom fatigue, I’m intrigued by your comments about co-presence. If you weren't bound by the limitations of current technology, what would a better Zoom experience feel like? What improvements would you like to see?

That's a great question! I mean, if you've ever experienced a co-presence experience on an XR device, it really is different, it's hard to put your finger on it. There’s kind of a je ne sais quoi of being with a virtual person. I think what it comes down to with presence is really about the bandwidth of communication, of sensing another person. The way we communicate is so much more than our words, it's the tone of our voice, our body language, the expressions on our face. It’s our proximity to the other person and our movement around the space that we're sharing. When you're in person with someone, you're consciously and unconsciously taking in all that information, and when you're on Zoom, all that information is communicated through much narrower bandwidth. Also, people are literally small on Zoom; even scale is disrupted from how we communicate in person.

An improved experience would take advantage of the simple attributes of spatial computing of mixed reality over a screen-based media, things like scale, proximity, and distance from others that combine to create a more natural communication experience. Even an avatar that's a little bit cartoony can deliver an incredible sense of presence, because you're seeing a person, you're hearing their voice, and seeing their body and head move in natural, expressive ways as they talk. And they're in real scale, they're in real proximity with you—that makes such a difference. So I think that that's what it comes down to: a higher bandwidth form of communication is needed. And when we're looking at little people on little grids on screens, it's perhaps lot more disruptive than we consciously realize.

Indeed! Okay, beyond the desk and the store lies the living room, I want to hear about your thoughts there as well. I know it’s a VR contrarian position to point out that VR's 'second coming' in 2015-6 fizzled with the nominal adoption of the Oculus Rift and HTC Vive. How would you describe what happened there and what is standing between that and folks bringing XR into their living rooms?

It comes down to brands and content–it’s the content ecosystem. I think that there's a lot of ingredients needed to grow a rich content ecosystem that can sustain mass adoption in a consumer space. One of them is the technology, it has to be good enough. Second, the interaction design of the platform and experience has to be easy and seamless–which takes so much work to get. And you know, the original Oculus and Vive dev kits, they definitely had some technical limitations. They were connected to a PC, the graphics were not so great, they made a lot of people nauseous, because this was all emerging interaction design–it took us awhile to kind of get a handle on the basics. It's difficult to adjust your settings in that context and it's also difficult to sell that platform as a consumer ready product. So that's, part of it. And there's other issues: the price point, the devices were tethered to a big graphics card on a beefed up gaming laptop. And all those factors yielded a product that was ready for consumers, in that step right before big brands and big IP could really release content on it. Now, we see the biggest IP in gaming on these platforms—and this when we see that the Oculus Quest is sold out.

Who are some of your favourite voices—designers and thinkers—that you think our readers should follow to expand their thinking about XR and immersive media?

I think Sophia Dominguez from Snapchat is excellent. Her work is amazing, I think Snapchat is an awesome, innovative force in the space, so I definitely would follow her. Someone else on Twitter, who I actually get notifications every time they tweet, because his content is so good, is Adam Varga. He’s a mixed reality interface designer based in Tokyo. He posts a lot of his work which is absolutely excellent, but he also has this huge appetite for content in the field, and is often surfacing interesting papers, talks, and experiments; his feed is like a really good industry newsletter. Paul Reynolds is also worth following, I worked with him for a long time and he's great—he has a podcast, XR Talk.

As far as organizations in the space that I think are doing great work. I work with FilmGate interactive in Miami, and I think they're an excellent nonprofit that's focused on storytellers working in emerging technology. They're going to have their festival in December, which is going to be really spectacular. They're bringing in a lot of innovative XR work and it will be online, as well as in person.

Also, R Lab based in New York is a really great center for XR research, they support some incubators. I really admired how they've taken a really vocal stand on the BLM movement, in fact I think it’s been one of the loudest voices around, calling for how it's an imperative for us working in XR to think about and bring and focus on social justice in our work, and elevate Black creators, and allocate more funds for Black led organizations. They also have an excellent newsletter every week.

Gray Area Foundation for the Arts in San Francisco is another awesome space. Their programming is really great. And, their Executive Director Barry Threw is an awesome guy, and I know at the beginning of the pandemic they struggled, like everybody, and they very successfully pivoted to creating excellent online programming.

Greg J. Smith is a Canadian writer and cultural worker based in Hamilton, Ontario. He is an editor for HOLO and a contributor to Musicworks, and his writing has appeared in publications including Creative Applications Network, Rhizome, and Back Office. Greg is a PhD candidate within the Department of Communication Studies and Multimedia at McMaster University, where he is researching the emergence of the programmable drum machine in the early 1980s.