On Mixed Reality and the Brave New Virtual World

“Social Network” by Kevin Dooley, used under CC 2.0

We are on the cusp of a great awakening and evolution through emerging tech.


Our brains, which are themselves highly complex organic computers, collectively comprise this vast, great network. “Humanity.” Your body is essentially an organic bio-shell – a highly advanced haptic suit, designed specifically to allow it to fully experience the physical world in which it exists, through the five senses – sight, smell, touch, sound, taste –  for the purpose of sending information to your brain from the time it develops in the womb, gathering, categorizing and storing data in a complex database deep within the brain where memory is stored.

 The information your brain gathers through the body about the physical world around it is simply data, random chunks of categorized and cross-referenced information  that the brain stores in memory for when you might need to access it later. The body gathers all this data through the senses, but data stored in databases of any kind isn’t terribly useful in and of itself without writing programs that access them, selectively pulling from that vast amount of data, examining the patterns, extrapolating from information at hand to make assumptions, and outputting more data.
“My Social Network” by Luc Legay, used under CC 2.0

Now here we are just past the midpoint of 2017, and Microsoft is investing heavily in pushing what they (and a few others) are called “mixed reality,” they’re partnered with Oculus, and meanwhile Zuckerberg and Facebook are churning away at integrating our collective obsession with the social network – one of our biggest sources of personal and interpersonal information – even more deeply into our daily lives. Apple is said be announcing something big with augmented reality in the fall and if you believe Robert Scoble (which I tend to on this) it’s going to be mind-blowing. Google is politely ignoring the term “mixed reality” for now, preferring to coin its own phrase with “immersive computing.”  We are at the beginnings of profoundly evolving the very way in which we as human beings gather data, communicate and connect with each other in ways that centuries ago – even decades ago – we could never have considered outside our imaginations. But the things we imagine with our brains evolve over time into things that become real.

This excellent thought-piece by Kamal Sinclair, director of Sundance Institute’s New Frontier Lab program, asserts that we have the power to code a new operating system for humanity, and it has a lot of smart things to say about the importance of diversity in storytelling and how this is necessary to seed many perspectives into humanity’s collective worldview. And she’s right about that. But the impact of this emerging tech on our collective human consciousness will go much deeper than addressing issues of social justice and equity (and, frankly, I would posit that while we can work toward increasing understandings and take direct actions that affect social equity when the system makes room and allows that, we will still be addressing social justice  issues probably until the very end of the world as we know it).

“Une représentation de mon réseau social dans Facebook” by Luc Legay, used under CC 2.0

The “program” of our humanity, what we really mean when we speak of “the system,” is written by our interactions with each other, all of them. Every single interaction you have now,  have ever had, will ever have with another human being writes your code into their brain and theirs into you.  Everything fed into the system for collective consumption – journalism (real and fake), movies, TV shows and webseries, books, magazines, commercials, print ads, billboards, podcasts, music and music videos, video games, virtual reality games – story and art in all their forms – gets stored in the organic databases in your computer-brain’s memory bank.

 Every interaction you have with another person gets stored there too, from your relationship with parents and siblings, through your friends, colleagues, enemies; even the strangers with whom you interact, or not – the homeless kid, the elderly woman taking too long in front of you in the checkout line, the parents with the crying baby on your flight, the stranger you get in a fight with in your brother’s Facebook thread  – every single interaction you have with another human inputs your words, along with the emotional connection to those words, the positive or negative energy you feel for each other, into your respective brain databases where it is stored to become part of the patterns that inform the way in which you both will live your lives.

In the same way that your physical devices like smartphones can share data back and forth over bluetooth without needing to be physically connected to each other, our brains connect to each other when we interact with and communicate with each other. This  “organic bluetooth” is sharing data back and forth all the time, every time we interact with each other.  We are inputting code into each other’s databases, creating the patterns within our computer-brains that, in their sum total, form the shape of who “we” are individually in relation to other human beings, who “we” are within the context of social groups into which we self-sort, and the collective interactions between those groups of individuals.  The “me” and the “you,” the “us” and the “them.”

“NetWork” by CEA +, used under CC 2.0

Will immersing in virtual experiences profoundly shift the ways in which we interact with each other? Idealistic, but I’m not convinced yet that the promise of VR as an “empathy machine” will play out the way we might like to imagine it could, when in the real world we walk right by the homeless person begging on the street corner without acknowledging their humanity, when people are beaten up and killed for their skin color or sexuality or whatever thing about them identifies them as “different.” Will gaining virtual perspectives ever truly overcome privilege, hoarding of resources, bullying, the need of humans to organize and separate into the “us” and the “them?” Maybe I’m cynical, but good luck with that.

The deeper immediate impact of all this emerging tech will come not from immersing ourselves in the sorrows and tragedies and joys of other people’s lives, and certainly not from inventing better and more realistic game mechanics for killing other people virtually, but from the integration of gaming structure and story into our daily lives and our interactions with other humans via augmented or “mixed” reality – both fancy terms for the idea of virtual images and worlds being layered on top of what you see in the real world.

“Dada Doodle for the Date” by Bill Strain, used under CC 2.0

Already we bring our social networks – our physical connections to other humans – with us wherever we go with our smartphones and tablets and laptops.  You can think of your smartphone as a proxy for the invisible connection you already share with friends and loved ones. It serves as a conduit that allows you to maintain the connections you share with others.  If you think of our soul connections between each other as having “on/off” switches, then you can think of the constant flow of communication between two people as the way in which we keep that connection switch flipped to “on.” Lose contact with an old friend for long enough, and you start to feel “disconnected” or distant from them. You want to reach out and “reconnect.”  So, we carry our smartphones in our pocket or purse, our laptops in our backpack. We connect into our social networks obsessively wherever we are to see what’s new, did someone “like” my post, my photo, my witty GIF comment?  Because we want to keep that connection switch to “on,” to feel valued by others and value them in return.

When augmented / mixed reality glasses are no bulkier than sunglasses, when people can wear them and look fashionable and not stupid while doing so, we will carry our connections with us all the time, everywhere we go, and our connections will become even more deeply integrated into all aspects of our lives than they are now. Conversely, when we lose a connection – a divorce, a death, the end of a friendship – the hole that loss creates will have an even deeper resonance (thanks, Facebook, for those constant reminders of memories and birthdays of dead friends and relatives). And also yet to be considered and determined: will increasing the intensity of our connections to our immediate circle of “people we know” in some way have a negative impact on our emotional bandwidth for feeling empathy for the broader circle of those we don’t know? Is it possible the tech that will allow us to deepen our social connections might also end up serving as a crutch that limits deeper emotional connection and growth?

This tech, when it is integrated into your life, will be able to seamlessly record and keep a record of events that you observe while wearing them. Your child’s birthday party. Your daughter’s wedding. The birth of your child. The last words of your mother on her deathbed. The fight with your neighbor over fence lines.  Your interaction with police in a traffic stop or at a protest. Everything, the (perceived) good and the (perceived) bad and the (perceived) ugly. If I had had this technology three years ago before my mom died, I could have recorded seamlessly and organically all those precious last conversations with her, and then recreated them now, with it seeming like she was right here in front of me talking to me today.

With AI that’s advanced enough and enough recorded interactions with another person in a database from which to extrapolate patterns, you could theoretically recreate someone interactively in holographic form in a way that would simulate interaction as that person actually would have done, and make them feel alive again. What if you could summon up your mom after she’s passed, sit down with her, ask her advice again? In the future, I think you’ll be able to. Could you also recreate a partner from a long-term relationship in a virtual avatar and “reprogram” that ghost of a past love into the partner you wanted them to be? Maybe. Creepy, but maybe. And where there’s a “maybe” you can bet there will be someone doing it.

Like it or not, the coming of mixed reality is the next level-up in the ongoing evolution of the human brain network that we call “humanity.”  Whatever virtual reality you care to live in, whatever virtual versions of yourself you care to create and be, you’ll be taking it with you wherever you go. Welcome to the brave new virtual world.

Let’s try not to mess it up.