I have been trying to learn Mandarin since 2015, and Japanese for much longer than that. Consequently, I’ve spent a long time thinking about how various tools could aid my language-learning. It occurred to me while watching a gorgeous dog play with its owners on a beach in Batemans Bay that, in a world where everybody speaks different languages, to be able to tell anyone, in real time, about your dog would be incredible. What’s its name? What does it like to eat? Are there any plants nearby that are toxic to it? What’s that insect wandering around in the vicinity of your dog, and does it pose any lethal danger to your pet?

Dogs, insects, plants, beaches. What the hell do these everyday objects have to do with language-learning? I say everything

The current situation of language-learning is that it is so disconnected from anything that is occurring in real life. So, when people learn a second language it often feels as if they are doing this simply to show off their brilliance in memorising long lists of seemingly irrelevant gobbledygook that someone from the relevant foreign country may or may not understand. If you’re lucky, you might get language exposure if you happen to point at a native speaker (who probably also wants to learn English from you) and get them to explain something in your shared environment in their own language. Through many years of trial and tribulation, you’ll likely get good enough at a language to be self-sufficient. However, you will never get the intimate one-on-one teaching of the most basic and fundamental aspects of a language that you got from your parents or caregivers
when you were growing up. Otherwise, if you’re really committed to learning a language, you will simply end up investing endless hours to begin a long and painfully boring career in translation. You learn all those words for what? To transfer them to another, different set of words?

Typically, language-learning is best done in childhood when there is an adult figure who can teach the child everything: from what the colours of the rainbow are to the shared social conventions of the shared language environment. Once the child reaches adulthood, there are many social barriers (such as the amount of time one is required to spend working in a job versus obtaining knowledge) which prevent a person from learning a new language in quite the same way. Consequently, less time can be invested in making sure that learners of a second language have the same immersive learning experiences that they had when introduced to their mother tongue. They have gone through the arduous process of first language acquisition, but their mother tongue becomes their cage. In particular, English acquisition is a gilded cage because of all the opportunities it offers, along with the limitations it has and Anglo-centric assumptions it forces us to make about the rest of the world.

Upon understanding this, my interest in augmented reality (AR) was ignited: use AR to teach language!

Since the rise of ‘Pokemon Go’, smartphone app developers have been scrambling to get on the AR-wave. But a quick search on the Android app store reveals only a few haphazard titles of truly underwhelming programs which merely superimpose various objects on the camera. The trouble with smartphones is they do not have big enough screens to incorporate many buttons or functions and they have limited battery lives for a constantly active internal camera. It would pretty awesome to have a ‘Pokemon Go’ for language learning, but if you were to go around tagging things in the hopes of teaching people words, in reality the best way to describe the situation would be: Confusion.

If people don’t know coding there’s no way that they can use a smartphone to teach non-English speakers wanting to learn physics that that round 3D object is called an ‘orange’ and that if you drop it on particular surfaces, it may roll, splatter, or be sliced in half depending on the situation that it finds itself in.

So, this is where I started looking into AR headsets.

Currently, the potential applications for AR headsets are huge. Language teachers can make language learning programmes using spatial learning without the need for expensive props. Mechanics can fix your car without having to consult a thick engineering manual. You can follow along with personal trainers in real time doing exercises no matter where you are in the world. Remember in Star Wars where Princess Leia was sending a message as a holographic image? That will be possible too. All you need to do is model a 3D-image with your hands, upload the file to your programme and attach information to that file.

Language learning companies such as Mondly are already beginning to take advantage of AR technology. Students using Mondly can interact with a virtual reality language instructor in real time to practice travel and daily life phrases in another language. However, since it is a smartphone app, it requires a lot of coding to create the graphics and artificial intelligence, and it removes the human component of interacting with a real person in another language. It teaches you that automated ‘service talk’ used in cafes and hotels. But is it an authentic experience? Does it encourage creativity and experimentation? Adventure and discovery? No, but these are improvements that current AR-research is beginning to address.

With apps like ‘italki’ (an app for tutors and students) and ‘LingQ’ (an app for micro-translating and creating audio versions of documents) adopting this kind of technology, you could do things like book real time language classes with students from around the world without worrying about learning props. Or you could facilitate users to look up words in any foreign language document they come across in normal life, be it a street sign or an instruction manual. Some apps such as ‘Drops’ focus on teaching concrete nouns as well as some abstract nouns and verbs, but with AR you could assign any object in real life with nouns to be learned, and go a step further by making videos of interactions between objects and verbs in other languages. A cooking lesson in French would no longer have to be translated into English to get a broader ‘reach’, it could be made understandable to anyone, regardless of what languages they know.

One day, when augmented reality headsets are more mainstream, adventure and discovery will be possible in every aspect of learning. We can take everything we love about the virtual world and place it back in the real world where it belongs.

The main argument against AR is that there is scope for people to delude themselves into a false reality. Additionally, people could trick themselves or others into thinking something is there when it isn’t or vice versa, resulting in injuries from falling in holes or hitting objects you can’t see due to an augmented obstruction. According to Meron Gribetz, co-founder and CEO of Metavision, augmented reality should not deceive users about the nature of reality, but simply add to it in a way that does not cause harm. For instance, with Pokemon Go, users are not deluded into thinking that the Pokemon they are interacting with are real. As long as users of an AR app are aware of what is real and what is not, there should be minimal problems.

Another challenge for AR is that computers still do not have the capacity to understand natural human language and they still rely on tools like barcodes and QR codes to identify things. However, if one were to create an online, crowdsourced platform using information generated by paying users (like many platforms now such as LingQ), communities made up of people who own AR headsets will naturally develop.

As with many emerging technologies, there will be implications for public safety, policy, and law which will need to be given careful consideration. But in many ways, I feel that a world with AR headsets would be an exciting place to live.

Zena Mason is a postgraduate student at ANU just about to begin a degree in linguistics. She has a special interest in language acquisition.