NEWSROOM
Movie subtitles – whenever you want
A new technology allows the hearing-impaired to watch a movie at whatever cinema they like, and never miss a line of dialogue. It was developed by researchers at the Auckland Bioengineering Institute with the concept honed during their participation in the Centre for Innovation and Entrepreneurship’s Velocity programme.
The device, called the Vivify headset, uses augmented reality to project subtitles below the cinema screen. A commercial trial of the headset is currently being run through Event cinemas in Newmarket, Auckland.
The system was developed by Marco Schneider and Tony Tse when they were both studying their PhD at the Institute.
Tony was investigating an underwater hearing aid for divers inspired by the mechanism of fish hearing, and Marco was developing a computational framework for analysing joint health from medical imaging data. (Marco is now a research fellow at the Institute, and Tony is completing his PhD.)
The idea for the device came out of a late-night conversation, when they were planning to watch a movie and Tony, who is hearing impaired, expressed his wish that more movies came with subtitles.
Subtitle data is included in most movies, but cinemas in New Zealand rarely use it, so cinematic options for the hearing-impaired are extremely limited – Event cinemas in Auckland only shows subtitles on Mondays, at certain sessions.
Their three-in-the-morning idea went on to win a prize in the Velocity Innovation Challenge in 2015 worth $1,000, which they used to develop the first prototype.
The most challenging aspect of the project in the beginning stages was finding someone in the cinema industry to talk to, to find out if their idea was viable and if the industry would support such technology being used in cinemas. Eventually they managed to present their idea to Event cinemas, which thought it a good one and encouraged them to develop the headsets.
The pair established the start-up company, Vivify, in 2017.
The initial prototype was “was pretty bulky”, says Marco, but has had several iterations since, with changes being made in response to feedback from people participating in pilot trials in which participants watched a movie wearing the headsets.
Changes included finding a way to make the subtitles stay in place when people turned their head to, for example, talk to their neighbour. “The subtitles would follow you, which was annoying and distracting,” says Tony. “So we’ve been trying to make it as seamless as possible.”
While their current prototype involved customising off-the-shelf headsets, they plan to build headsets from scratch, make them more aesthetically pleasing and lighter.
The current prototype model is connected to an iPhone that comes with the headset, but if they can dispense with the phone it make the headset much lighter. They are also working on adding other languages and expect to have Chinese subtitles working soon.
The current trial may reveal the need for further refinements but there is already good evidence that the headsets will be very much in demand.
“We’ve received a lot of requests from across New Zealand asking us to run trials in cinemas in their cities,” says Marco. “It’s very encouraging – people clearly want this technology.”
If you would like to use the device when you next visit, you can book a pair of headsets through the Vivify website.
Now to Love: Subtitle technology for the hearing impaired
A new technology allows the hearing-impaired to watch a movie at whatever cinema they like, and never miss a line of dialogue. It was developed by researchers at the Auckland Bioengineering Institute with the concept honed during their participation in the Centre for Innovation and Entrepreneurship’s Velocity programme.
The device, called the Vivify headset, uses augmented reality to project subtitles below the cinema screen. A commercial trial of the headset is currently being run through Event cinemas in Newmarket, Auckland.
The system was developed by Marco Schneider and Tony Tse when they were both studying their PhD at the Institute.
Tony was investigating an underwater hearing aid for divers inspired by the mechanism of fish hearing, and Marco was developing a computational framework for analysing joint health from medical imaging data. (Marco is now a research fellow at the Institute, and Tony is completing his PhD.)
The idea for the device came out of a late-night conversation, when they were planning to watch a movie and Tony, who is hearing impaired, expressed his wish that more movies came with subtitles.
Subtitle data is included in most movies, but cinemas in New Zealand rarely use it, so cinematic options for the hearing-impaired are extremely limited – Event cinemas in Auckland only shows subtitles on Mondays, at certain sessions.
Their three-in-the-morning idea went on to win a prize in the Velocity Innovation Challenge in 2015 worth $1,000, which they used to develop the first prototype.
The most challenging aspect of the project in the beginning stages was finding someone in the cinema industry to talk to, to find out if their idea was viable and if the industry would support such technology being used in cinemas. Eventually they managed to present their idea to Event cinemas, which thought it a good one and encouraged them to develop the headsets.
The pair established the start-up company, Vivify, in 2017.
The initial prototype was “was pretty bulky”, says Marco, but has had several iterations since, with changes being made in response to feedback from people participating in pilot trials in which participants watched a movie wearing the headsets.
Changes included finding a way to make the subtitles stay in place when people turned their head to, for example, talk to their neighbour. “The subtitles would follow you, which was annoying and distracting,” says Tony. “So we’ve been trying to make it as seamless as possible.”
While their current prototype involved customising off-the-shelf headsets, they plan to build headsets from scratch, make them more aesthetically pleasing and lighter.
The current prototype model is connected to an iPhone that comes with the headset, but if they can dispense with the phone it make the headset much lighter. They are also working on adding other languages and expect to have Chinese subtitles working soon.
The current trial may reveal the need for further refinements but there is already good evidence that the headsets will be very much in demand.
“We’ve received a lot of requests from across New Zealand asking us to run trials in cinemas in their cities,” says Marco. “It’s very encouraging – people clearly want this technology.”
If you would like to use the device when you next visit, you can book a pair of headsets through the Vivify website.
Now to Love: Subtitle technology for the hearing impaired
EMAIL
CIE@AUCKLAND.AC.NZ
POSTAL ADDRESS
THE UNIVERSITY OF AUCKLAND BUSINESS SCHOOL
PRIVATE BAG 92019, AUCKLAND
