What are Apple Live Captions and how do they work?

What are Apple Live Captions and how do they work?

– It can transcribe any content on your device in English

Apple has announced a slew of new accessibility features, including Live Captions, which are set to roll out later this year to the iPhone, iPad, Mac, and Apple Watch. Here’s everything you need to know.

What are Apple Live Captions?

With Live Captions, if you want to watch a video with the volume off, you can read automatically generated captions at the bottom of your screen in real time. Apple said it can be used on streaming services, FaceTime calls, and other video conferencing apps. This is a useful feature for anyone, and for the hearing loss community, it’s an accessibility feature that will make it easier to watch videos on the iPhone, iPad, Mac, and Apple Watch. A similar feature is already available on Google Pixel phones and select Android phones.

Like Google’s approach, Apple said its captions will be generated on your devices, keeping your information private. But the company hasn’t yet said how you’ll enable Live Captions.

Where can you use Apple Live Captions?

Apple’s Live Captions will be supported in FaceTime calls, video conferencing apps (with automatic attribution to identify the speaker), and streaming video. The feature will also work during in-person conversations.

Which languages does Apple Live Captions support?

Apple’s Live Captions will transcribe any audio content in English across the iPhone, iPad, and Mac. 

When will Apple Live Captions be available?

Apple said it will announce a Live Captions beta later in 2022. The beta will launch in the US and Canada for iPhone 11 and later, iPad models with the A12 Bionic CPU and later, and Mac models with Apple Silicon CPUs. A software update with additional accessibility features will also arrive later this year.

One of those other new features is Door Detection. It will help blind or low-vision iPhone or iPad users find doors. It will even describe whether the door is closed and if it can be opened by pushing, pulling, or turning the handle. Because it uses lidar and machine learning, Door Detection will only be available on iPhone Pro and iPad Pro models.

Apple is also adding expanding the Assistive Touch gesture recognition controls it added last year, with Quick Actions on Apple Watch. Your Apple Watch will be able to recognise a double pinch to end a call, dismiss notifications, take a picture, pause or play media, or start a workout. Apple can also now notify your iPhone when a doorbell or appliance goes off.

Maggie Tillman, US contributing editor  · 18 May 2022 · ExplainerAPPLE