For years, smart devices have waited for people to speak to them, tap a screen or press a button. That is beginning to change.
Apple’s latest Apple Watch Series 12 and Apple Watch Ultra 4 are among the newest devices designed to understand what is happening around the user, including conversations and sounds in the surrounding environment. The company has introduced a set of features called Audio Intelligence, which uses the watch’s microphone and on-device processing to help users remember parts of conversations and recognise important sounds.
From commands to context
One of the most notable features is Siri Recap. When enabled, it can listen for conversations during the day and turn them into short notes containing a title, a summary and key points. Apple says users can decide when and where the feature takes notes and can switch it off through the Control Centre. The feature is expected to arrive in beta later this year.
There is also Live Rewind, designed for moments when someone has missed what was just said. Double-clicking the Digital Crown can bring up a transcription of the previous 15 seconds of speech. Apple gives the example of missing instructions in a noisy restaurant or forgetting something said during a conversation.
The features represent a change in the way wearable technology interacts with its surroundings. Instead of waiting for a specific command, the device can process sounds around it and use artificial intelligence to identify information that may be useful to its wearer.
What happens to the audio?
Apple says the system has been designed with privacy in mind. The company says Audio Intelligence does not create or store audio recordings and that raw audio is inaccessible to the operating system, apps, the user and Apple.
The S11 chip used in the new watches has a Secure Exclave, a hardware-isolated area that processes the audio before deleting it, according to Apple.
The company also says the features do not identify individual speakers. Siri Recap creates a high-level summary rather than a complete transcript and is designed to exclude sensitive information such as financial information and government-issued identifiers.
Live Rewind, meanwhile, plays an audible chime when activated and displays a microphone indicator, providing a signal to people nearby.
The rise of ambient AI
Apple’s technology is part of a wider move towards what is increasingly being described as ambient AI, a technology that can understand its surroundings rather than relying entirely on explicit commands.
The idea is already spreading beyond smartwatches. AI-enabled glasses, voice recorders, earbuds and other wearable devices are being developed to see, hear or understand what is happening around their users.
Reports say that companies pursuing these devices are facing growing privacy concerns, particularly around recording and whether people nearby have consented to being captured by an AI-enabled device.
That brings up a practical question that goes beyond Apple that what happens when a device understands a conversation involving people who never chose to use it?
The question is not limited to smartwatches. Companies developing AI glasses and other wearable devices are also having to establish how users can signal when a device is recording or processing information.
Reports say that some companies are exploring camera-free glasses and user-activated recording devices as alternatives, while others are focusing on processing information locally rather than continuously sending it to cloud servers.
But nonetheless, the direction of personal computing is changing. The smartphone may still be the main screen people carry, but the next phase of computing could involve devices that do much of their work without waiting for users to look at a screen at all.
