Apple will update iOS notification summaries after BBC headline mistake – Ars Technica
It’s unfortunately not possible for Apple Intelligence to make zero errors.
Apple plans to release a software update that is meant to help users understand better that its notification summaries are AI-generated and may contain errors, according to a recent BBC news story. The update is a response to reports that the summaries gave users misleading information about world events.For example, one false summary suggested to at least one user that Luigi Mangione, the alleged murderer of UnitedHealthcare CEO Brian Thompson, had committed suicide. The notification in question was meant to summarize the most important tidbits from 22 BBC news app notifications, according to a widely circulated screenshot.Apple hasn’t publicly specified exactly what will be changed to better inform users—only that it will be a software change that will “further clarify” when the notifications have been generated via the feature that resides under the Apple Intelligence umbrella.The notification summaries, which are generated for individual users and devices via a custom large language model (LLM) Apple has trained, already had a small icon on them to indicate that they were not normal, unaltered notifications from the apps in question.However, that icon’s meaning may not be immediately apparent to most users.Users can disable the Apple Intelligence notification summaries by accessing the Settings app on their mobile devices.When contacted for a statement, an Apple spokesperson told Ars:Apple Intelligence is designed to help users get everyday tasks done faster and more easily. This includes optional notification summaries, which provide users who choose to opt in a way to briefly view information from apps and tap into the full details whenever they choose. These are identified by a summarization icon, and the original content is a quick tap away. Apple Intelligence features are in beta and we are continuously making improvements with the help of user feedback. A software update in the coming weeks will further clarify when the text being displayed is summarization provided by Apple Intelligence. We encourage users to report a concern if they view an unexpected notification summary.The BBC stories about the error-prone AI have often seemed to lack understanding of how the Apple Intelligence notification summaries work—for example, in suggesting that all users received the offending notification about Mangione. The wording of the summaries varies on individual devices depending on what other notifications were received around the same time.Nevertheless, it’s a serious problem when the summaries misrepresent news headlines, and edge cases where this occurs are unfortunately inevitable. Apple cannot simply fix these summaries with a software update. The only answers are either to help users understand the drawbacks of the technology so they can make better-informed judgments or to remove or disable the feature completely. Apple is apparently going for the former.We’re oversimplifying a bit here, but generally, LLMs like those used for Apple’s notification summaries work by predicting portions of words based on what came before and are not capable of truly understanding the content they’re summarizing.Further, these predictions are known to not be accurate all the time, with incorrect results occurring a few times per 100 or 1,000 outputs. Deploying this technology at scale without users really understanding how it works is risky at best, whether it’s with the iPhone’s summaries of news headlines in notifications or Google’s AI summaries at the top of search engine results pages. Even if the vast majority of summaries are perfectly accurate, there will always be some users who see inaccurate information.We wrote at length a few weeks ago about how the Apple Intelligence rollout seemed rushed, counter to Apple’s usual focus on quality and user experience—and with current technology, there is no amount of refinement to this feature that Apple could have done to reach a zero percent error rate with these notification summaries.We’ll see how well Apple does making its users understand that the summaries may be wrong, but making all iPhone users truly grok how and why the feature works this way would be a tall order.Update: The previous version of this article contained a sentence that said that the BBC had claimed the notifications came from an Apple-made app, but this was based on a misreading of the BBC’s statement. The sentence has been removed.Ars Technica has been separating the signal from
the noise for over 25 years. With our unique combination of
technical savvy and wide-ranging interest in the technological arts
and sciences, Ars is the trusted source in a sea of information. After
all, you don’t need to know everything, only what’s important.
