Google has shed some light on how it uses private sensor data from your Android phone to update its machine learning features, such as instant translation, without sending private data to its cloud servers.
Last September, Google introduced Live Caption, Now Playing and Smart Reply, features of Android’s Private Compute Core (PCC), a new type of isolated virtual sandbox in Android 12 and later, protecting the PCC and its features operating system and applications.
Google also introduced and more recently made available Private Compute Services (PCS), a “private route” to update and improve machine learning models without infringing on user privacy. The data processed in PCC is transferred via PCS to the Google cloud.
The detailed Private Compute Core architecture
The company has now given a more detailed description of PCC’s architecture, including a recently released technical document that aims to build trust through transparency.
“PCC allows features to communicate with a server to receive model updates and contribute to global model training through Private Computing Services (PCS), the core of which has been released for free” , explains Google in the document.
As Google engineers note, PCC can accommodate sophisticated ML features – such as Live Caption and Smart Reply, as well as blanking the screen when the user looks away – due to limitations placed on it. The CCP processes a large amount of sensitive data from the device, including audio data, images, text, operating system application data, and sensor data, including microphone, camera and GPS.
“The hosted functions themselves, running inside PCC, can be closed and updated. In this way, PCC enables machine learning functions to process ambient and OS-level data and improve over time, while limiting the availability of information about individual users to servers or apps. “, detail the Google engineers.
Preserve privacy
Ambient and OS-level data includes raw data from device sensors – such as camera, mic or screen content, data generated by analytics or inferences based on OS-level data and metadata.
Google engineers Dave Kleidermacher, Dianne Hackborn, and Eugenio Marchiori explain in a blog post that they use federated learning and federated analysis to update the ML models behind PCC functionality, while preserving the data confidentiality. Additionally, network calls to improve the performance of these models can be monitored with the PCS.
“The distributed trust paradigm, where credibility is built from verification by multiple sources of trust, continues to extend this core value. The opening of data protection mechanisms and processes is a step towards the verification of privacy, ”say Google engineers.
PCS is an APK, which provides application protocol interfaces for PCC components. The paper notes that PCS’s Federated Learning and Federated Analytics enables “privacy-preserving machine learning and analysis without centralized data collection.”
How does Federated Learning work?
Android sends aggregated data from many devices to Google’s cloud, but only as calculation results, after the calculation has been performed on the device using locally stored data. Since federated learning is difficult to explain, Google refers to its own comic explaining how it works.
“The underlying techniques consist of pushing a computational graph (e.g., machine learning model) to the device, performing computations on locally stored data, and returning only the results of the computations,” Google says in its technical document.
“Results from multiple devices are aggregated and used to improve device functionality and user experience. Individual device results are not visible to the orchestration server through the use of the multi-party Secure Aggregation compute protocol, which ensures that only aggregates from multiple devices (e.g., thousands) are made available servers and model and feature developers. »
Google invites researchers to analyze its claims and implementations of the PCC features detailed in the technical document.
Source: ZDNet.com