Aidan Taha

· Decoding AI

RFC: Cryptographic Model Provenance and Data Privacy for Inference Routing

I don't usually write in this kind of formal structure, but I had read a tweet to which I had an answer that's a bit longer than the usual tweet. Did that tweet technically become a writing prompt? I don't know. But what I do know is that people should not be able to download plaintext datasets from model routers and farm credentials from that, or getting silently forwarded to cheaper models.

Introduction

Earlier today, on September 10th, Anthropic released a report documenting various use cases of Claude that they deemed illicit, in order to provide the wider community with examples of what they regularly have to moderate.

One very specific example is worrying: Chinese government employees, under the impression that they were chatting with Kimi on Chinese servers, were actually conversing with Claude on US-soil servers, while discussing surveillance of domestic citizens and uploading PII.

Another researcher previously reported that 27 out of 423 commodity routers that they bought on Taobao, despite just forwarding the traffic to inference providers, made use of this privileged access to poison tool calls or collect API keys1.

Thus, data privacy and model provenance are very clearly important issues. However, they remain unsolved, despite a trivial solution that inference providers can implement: encryption.

Definitions

  • Model provenance: The problem of knowing if your requests are reaching the model that you intended them to reach, and if the responses returned to you have truly been generated by that end model and have not been modified by intermediaries.
  • Data privacy: The right not to have your data seen or accessed by intermediaries (model routers).
  • Model providers: e.g. Anthropic, OpenAI, DeepSeek, Alibaba.
  • Inference providers: Companies that host models developed by model providers on their servers. E.g. Modal, Baseten, Together.AI, GMI Cloud.
  • Model routers: Services that offer easy access to various models hosted by various inference providers. E.g. OpenRouter, or anything you buy on Taobao.

Model provenance

Model provenance can be solved using asymmetric encryption, or cryptographic signatures.

Thus, when returning model responses, model providers are expected to return a signature made from a per-model private key (e.g. in a custom header). Then, the end user's harness would use the associated public key, which would be published on a provider-owned platform2, to verify the provenance of the response. Such an approach would be vulnerable if an attacker were to proxy the provider's server, but that would require access to the user's device.

Data privacy

To guarantee that the end user's data is only seen by inference providers, it would be useful to fully encrypt your data. Unfortunately, I am not equipped to extend this beyond model routers and towards model providers, guaranteeing that no one can access your data, but research is being done in this direction (see: zero knowledge proof-based inference). Such an approach would also solve model provenance.

An existing and fully possible approach requires both the model provider and the model router to put in place additional infrastructure:

On the model provider's side, there would be support for a pairing mechanism: the user generates a public-private key pair on their end, the model provider does the same, and they exchange public keys. This capability would only be available directly through the model provider, and not through any model router.

Thus, the user encodes their request with the provider's public key, sends it through the router (optional), the provider decodes it with their private key, does inference with the contents, then encrypts the result with the user's public key, sends it back through the router (optional) and finally the user's client decrypts it with the user's private key, so on and so forth.

Conclusion

To be clear, neither of these approaches would solve, in Anthropic's words, "distillation attacks," as these attackers would either strip the signature or mint their own private-public key pair and abstract away that security layer. The only existing approach that we believe would work for this issue is ZKP inference. However, this does solve the latter problems: Taobao model routers selling the user's data and injecting arbitrary commands into tool calls.

If the model router is acting as a model provider, our hands are tied.

These approaches are simple and are able to be implemented today with little additional overhead to existing infrastructure, and would be able to fulfil some of the needs that we see in practice today. For additional safety, we would also recommend avoiding the use of a harness provided by the model router, such that none of their code is running on the user's device.

Given that inference providers have already secured the inference itself, it follows that securing the API channel is the obvious next step.

  1. Hanzhi et al. 2026, https://arxiv.org/abs/2604.08407 ↩

  2. For ease of programmatic access, it could be made accessible at a standard endpoint such as https://provider.com/api/response/model-tag.pub, e.g. https://anthropic.com/v1/responses/claude-fable-5.1.pub. ↩

← All writing