Cast by vitalik.eth
Doing a bit of a self-experiment. Goal: use my personal health and travel data to provide personalized diet and exercise recommendations for me, using frontier models but in a way that avoids leaking to them any private information. Strategy: use a local model (Qwen 3.8 Flash Next) to orchestrate, and use powerful remote models as a tool call to benefit from higher-level thinking and knowledge that the local model does not have. Three-layer approach to privacy: Avoid leaking personally identifiable information, or leaking my identity through writing style -> local model writes the queries to the frontier model, not me Avoid leaking who I am through the payment channel -> zkAPI Avoid leaking who I am through networking / IP -> Tor You need all three (and finally we have all three, at least to some extent) A skill file teaches the local model when and how to construct minimally-data-revealing requests to remote models. Use zkAPI-wrapped-with-Tor as a CLI tool. And everything works! I got the recommendations back, info from frontier models helped to improve them. Main deficiencies: * Tor is really not optimized for request-by-request de-linking, which is the only form of network-layer privacy that really makes sense in today's world (long-running identifiers are too fragile). Probably not private enough, and latency 10-100x higher than it could be, at the same time. * The skill file's request construction strategies are definitely far from optimal. * Qwen 3.8 Flash Next is still too slow for comfort. It's comfortably running at 20-30 TPS, but it would only really start to feel fast at 100+ * There is a tradeoff: the more careful you are about what data you give to a remote model, the less it can help you
23 replies
What are you running qwen on? I just bought some hardware for local and want to try a similar setup
strix halo laptop though I would recommend you buy one that goes all the way up to 100+ watts; GPD win max 3 is one, the newly announced HP one is another
Is a requirement on your part to have it on a laptop? I just bought a spark and thinking on the mac studio and will connect to it whilst I travel. Although a laptop with local ai sounds dope…
with your global travel i recommend incorporating your destination’s climate, matching your diet to local recipes and ingredients.
Public layer coordinates; private layer reasons. Mixing them leaks strategy for free.
That’s an interesting idea. What if we added a layer of encryption or some form of obfuscation, like word substitution or inserting extraneous words that only a specific own agent could decrypt? Perhaps that would allow us to gather more information securely, without risking a leak?
The tradeoff at the end is the real bottleneck - curious how much recommendation quality drops when the local model has to strip context before handing off to the frontier model.
Я написал вам в личные сообщение, пожалуйста прочтите, я надеюсь на вас
This 3-layer architecture highlights a fundamental shift in user-centric AI: treating local models as privacy-preserving orchestrators and remote LLMs purely as stateless reasoning engines. The trade-off between contextual utility and data leakage is where zk-proofs and local intent-parsing will need to innovate most. Improving request-by-request network de-linking without Tor's latency penalty seems like the missing link for seamless, private personal AI
Doesn’t the frontier model know it’s you from the API key from your account? I must be missing something. 
zkapi generates a fresh api key from one of their accounts for each conversation
This is a really thoughtful approach to personalized AI. Using a local model to control what gets shared with frontier models adds an important privacy layer.
Why not run a sota oss model in a private vps? No chance of leak of data then. No labs with traces, only you have traces to fine tune your model later
Using local models takes quantified self to the next level. You can now integrate smart devices into one single dashboard. Interesting to pass through to the frontier models, although I would still be worried about data leak

