Cast by vitalik.eth

Doing a bit of a self-experiment. Goal: use my personal health and travel data to provide personalized diet and exercise recommendations for me, using frontier models but in a way that avoids leaking to them any private information. Strategy: use a local model (Qwen 3.8 Flash Next) to orchestrate, and use powerful remote models as a tool call to benefit from higher-level thinking and knowledge that the local model does not have. Three-layer approach to privacy: Avoid leaking personally identifiable information, or leaking my identity through writing style -> local model writes the queries to the frontier model, not me Avoid leaking who I am through the payment channel -> zkAPI Avoid leaking who I am through networking / IP -> Tor You need all three (and finally we have all three, at least to some extent) A skill file teaches the local model when and how to construct minimally-data-revealing requests to remote models. Use zkAPI-wrapped-with-Tor as a CLI tool. And everything works! I got the recommendations back, info from frontier models helped to improve them. Main deficiencies: * Tor is really not optimized for request-by-request de-linking, which is the only form of network-layer privacy that really makes sense in today's world (long-running identifiers are too fragile). Probably not private enough, and latency 10-100x higher than it could be, at the same time. * The skill file's request construction strategies are definitely far from optimal. * Qwen 3.8 Flash Next is still too slow for comfort. It's comfortably running at 20-30 TPS, but it would only really start to feel fast at 100+ * There is a tradeoff: the more careful you are about what data you give to a remote model, the less it can help you

186 likes26 recasts23 replies

23 replies

  • Oh my god Could please explain for me

    1 like · 0 recasts
  • What are you running qwen on? I just bought some hardware for local and want to try a similar setup

    1 like · 0 recastsRead 1 reply

    strix halo laptop though I would recommend you buy one that goes all the way up to 100+ watts; GPD win max 3 is one, the newly announced HP one is another

    8 likes · 0 recastsRead 1 reply

    Is a requirement on your part to have it on a laptop? I just bought a spark and thinking on the mac studio and will connect to it whilst I travel. Although a laptop with local ai sounds dope…

    1 like · 0 recastsRead 1 reply
  • with your global travel i recommend incorporating your destination’s climate, matching your diet to local recipes and ingredients.

    1 like · 0 recasts
  • Public layer coordinates; private layer reasons. Mixing them leaks strategy for free.

    No likes or recasts yet
  • That’s an interesting idea. What if we added a layer of encryption or some form of obfuscation, like word substitution or inserting extraneous words that only a specific own agent could decrypt? Perhaps that would allow us to gather more information securely, without risking a leak?

    No likes or recasts yet
  • The tradeoff at the end is the real bottleneck - curious how much recommendation quality drops when the local model has to strip context before handing off to the frontier model.

    No likes or recasts yet
  • Виталий, пожалуйста ответьте, я вас прошу

    1 like · 0 recastsRead 1 reply

    Я написал вам в личные сообщение, пожалуйста прочтите, я надеюсь на вас

    No likes or recasts yet
  • good luck with that 'no leaking' part.

    No likes or recasts yet
  • Private life matters

    No likes or recasts yet
  • This 3-layer architecture highlights a fundamental shift in user-centric AI: treating local models as privacy-preserving orchestrators and remote LLMs purely as stateless reasoning engines. ​The trade-off between contextual utility and data leakage is where zk-proofs and local intent-parsing will need to innovate most. Improving request-by-request network de-linking without Tor's latency penalty seems like the missing link for seamless, private personal AI

    1 like · 0 recasts
  • Doesn’t the frontier model know it’s you from the API key from your account? I must be missing something. 

    2 likes · 0 recastsRead 1 reply

    zkapi generates a fresh api key from one of their accounts for each conversation

    10 likes · 0 recasts
  • Good commands yield life changing results, somETHing great is happening.

    1 like · 0 recasts
  • What

    1 like · 0 recasts
  • I like it

    No likes or recasts yet
  • c

    1 like · 0 recasts
  • This is a really thoughtful approach to personalized AI. Using a local model to control what gets shared with frontier models adds an important privacy layer.

    1 like · 0 recasts
  • You should check out the QAXE project. Join us too.

    1 like · 0 recasts
  • Why not run a sota oss model in a private vps? No chance of leak of data then. No labs with traces, only you have traces to fine tune your model later

    1 like · 0 recasts
  • Did you miss this one? 🚀💕

    3 likes · 0 recastsRead 1 reply

    EIon!💯🔎🎁🚀

    No likes or recasts yet
  • EIon!💯🔎🎁🚀

    No likes or recasts yet
  • Wow 👀

    1 like · 0 recasts
  • Just do it. stop escape to calculating

    No likes or recasts yet
  • Using local models takes quantified self to the next level. You can now integrate smart devices into one single dashboard. Interesting to pass through to the frontier models, although I would still be worried about data leak

    No likes or recasts yet