Cast by kenny

2 replies

• 

  kenny3h

  good thread walking through the tech here as well @jvaleska.eth

  

  @0xPD33

  The challenge of optimizing LLM inference on phones is way greater than it should be. This is partly because not many people are doing it, but also because on most systems it seems incredibly difficult to get proper access to the hardware underneath the OS. I'll break down what I've tried and wh…

  1 likes0 recasts1 replies
  

  jvaleska.eth3h

  yes, this was a very interesting reading, anyway, I dont think my phone can run any of these 😂

  2 likes0 recasts1 replies
  

  kenny3h

  def need a decently high-powered Android, mine isn't amazing (Pixel 8a) but I was lucky enough it just snuck under the bar for being capable enough

  2 likes0 recasts0 replies
• 

  jvaleska.eth3h

  the customizable knowledge feels interesting enough with an open source repo it means anyone can fork and create their own knowledge packs

  2 likes0 recasts0 replies