makeasnek@lemmy.ml to AI@lemmy.mlEnglish · 5 months agoLLM ASICs on USB sticks?lemmy.mlimagemessage-square15fedilinkarrow-up127arrow-down16file-textcross-posted to: [email protected]
arrow-up121arrow-down1imageLLM ASICs on USB sticks?lemmy.mlmakeasnek@lemmy.ml to AI@lemmy.mlEnglish · 5 months agomessage-square15fedilinkfile-textcross-posted to: [email protected]
Source: nostr https://snort.social/nevent1qqsg9c49el0uvn262eq8j3ukqx5jvxzrgcvajcxp23dgru3acfsjqdgzyprqcf0xst760qet2tglytfay2e3wmvh9asdehpjztkceyh0s5r9cqcyqqqqqqgt7uh3n Paper: https://arxiv.org/abs/2406.02528
minus-squarekakeslinkfedilinkarrow-up10·5 months agoNever really occurred to me before how huge a 10x savings would be in terms of parameters on consumer hardware. Like, obviously 10x is a lot, but with the way things are going, it wouldn’t surprise me to see that kind of leap in the next year or two tbh.
Never really occurred to me before how huge a 10x savings would be in terms of parameters on consumer hardware.
Like, obviously 10x is a lot, but with the way things are going, it wouldn’t surprise me to see that kind of leap in the next year or two tbh.