Wanderer atop the sea of clouds or whatever on Nostr: verita84 baunilha How much VRAM do you have currently? I can benchmark my tok/s at ...
verita84 (npub1fdt…uwwr) baunilha (npub1cp7…nqrh) How much VRAM do you have currently? I can benchmark my tok/s at different context sizes if you'd like that. At 130k I get about 25 tok/s.