Pinned post
Here is our new quantization of a 3-trillion parameter model.
Yes it runs on your 12GB GPU!!!
(Disclaimer: Also requires 1TB of DDR50 RAM, a direct liquid nitrogen line from the local hospital, and a prayer to the tech gods. Side effects include 0.5 tokens per year and your motherboard physically melting.)
Yes it runs on your 12GB GPU!!!
(Disclaimer: Also requires 1TB of DDR50 RAM, a direct liquid nitrogen line from the local hospital, and a prayer to the tech gods. Side effects include 0.5 tokens per year and your motherboard physically melting.)