An interesting observation by Xiang Zhang:. TL;DR: number...
An interesting observation by Xiang Zhang:
TL;DR: number of parameters is a more determining factor than numerical precision for large language model performance. Given a memory constraint, one should maximize the number of parameters by quantizing at the highest level possible.
https://www.xzh.me/2023/09/a-perplexity-benchmark-of-llamacpp.html
TL;DR: number of parameters is a more determining factor than numerical precision for large language model performance. Given a memory constraint, one should maximize the number of parameters by quantizing at the highest level possible.
https://www.xzh.me/2023/09/a-perplexity-benchmark-of-llamacpp.html
Источник: gonzo-обзоры ML статей
2023-09-10 06:34:37