 Andrej Karpathy
|
38bfac90a8
|
bigchange: add multiquery support in run.c. we can now train and inference multiquery models (where n_kv_heads < n_heads). this also means that we, in principle, support Llama 2 34B and 70B models, which are multiquery
|
2023-08-13 19:34:05 +00:00 |
|
|
Andrej Karpathy
|
b0cfa2458d
|
ok i can train and sample a model with a custom tokenizer
|
2023-08-11 16:47:29 +00:00 |
|
 Daniel Gross
|
518524f458
|
default to whatever system has
|
2023-07-23 10:41:03 -07:00 |
|
|
Andrej Karpathy
|
9414e7a45e
|
tweaks and add a simple test
|
2023-07-23 14:52:08 +00:00 |
|
|
Andrej Karpathy
|
5b161abb9a
|
somewhere ~20 hours later
|
2023-07-23 05:23:45 +00:00 |
|