d47fc41b6a
Merge branch 'ci-tiny-model' of https://github.com/Majdoddin/llama2.c into Majdoddin-ci-tiny-model
2023-08-16 02:20:34 +00:00
ca67253f28
smallfix: not sure what the point of this indirection was
2023-08-15 16:09:33 +00:00
4c63c5608d
shorten top comment on run.c file
2023-08-15 16:07:48 +00:00
a47f9b3969
collapsing copy paste code because it's driving my ocd crazy
2023-08-15 16:03:11 +00:00
87b11edf27
modifiying test_all so it can safely run on windows
2023-08-15 16:01:53 +00:00
66c9f5e6c8
Adding pytest with the tiny model to macOS and windows (except amd64_arm64) runners
2023-08-15 15:58:04 +00:00
88eb238255
add tests into Makefile convenience
2023-08-15 15:57:27 +00:00
600cedb33d
Merge pull request #297 from karpathy/feature/utf8
...
Add UTF-8 support to prompts
2023-08-14 19:54:49 -07:00
fe2de68688
fix sample.py from tokenizer changes before
2023-08-15 02:33:01 +00:00
a9a0628c92
thoroughly commented the UTF-8 byte reading code
2023-08-15 02:18:49 +00:00
d459fd4243
add back careful processing of the byte tokens
2023-08-15 01:42:33 +00:00
4bf36ecc17
get rid of the special byte decoding logic
2023-08-15 01:04:10 +00:00
8417cb438d
Merge branch 'utf8' of https://github.com/atamurad/llama2.c into feature/utf8
2023-08-15 00:18:53 +00:00
94a3a5e0a5
Merge branch 'master' of github.com:karpathy/llama2.c
2023-08-14 14:52:15 +00:00
32c1ff97fb
missed p->dim to kv_dim for k,v vectors. we're not doing anything wrong we're just being wasteful with memory. thanks @xefoci7612 for pointing out
2023-08-14 14:52:07 +00:00
013e012b87
Merge pull request #286 from Nick-infinity/master
...
[Feat]: Add support for meta llama hf model conversion
2023-08-14 07:46:39 -07:00
50f970d170
Merge pull request #289 from chenyangMl/update_readme
...
Update readme to introduce llama2.c-zh
2023-08-14 07:41:13 -07:00
2a9a4c4e14
update readme wiht a simple line to introduce llama2.c-zh
2023-08-14 15:12:30 +08:00
79900ff68e
update readme wiht a simple line to introduce llama2.c-zh
2023-08-14 15:00:33 +08:00
82ad2ba34e
remove tiktoken as dependency
2023-08-14 05:53:57 +00:00
c39f19f1a9
[Feat]: Add support for meta llama hf model conversion
...
Description:
Llama 2 hf models have weights stored with diff name
Signed-off-by: Nikhil Gupta <nikhilg.me@gmail.com >
2023-08-14 10:18:51 +05:30
bae0bcf484
Small tweaks to Readme intro
2023-08-13 20:03:00 -07:00
45afa91dca
the accum function has been bothering me, there is no real need to add a function here, it does something trivial and is only used twice, scrap
2023-08-14 02:54:27 +00:00
854c97b660
turn topp 0.9 back on by default thanks to recent PR contributions truncating before quicksort
2023-08-14 00:12:45 +00:00
4a2c375df9
Merge pull request #276 from jrudolph/improve-top-p
...
optimize sample_topp by filtering out small value elements up front
2023-08-13 17:05:38 -07:00
b3d6a9e6b5
Merge pull request #285 from karpathy/feature/civ2
...
Upgrading CI to run our new pytest
2023-08-13 16:55:01 -07:00
091c799653
Merge branch 'master' into feature/civ2
2023-08-13 16:54:24 -07:00
c970f69334
oops i should probably call this function lol
2023-08-13 23:48:01 +00:00
223a67048a
add optional manual dispatch of actions
2023-08-13 23:39:37 +00:00
86325bf7e8
attempt to upgrade the CI to run our pytest
2023-08-13 23:35:29 +00:00
b51c63b9f2
Merge pull request #283 from wizzard0/wizzard0-mention-1
...
Add TypeScript port
2023-08-13 14:36:10 -07:00
8506036185
remove 'revive tests' as a todo from the readme
2023-08-13 21:23:27 +00:00
f0024cfc88
revive tests. now that we have a tiny stories260K model this only requires a 2MB download. phew
2023-08-13 21:22:44 +00:00
0805cb2c31
tiny whitespace fix to try to eliminate scrollbar
2023-08-13 13:40:09 -07:00
b2cce341e0
oops typo fix in readme
2023-08-13 13:39:12 -07:00
3e989e21f2
link to stories260K model
2023-08-13 20:38:05 +00:00
58075b5ac5
update API of sample.py to be better, small changes here
2023-08-13 20:31:32 +00:00
36b54321e5
bugfix: allocate +1 in tokens buffer for dummy whitespace
2023-08-13 23:23:32 +03:00
1bcb2d18d6
Merge pull request #284 from karpathy/feature/customtokenizer
...
multiquery support add
2023-08-13 12:38:06 -07:00
38bfac90a8
bigchange: add multiquery support in run.c. we can now train and inference multiquery models (where n_kv_heads < n_heads). this also means that we, in principle, support Llama 2 34B and 70B models, which are multiquery
2023-08-13 19:34:05 +00:00
b28c1e26c5
Merge pull request #275 from icppWorld/webassembly-internet-computer
...
Notable fork section for WebAssembly
2023-08-13 10:14:39 -07:00
5295cbb821
Merge pull request #281 from lintian06/original_llama2
...
Update README.md for a new rust port.
2023-08-13 10:14:00 -07:00
12dec61fbf
Merge pull request #282 from mihainadas/master-1
...
Fixes https://github.com/karpathy/llama2.c/issues/280
2023-08-13 10:13:08 -07:00
0e6213c6e0
Mention I can run the full 7B model
2023-08-13 20:02:34 +03:00
1d68a36d14
Add TypeScript port
...
I've never been so happy to have missed that the JS port already exists :D also it was nice to discover that the JS can reach 80% of the single-threaded C speed (10 tokens/s for TinyStories-110M)
2023-08-13 19:10:07 +03:00
570789aa04
Fixes https://github.com/karpathy/llama2.c/issues/280
...
There was a small bug in tinystories.py, described here: https://github.com/karpathy/llama2.c/issues/280
This commit simply passes vocab_size to get_tokenizer_model_path to avoid silent crash when processing shards (in process_shard)
2023-08-13 17:49:10 +03:00
27adb082f1
Update README.md
2023-08-13 21:58:14 +08:00
daa9fd9b8a
sort vocabulary for faster lookup with bsearch()
2023-08-13 15:02:11 +03:00
8b472ded1f
Merge pull request #272 from karpathy/feature/customtokenizer
...
Big Change: Custom Tokenizer training: add the ability to train custom tokenizers instead of using the pretrained Llama 2 tokenizer. This is useful in custom, narrow-domain LLMs because smaller vocab sizes make much smaller, faster, and potentially more capable models. For example, in tinystories a vocab size 4096 custom tokenizer compresses the input text sequences about as well as the Llama 2 tokenizer with vocab size 32000. The result is also "safer" because a badly trained model can't accidentally e.g. output some random chinese character and rapidly go "off the rails" in subsequent tokens.
2023-08-12 20:31:21 -07:00
9ff459b925
todo changes
2023-08-13 03:24:31 +00:00
Andrej Karpathy
Ruhollah Majdoddin
chenyang
Nikhil Gupta
atamyrat
Oleksandr Nikitin
Mihai Nadăș
Tian Lin