BlinkDL/RWKV-LM
RWKV is an RNN with transformer-level LLM performance. It can be directly trained like a GPT (parallelizable). So it's combining the best of RNN and transformer - great performance, fast inference, saves VRAM, fast training, "infinite" ctx_len, and free sentence embedding.
Stars and forks stats for /BlinkDL/RWKV-LM
210 forks on 2023-03-14210 forks on 2023-03-15212 forks on 2023-03-16218 forks on 2023-03-17220 forks on 2023-03-18224 forks on 2023-03-19224 forks on 2023-03-20225 forks on 2023-03-21225 forks on 2023-03-22225 forks on 2023-03-23253 forks on 2023-03-24264 forks on 2023-03-25273 forks on 2023-03-26288 forks on 2023-03-27291 forks on 2023-03-28296 forks on 2023-03-29296 forks on 2023-03-30301 forks on 2023-03-31301 forks on 2023-04-01301 forks on 2023-04-02316 forks on 2023-04-03319 forks on 2023-04-04323 forks on 2023-04-05328 forks on 2023-04-06335 forks on 2023-04-07340 forks on 2023-04-08343 forks on 2023-04-09349 forks on 2023-04-10354 forks on 2023-04-11357 forks on 2023-04-12362 forks on 2023-04-13362 forks on 2023-04-14365 forks on 2023-04-15367 forks on 2023-04-16367 forks on 2023-04-17372 forks on 2023-04-18372 forks on 2023-04-19377 forks on 2023-04-20383 forks on 2023-04-21383 forks on 2023-04-22383 forks on 2023-04-23385 forks on 2023-04-24388 forks on 2023-04-25389 forks on 2023-04-26390 forks on 2023-04-27391 forks on 2023-04-28392 forks on 2023-04-29392 forks on 2023-04-30393 forks on 2023-05-01393 forks on 2023-05-02393 forks on 2023-05-03393 forks on 2023-05-04393 forks on 2023-05-05396 forks on 2023-05-06399 forks on 2023-05-07403 forks on 2023-05-08404 forks on 2023-05-09405 forks on 2023-05-10405 forks on 2023-05-11408 forks on 2023-05-12414 forks on 2023-05-13415 forks on 2023-05-14422 forks on 2023-05-15431 forks on 2023-05-16438 forks on 2023-05-17447 forks on 2023-05-18450 forks on 2023-05-19454 forks on 2023-05-20456 forks on 2023-05-21460 forks on 2023-05-22468 forks on 2023-05-23488 forks on 2023-05-24500 forks on 2023-05-25506 forks on 2023-05-26509 forks on 2023-05-27513 forks on 2023-05-28517 forks on 2023-05-29520 forks on 2023-05-30522 forks on 2023-05-31526 forks on 2023-06-01529 forks on 2023-06-02533 forks on 2023-06-03533 forks on 2023-06-04535 forks on 2023-06-05539 forks on 2023-06-06542 forks on 2023-06-07547 forks on 2023-06-08549 forks on 2023-06-09552 forks on 2023-06-10552 forks on 2023-06-11
552forks in total +355last 90 days
2 485 stars on 2023-03-142 485 stars on 2023-03-152 509 stars on 2023-03-162 534 stars on 2023-03-172 552 stars on 2023-03-182 583 stars on 2023-03-192 583 stars on 2023-03-202 632 stars on 2023-03-212 664 stars on 2023-03-222 664 stars on 2023-03-233 561 stars on 2023-03-243 762 stars on 2023-03-254 068 stars on 2023-03-264 276 stars on 2023-03-274 328 stars on 2023-03-284 480 stars on 2023-03-294 480 stars on 2023-03-304 606 stars on 2023-03-314 606 stars on 2023-04-014 606 stars on 2023-04-024 777 stars on 2023-04-034 827 stars on 2023-04-044 868 stars on 2023-04-054 960 stars on 2023-04-065 053 stars on 2023-04-075 118 stars on 2023-04-085 190 stars on 2023-04-095 286 stars on 2023-04-105 374 stars on 2023-04-115 436 stars on 2023-04-125 475 stars on 2023-04-135 475 stars on 2023-04-145 511 stars on 2023-04-155 549 stars on 2023-04-165 549 stars on 2023-04-175 596 stars on 2023-04-185 596 stars on 2023-04-195 677 stars on 2023-04-205 723 stars on 2023-04-215 723 stars on 2023-04-225 749 stars on 2023-04-235 793 stars on 2023-04-245 843 stars on 2023-04-255 869 stars on 2023-04-265 908 stars on 2023-04-275 935 stars on 2023-04-285 954 stars on 2023-04-295 968 stars on 2023-04-305 984 stars on 2023-05-015 984 stars on 2023-05-025 984 stars on 2023-05-035 984 stars on 2023-05-045 984 stars on 2023-05-056 098 stars on 2023-05-066 128 stars on 2023-05-076 150 stars on 2023-05-086 181 stars on 2023-05-096 212 stars on 2023-05-106 245 stars on 2023-05-116 343 stars on 2023-05-126 391 stars on 2023-05-136 428 stars on 2023-05-146 509 stars on 2023-05-156 632 stars on 2023-05-166 729 stars on 2023-05-176 806 stars on 2023-05-186 845 stars on 2023-05-196 880 stars on 2023-05-206 898 stars on 2023-05-216 930 stars on 2023-05-227 123 stars on 2023-05-237 429 stars on 2023-05-247 593 stars on 2023-05-257 658 stars on 2023-05-267 693 stars on 2023-05-277 744 stars on 2023-05-287 800 stars on 2023-05-297 864 stars on 2023-05-307 913 stars on 2023-05-317 953 stars on 2023-06-017 991 stars on 2023-06-028 042 stars on 2023-06-038 042 stars on 2023-06-048 075 stars on 2023-06-058 130 stars on 2023-06-068 185 stars on 2023-06-078 230 stars on 2023-06-088 269 stars on 2023-06-098 300 stars on 2023-06-108 300 stars on 2023-06-11
8.3kstars in total +6.1klast 90 days
This is stars and forks stats for
/BlinkDL/RWKV-LM repository. As of 11 Jun, 2023 this repository has
8300 stars and
552 forks.
The RWKV Language Model (and my LM tricks) RWKV: Parallelizable RNN with Transformer-level LLM Performance (pronounced as "RwaKuv", from 4 major params: R W K V) RWKV is an RNN with Transformer-level LLM performance, which can also be directly trained like a GPT transformer (parallelizable). And it's 100% attention-free. You only need the hidden state at position t to compute the state at position t+1. You can use the "GPT" mode to quickly compute the hidden state for the "RNN" mode. So it's combining...