Commits

Branch selector

 main

User selector

vinx13

Datepicker

All time

Commit History

Commits on Oct 17, 2024

• [Doc] Add quantization documentation (#2984)

  Show description for 6da6aca
  vinx13

  authored
  6da6aca

  Copy full SHA for 6da6aca

Commits on Sep 30, 2024

• [Serving] Add error message for missing param in the loader (#2952)

  vinx13

  authored
  d516292

  Copy full SHA for d516292

• [Serving] Remove draft tokens after finishing request (#2953)

  Show description for 188e68e
  vinx13

  authored
  188e68e

  Copy full SHA for 188e68e

Commits on Sep 28, 2024

• [Fix] Fix draft token indices in eagle drafting (#2949)

  vinx13

  authored
  d77a1f4

  Copy full SHA for d77a1f4

Commits on Sep 7, 2024

• [Serving] Tree drafting (#2879)

  Show description for 20e7c4c
  vinx13

  authored
  20e7c4c

  Copy full SHA for 20e7c4c

Commits on Aug 15, 2024

• [Serving] Requiring explicit passing draft tokens to logit processor (#2810)

  Show description for 2d63553
  vinx13

  authored
  2d63553

  Copy full SHA for 2d63553

Commits on Aug 7, 2024

• [Config] Fix model limits (#2765)

  vinx13

  authored
  8099747

  Copy full SHA for 8099747

Commits on Aug 3, 2024

• [Bench] Allow running cuda-profile on existing mlc endpoint (#2737)

  vinx13

  authored
  01b5f68

  Copy full SHA for 01b5f68

• [Serving] Add prefill-mode to cli option (#2735)

  Show description for 28a1016
  vinx13

  authored
  28a1016

  Copy full SHA for 28a1016

Commits on Aug 2, 2024

• [Bench] Remove index column in csv output (#2728)

  vinx13

  authored
  9befc0d

  Copy full SHA for 9befc0d

Commits on Jul 30, 2024

• [Serving] Fix handling of num_tokens_for_next_decode in spec decoding (#2709)

  vinx13

  authored
  d54007b

  Copy full SHA for d54007b

• Turn on custom allreduce by default in O3 (#2706)

  vinx13

  authored
  3c7a6d5

  Copy full SHA for 3c7a6d5

Commits on Jul 26, 2024

• [Serving] Fix spec decoding call packed with rvalue (#2699)

  vinx13

  authored
  803becc

  Copy full SHA for 803becc

Commits on Jun 12, 2024

• [Serving] Apply tree structure in draft token verification (#2563)

  Show description for dcece51
  vinx13

  authored
  dcece51

  Copy full SHA for dcece51

Commits on Jun 5, 2024

• [FP8] Use f32 scale to enable better fusion (#2505)

  vinx13

  authored
  64e33c5

  Copy full SHA for 64e33c5

Commits on Jun 3, 2024

• Enable n-sampling for Medusa spec decoding (#2495)

  Show description for 91cc194
  vinx13

  authored
  91cc194

  Copy full SHA for 91cc194

Commits on Jun 1, 2024

• Fix tokenizers encode batch (#2484)

  vinx13

  authored
  589c76f

  Copy full SHA for 589c76f

Commits on May 30, 2024

• [CUDA] Turn on cuda graph at O2 (#2467)

  vinx13

  authored
  c4d337d

  Copy full SHA for c4d337d

Commits on May 29, 2024

• Implemented FP8 calibration (#2454)

  Show description for 249b945
  

  vinx13

  and

  MasterJH5574

  authored
  249b945

  Copy full SHA for 249b945

Commits on May 15, 2024

• Fix cublas offloading (#2343)

  vinx13

  authored
  2bbbd52

  Copy full SHA for 2bbbd52

Commits on May 14, 2024

• [Serving] Add Medusa speculative decoding (#2337)

  Show description for b247f8d
  vinx13

  authored
  b247f8d

  Copy full SHA for b247f8d

Commits on May 13, 2024

• [Eagle] Fix the requests for additional decode in eagle verify (#2336)

  vinx13

  authored
  821ee5d

  Copy full SHA for 821ee5d

Commits on May 12, 2024

• [Serving] Refactor to consolidate new request prefill (#2329)

  vinx13

  authored
  21feb70

  Copy full SHA for 21feb70

Commits on May 10, 2024

• Skip cublas dispatch for single batch (#2315)

  vinx13

  authored
  347222c

  Copy full SHA for 347222c

• [Model] Removing unnecessary reshapes in get_logits (#2314)

  vinx13

  authored
  b01cfab

  Copy full SHA for b01cfab

• [Serving] Log batch size in NVTX (#2312)

  vinx13

  authored
  ea391de

  Copy full SHA for ea391de

Commits on May 9, 2024

• Enable cuda graph for batch_verify (#2304)

  vinx13

  authored
  33c15e7

  Copy full SHA for 33c15e7

• [Sampler] Remove unneeded output_prob_dist param (#2300)

  vinx13

  authored
  10f3e4d

  Copy full SHA for 10f3e4d

Commits on May 7, 2024

• [Eagle] Run additional decode for draft model when all proposals are accepted (#2294)

  vinx13

  authored
  3621bf6

  Copy full SHA for 3621bf6

Commits on May 4, 2024

• [Eagle] Fix missing broadcast in hidden states gather/scatter (#2271)

  Show description for 8d58e52
  vinx13

  authored
  8d58e52

  Copy full SHA for 8d58e52

Commits on May 3, 2024

• [Eagle] Fix token shifting for prefill step (#2266)

  vinx13

  authored
  b124b0b

  Copy full SHA for b124b0b

• [Model] Fix llama2 chat template and remove redundant separator added by engine (#2264)

  Show description for 878be83
  vinx13

  authored
  878be83

  Copy full SHA for 878be83

Commits on May 2, 2024

• [Model] Remove redundant space in llama2 tokenizer (#2263)

  vinx13

  authored
  e756f23

  Copy full SHA for e756f23

Commits on May 1, 2024

• [Sampler] Add missing sync in gpu verifier (#2262)

  vinx13

  authored
  8e5af29

  Copy full SHA for 8e5af29

• [Sampler] Avoid unnecessary sync in GPU verifier (#2260)

  vinx13

  authored
  9941b4f

  Copy full SHA for 9941b4f

Pagination

Previous

Next