Commits

Branch selector

 main

User selector

pcuenca

Datepicker

All time

Commit History

Commits on Jul 11, 2025

• Update HubApi tests to account for resolve-cache URLs (#209)

  pcuenca

  authored
  e5bf062

  Copy full SHA for e5bf062

• Upgrade to jinja 1.2.1 (#208)

  pcuenca

  authored
  72600cf

  Copy full SHA for 72600cf

Commits on Jun 2, 2025

• Add some backwards-compatible getters (#206)

  Show description for 50e9c0d
  pcuenca

  authored
  50e9c0d

  Copy full SHA for 50e9c0d

Commits on May 28, 2025

• Disable network monitor win an env var (#203)

  pcuenca

  authored
  85b5bac

  Copy full SHA for 85b5bac

Commits on Jan 31, 2025

• Public tokenizer errors, hasChatTemplate (#171)

  Show description for 55710dd
  pcuenca

  authored
  55710dd

  Copy full SHA for 55710dd

• Add the canonical cached token location (#170)

  Show description for 47d6c65
  pcuenca

  authored
  47d6c65

  Copy full SHA for 47d6c65

Commits on Jan 27, 2025

• Override Llama tokenizer post-processor if necessary (#163)

  Show description for 9062cac
  pcuenca

  authored
  9062cac

  Copy full SHA for 9062cac

Commits on Jan 24, 2025

• Upgrade jinja (#160)

  Show description for 8a83416
  pcuenca

  authored
  8a83416

  Copy full SHA for 8a83416

• Deepseek R1 tokenization support and fixes (#159)

  Show description for 1fab24c
  pcuenca

  authored
  1fab24c

  Copy full SHA for 1fab24c

Commits on Jan 17, 2025

• Bert fixes (#157)

  Show description for 313fbd7
  pcuenca

  authored
  313fbd7

  Copy full SHA for 313fbd7

Commits on Jan 9, 2025

• Fix Phi 4 tokenization (#153)

  Show description for a867fea
  

  pcuenca

  and

  DePasqualeOrg

  authored
  a867fea

  Copy full SHA for a867fea

Commits on Dec 24, 2024

• Metadata test: use "resolve" URL

  pcuenca

  authored
  fc95ce1

  Copy full SHA for fc95ce1

Commits on Sep 30, 2024

• Llama test tokenizer uses the new merges style (#132)

  pcuenca

  authored
  71963c3

  Copy full SHA for 71963c3

Commits on Sep 27, 2024

• Support new merges serialization (#131)

  Show description for 4f31334
  pcuenca

  authored
  4f31334

  Copy full SHA for 4f31334

Commits on Sep 26, 2024

• Llama 3.2 tokenizer tests (#130)

  pcuenca

  authored
  0f23067

  Copy full SHA for 0f23067

Commits on Sep 1, 2024

• Chat templates by @maiqingqiang (#104)

  Show description for 5d89b5d
  

  pcuenca

  and

  johnmai-dev

  authored
  5d89b5d

  Copy full SHA for 5d89b5d

Commits on Aug 19, 2024

• Support "fuse unknown" configuration (#117)

  Show description for 1fdb4f4
  pcuenca

  authored
  1fdb4f4

  Copy full SHA for 1fdb4f4

• Tokenizer fixes (#113)

  Show description for 4c8cf07
  pcuenca

  authored
  4c8cf07

  Copy full SHA for 4c8cf07

Commits on Jul 18, 2024

• Update swift-argument-parser to >= 1.4.0 (#106)

  pcuenca

  authored
  e72d032

  Copy full SHA for e72d032

Commits on Apr 28, 2024

• Added Tokens (#93)

  Show description for 0a606f5
  pcuenca

  authored
  0a606f5

  Copy full SHA for 0a606f5

Commits on Mar 19, 2024

• CI: self-hosted runner (#75)

  Show description for 4f91561
  pcuenca

  authored
  4f91561

  Copy full SHA for 4f91561

Commits on Mar 18, 2024

• Make BertTokenizer public (#73)

  pcuenca

  authored
  0add1a4

  Copy full SHA for 0add1a4

Commits on Mar 15, 2024

• Export bosToken and eosToken (#70)

  pcuenca

  authored
  3bd0226

  Copy full SHA for 3bd0226

• Remove old Architecture.swift (#69)

  Show description for 07f6707
  pcuenca

  authored
  07f6707

  Copy full SHA for 07f6707

• Remove leftover function (#67)

  pcuenca

  authored
  5105f72

  Copy full SHA for 5105f72

• Fix Whisper tokenization tests (#68)

  Show description for a741be8
  pcuenca

  authored
  a741be8

  Copy full SHA for a741be8

Commits on Mar 13, 2024

• Fix download progress (#60)

  Show description for 95194e6
  

  pcuenca

  and

  davidkoski

  authored
  95194e6

  Copy full SHA for 95194e6

Commits on Feb 29, 2024

• Use fallback config if class not defined (#53)

  Show description for 03d86ac
  pcuenca

  authored
  03d86ac

  Copy full SHA for 03d86ac

• Gemma tokenizer, fix Unicode split (#52)

  Show description for bbbd7bf
  

  pcuenca

  and

  davidkoski

  authored
  bbbd7bf

  Copy full SHA for bbbd7bf

Commits on Feb 2, 2024

• Fix tokenizer api (#43)

  Show description for ff4026c
  pcuenca

  authored
  ff4026c

  Copy full SHA for ff4026c

Commits on Feb 1, 2024

• Generalized Unigram support, T5 (#38)

  Show description for afc5fe9
  pcuenca

  authored
  afc5fe9

  Copy full SHA for afc5fe9

Commits on Jan 12, 2024

• Fix Hub dependency in TokenizerTests

  pcuenca

  committed
  5e4031e

  Copy full SHA for 5e4031e

• Rename executable to avoid conflicts

  pcuenca

  committed
  e1137d0

  Copy full SHA for e1137d0

Commits on Jan 4, 2024

• Snapshot download from the Hugging Face Hub (#33)

  Show description for cb0ce3f
  pcuenca

  authored
  cb0ce3f

  Copy full SHA for cb0ce3f

Commits on Dec 29, 2023

• Public token <-> id conversion + Unknown Tokens (#32)

  Show description for 233aa86
  pcuenca

  authored
  233aa86

  Copy full SHA for 233aa86

Pagination

Previous

Next