This is the source code for Local Browser AI.
Inference timestamps:
TStart: timestamp of starting the inferenceTFirstTok: timestamp of receiving the first token (when not streaming, it's equal toTD)TEnd: timestamp of ending the inference (including if it was interrupted)
Inference durations:
| Duration | Streaming | Not Streaming | Description |
|---|---|---|---|
TD |
TD = TEnd - TStart |
TD = TEnd - TStart |
Total duration of inference. |
TTFT |
TTFT = TFirstTok - TStart |
TD |
Time to first token. |
ID |
TEnd - TFirstTok |
TD |
The total time it took to generate tokens. |
TPS |
T / ID |
T / ID |
Tokens per second (during ID, not TD). |
- Chrome Web Store
- Microsoft Edge Add-ons
- For development, head over to chrome://extensions/ and click
Load unpackedand point to where you have cloned this repo.
- A graphic card with at least 4GB VRAM.
- Windows, Linux, Mac, or Chrome OS.
- At least 20GB storage and a good enough internet.
- See the full requirements.
We lightly use esbuild for absolute minimum (packaging).
npm run buildWe use the Node.js native testing framework.
npm testWe use npm as a package manager but since we don't want to build the entire code base (currently at least),
the packages are exported to the vendors directory and updated using [update-vendors.js].
It's a bit unconventional, I know, but it's my code base and I'm experimenting with tooling.
The idea is for the main application code to go to the browser as is without any transformation. We'll see where we end up. 🙂
- Prompt API proposal
- Chrome Developer Docs for extensions: Prompt API
- Chrome Developer Docs: Prompt API
- Chrome Generative AI Privacy, Terms and Policy
- Chrome: Built-in AI
- Microsoft: Prompt API
Made in Sweden by Alex Ewerlöf