Direct connection
The page talks to the LM Studio HTTP server on this machine. Loaded models are discovered through the standard OpenAI-compatible endpoint. There is no relay, no tunnel, and no account.
GET /v1/models
Local inference lab · LM Studio
A live harness for LM Studio’s local server. Load a few models, pin a set of workloads, and get measured throughput, latency, and quality — ranked on the hardware in front of you. Nothing is uploaded, nothing leaves localhost.
no cloud · no uploads · localhost:1234
| # | Model | tok/s | time | Quality |
|---|
No archived runs yet — run a benchmark and the models will be kept here.
Inside the harness
Six design choices keep the numbers honest. No synthetic scoring, no remote backend, no calibration to look good.
The page talks to the LM Studio HTTP server on this machine. Loaded models are discovered through the standard OpenAI-compatible endpoint. There is no relay, no tunnel, and no account.
GET /v1/models
Sampling is pinned so models are compared under the same conditions: temperature 0.2, top-p 0.9, and no system prompt unless the workload requires one. Differences come from the model, not the settings.
temp 0.2 · top_p 0.9
Throughput uses the exact completion-token count divided by decode time only — prompt processing (TTFT) is measured separately.
decode tok/s · TTFT split out
Code generation, constrained reasoning, and long-form writing. Each stresses a different bottleneck: decode speed, planning depth, and context discipline.
code · reason · prose
Models are ordered by measured tokens per second. Quality is reported as a badge alongside, so a fast model that misses the task is easy to spot.
sort: mean tok/s
Completed runs are kept in the browser’s local storage. Reload the page, swap a quant, load a new model, and compare against the last run without a spreadsheet.
localStorage · archived on finish
The live run
Requires the LM Studio local server to be running (Settings → Server → Start Server). Models already loaded in LM Studio appear below.
Checking server…
No models found. Load one in LM Studio, then check the server again.
| Rank | Model | Mean tok/s | Best quality | Runs |
|---|
| Model | Workload | Tokens | TTFT | Total | tok/s (decode) | Quality |
|---|
Methodology
Five questions, answered plainly. If a number looks wrong, start here before blaming the model.
Tokens per second is the completion-token count divided by wall-clock time, timed from the request leaving this page to the full response arriving. When a workload runs more than one repeat, the cells are averaged before ranking.
Local inference is noisy. Thermal throttling, memory-bandwidth contention, and background processes all shift decode speed. Run several repeats and compare averages, not single spikes.
Each workload ships with a lightweight rubric checked against the response text: code fences present, the numeric answer extracted, a word-count band, or valid JSON. It is a coarse gate, not a judge.
Context is reported next to each model when the server exposes it. The harness never pads the prompt; the longest workload stays well under 2k tokens so decode speed remains the story.
Temperature and top-p are locked per workload so differences in output reflect the model. Change them if you like — the run header records exactly what was used.
Archive
Every finished run is written to local storage. Compare a new quant against last week without copying anything out.
No archived runs yet. Finish a benchmark and it will appear here.