TL;DR
Get business pricing on tech for your team
- Business-only prices and quantity discounts
- Tax-exempt purchasing
- Multiple users, one account, clear invoices
Strata, an open-source local inference project, says it can run the 125-billion-parameter Qwen3.8-Flash-Next model on supported gaming PCs. Its published tests report up to 94 generated tokens per second on an RTX 5070, not 100 trillion tokens per second; the supplied results do not document an RTX 4090 test.
The open-source Strata project says its software can run the 125-billion-parameter Qwen3.8-Flash-Next model on supported consumer PCs, with the project reporting up to 94 generated tokens per second on an RTX 5070. The supplied results do not show a test on an RTX 4090, and the claim in the topic wording of “100T/s” is not supported by the report: its speed figures are tokens per second, not trillions of tokens per second.
Strata’s published tests cover two systems: an RTX 5070 with 12 GB of VRAM, a Ryzen 5 7600 and 64 GB of system RAM; and an AMD RX 9070 XT with 16 GB, a Ryzen 9 3900X and 47 GB of RAM. On the NVIDIA system, the project reports generation speeds ranging from 53 to 94 tokens per second across listed model formats. Its fastest figure, 94 tokens per second, was for the Q2_0 format. The project says a token is about three-quarters of a word.
The benchmark also separates answer generation from processing an incoming prompt. For the NVIDIA tests, the listed prompt-processing rates range from 1,620 to 2,650 tokens per second; the AMD results range from 1,110 to 1,420. Strata says those prompt figures were measured with a 32,000-token prompt and that generation tests used 4,000-token answers. The NVIDIA Q2_0 result used engine version 0.1.36; the other NVIDIA rows used version 0.1.26.
The repository lists 12 GB of VRAM, at least 32 GB of system RAM and about 80 GB of disk space as baseline requirements. It says the model download is about 70 GB and that startup loads roughly 35–55 GB into RAM, with some memory reserved for the graphics card. The project describes the software as free and open source and says processing stays on the user’s PC.
Local Access to a 125B Model
The reported results point to a practical change in access: users with supported gaming hardware may be able to run a very large model without sending prompts to a remote service. Local use can matter to people working with private documents or code, though the project’s privacy statement describes its design and is not an independent security audit.
The speed figures also put the project’s claim in perspective. Strata says 60 generated tokens per second is faster than a person can read, but measured speed varies by format, hardware and software version. The benchmark is not evidence that every supported PC will match the fastest result, nor does it establish model quality for a particular task.
As an affiliate, we earn on qualifying purchases.
What Strata’s Tests Cover
Qwen3.8-Flash-Next is presented by the repository as a model that would usually require server hardware. Strata’s approach uses smaller, compressed model formats, trading some model size and potentially quality for the ability to run on a PC with limited graphics memory. The project’s recommendations vary with system RAM: it suggests Coder for 32 GB, IQ2_XS for 48 GB, and several options—including IQ3_S—at 64 GB.
The published headline figure in the supplied test table is from an RTX 5070, not an RTX 4090. Strata separately estimates that an RTX 3090 with 24 GB of VRAM should generate about 100–140 tokens per second, but labels that as an expectation rather than a measured result in the benchmark table. No RTX 4090 result is included in the material provided.
“Nothing leaves your PC.”
— Strata project, GitHub repository
high performance gaming PC with RTX 4090
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
RTX 4090 Results Are Missing
The supplied repository material does not give an RTX 4090 benchmark, nor does it explain the “100T/s” wording in the topic. Its results use tokens per second and report a maximum of 94 tokens per second in the listed RTX 5070 tests. The basis for any 4090 speed estimate is therefore unknown.
The report also does not provide independent verification of the benchmark setup, repeated-run variability, or a direct comparison with hosted services. Quantized formats can differ in output quality, while the project’s performance table does not establish how each option performs across a broad range of prompts or everyday applications. Its compatibility claims and estimated speed for other cards should be read as project guidance unless separately tested.
large VRAM graphics card for AI models
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
Further Tests Could Clarify Speed
Readers looking to verify the claim can consult Strata’s linked DETAILS.md tables and compatibility documentation, then compare results on their own hardware using the same model format and test conditions. A useful follow-up would be a clearly documented RTX 4090 run, including software version, prompt and output lengths, memory use and repeated measurements.
Strata says its installer detects the PC’s graphics card and RAM, recommends a model format, and can resume an interrupted download. The project also lists community results for other cards, but the supplied report does not identify a date or schedule for further benchmark releases.
As an affiliate, we earn on qualifying purchases.
Key Questions
Does the report show Qwen3.8-Flash-Next running on an RTX 4090?
No. The supplied benchmark table reports tests on an RTX 5070 and an AMD RX 9070 XT. It does not include an RTX 4090 result.
What does “100T/s” mean in the project’s results?
The supplied material does not support a rate of 100 trillion tokens per second. Strata reports speeds in tokens per second, with its listed RTX 5070 generation result reaching 94 tokens per second.
What hardware does Strata list as the minimum?
The project lists a supported graphics card with at least 12 GB of VRAM, 32 GB or more of system RAM, and about 80 GB of free disk space. It lists supported NVIDIA and AMD cards and Windows or Linux.
Does running the model locally mean prompts stay private?
Strata says its local setup keeps processing on the PC. That is the project’s description; the supplied material does not include an independent security or privacy audit.
Source: hn
Halloween Picks
halloween
As an affiliate, we earn on qualifying purchases.
