- Independent study
- 240 benchmark runs
- 8 harnesses
Independent studyDeepSeek V4 HarnessChoose by outcome, speed, or cost
Eight DeepSeek V4 harnesses compared on pass rate, speed and cost per success.
Picked a harness? Let a cloud agent run the repeat work.
Describe a task once. SpringBrand runs it on a schedule and sends you the result.
Opens SpringBrand in a new tab with your task filled in. 300 free credits to start.
- Connect SpringBrand MCP to DSHOne MCP sign-in gives your local agent live search, social data and 861 APIs.
- Run DSH tasks in the cloud 24/7Keep a job running on a schedule after your laptop goes to sleep.
- Estimate your DeepSeek API billTurn tokens, cache hits and peak hours into a monthly cost.
- Fix an install errorPort 3080 in use, command not found, Node version and four more fixes.
In an independent DeepSeek V4 Flash study, Pi had the highest reported pass rate, Claude Code was fastest, and DeepAgents had the lowest directly comparable cost per success. DeepSeek's own DSH launched two days after the study, so it was not tested.
Independent guide · Study and official release dates checked 18 August 2026.
Pi leads on reported completion, but its setup differs. Claude Code leads on median speed. DeepAgents is the clearest low-cost result. Choose the metric that matches your workload.
Benchmark facts at a glance
- Harnesses
- 8
- Tasks per harness
- 30
- Total runs
- 240
- Reported successes
- 129 (53.8%)
- Highest pass rate
- Pi · 66.7%*
- Fastest median
- Claude Code · 122.7s
- Lowest comparable cost
- DeepAgents · $0.045
- Official DSH
- Not included
Recommendation tool
Which DeepSeek V4 harness fits your goal?
Pick the outcome you value most. The answer uses the published study results and flags conditions that make a number harder to compare.
Recommendation
Highest reported pass rate: Pi
Pi completed 20 of 30 tasks (66.7%), but the study used a different reasoning setting and two model providers. OMP is the cleaner comparison when you want fewer test-condition caveats.
Published results
DeepSeek V4 harness benchmark: 8 agents, 240 runs
Composio ran 30 coding tasks per harness and reported 129 successful runs out of 240 (53.8%). Sort the table by the metric you care about.
| Harness | Pass rate | Median time | Cost / success | Use this result when… |
|---|---|---|---|---|
| Pi | 66.7% | 132.2s | $0.028* | You accept the different reasoning/provider setup. |
| Prime | 62.5%* | 242.1s | $0.131 | You account for 6 runs that could not be scored. |
| OMP | 56.7% | 272.4s | $0.103 | You want the strongest less-qualified pass result. |
| Claude Code | 53.3% | 122.7s | $0.195 | Median completion speed matters most. |
| Codex | 53.3% | 245.0s | $0.081 | You want a middle-cost coding agent baseline. |
| DeepAgents | 53.3% | 187.1s | $0.045 | You want the lowest directly comparable cost. |
| Hermes | 50.0% | 175.5s | $0.056+ | You value low runtime-token use (~192K/task). |
| OpenCode | 46.7% | 129.7s | $0.073 | You value speed and can accept the lower pass rate. |
* Pi used a different reasoning configuration and two providers. Prime had 24 valid results and 6 runs that could not be scored. Hermes cost excludes an additional component reported by the study.
Read before choosing
Three details change what the DeepSeek V4 harness numbers mean
The headline ranking is useful only when its test date and setup match the decision you are making now.
DSH was not tested
The study was published 11 August. Official DeepSeek Harness launched 13 August, so DSH cannot be placed in this table.
Costs are historical
DeepSeek changed V4 pricing on 16 August. The cost-per-success values describe the study period, not today's budget.
Pass rate hides setup differences
Pi's leading number used a different reasoning setting and two providers. Prime's percentage uses only 24 valid runs.
Start with Pi if you can reproduce its exact setup, OMP if you want fewer comparison caveats, Claude Code for median speed, or DeepAgents for the clearest cost result. Then rerun three tasks from your own workload.
Official option
Where official DeepSeek DSH fits
DSH is the most direct DeepSeek-owned harness, with Standard, Code, Minimal and Creator modes plus a local Web UI. The benchmark above gives no basis for calling it faster, cheaper or more accurate than the eight tested tools.
Choose DSH for first-party fit
Use it when DeepSeek's model integration, replaceable plugins and local inspection matter more than a cross-harness score.
Choose Minimal for your test
Minimal mode reduces harness assistance to persistent shell and file editing, making task comparisons easier to interpret.
Re-price every run
Use the current DeepSeek pricing calculator; do not reuse the benchmark's August 11 cost figures.
npx @deepseek-ai/dsh web
Local DSH vs cloud agent
Keep DSH for local work. Run long and scheduled jobs in the cloud.
DeepSeek Harness runs on your computer, which is the right place for code and private files. Jobs that must run overnight, on a schedule or for a team need a machine that never sleeps.
Long tasks
Local DSH
Runs inside your local session. If the session crashes or the laptop sleeps, the run stops with it.
A 40-minute DSH task crashed at minute 39 (GitHub discussion #2564)SpringBrand cloud agent
Runs on SpringBrand servers 24/7, so closing the laptop does not stop the job.
Schedules
Local DSH
Users schedule runs with an external cron job or a community plugin, and the computer has to be on at that time.
Before a community plugin, scheduling meant external cron (GitHub discussion #1600)SpringBrand cloud agent
Built-in schedules: daily, weekly or at a set time.
Check from your phone
Local DSH
Remote access only works while your computer is online. In the community desktop app, a dropped phone session has to be scanned and approved again on the computer.
Phone remote sessions cannot reconnect after a drop (dataelement/dsh-desktop issue #179)SpringBrand cloud agent
Open results in the browser on any device. Your computer can stay off.
Share with a team
Local DSH
Sessions and settings live in your own ~/.dsh folder.
SpringBrand cloud agent
Share runs and results with your team, or publish an agent to the marketplace.
Tools
Local DSH
The MCP servers and plugins you find, vet and configure yourself, one key per provider.
Users ask for review of third-party plugins for safety (GitHub discussion #1115)SpringBrand cloud agent
One catalog of 861 APIs from 28 providers behind a single sign-in.
Cost
Local DSH
DSH itself is free and MIT-licensed. You pay your model provider for tokens and keep your computer running.
SpringBrand cloud agent
Starts with 300 free credits. Each run spends credits.
SpringBrand is independent, not affiliated with DeepSeek. Evidence links point to public GitHub posts, checked 25 October 2026.
FAQ
DeepSeek V4 harness questions
DeepSeek V4 harness questions from the reference page.
- Independent benchmark
- DSH not tested
- Re-run your own tasks
Pi has the highest reported pass rate in the August 11 study, but its setup differs. Claude Code is fastest by median time, while DeepAgents has the lowest directly comparable cost per successful task.
Sources
Sources and benchmark boundaries
Independent guide · Study and official release dates checked 18 August 2026.
- Composio DeepSeek V4 Flash harness study
Published pass rate, median time, cost, token, and setup notes for eight harnesses.
- Official DeepSeek Harness repository
DSH ownership, local Web UI, modes, plugins, and release timing.
- Official DeepSeek pricing
Current model pricing used when evaluating new runs after the study.
Next step
Picked a harness? Run the job on a schedule.
Local harnesses stop when your laptop sleeps. SpringBrand runs agent tasks in the cloud on a daily or weekly schedule, with search, social data and generation tools built in, and 300 free credits to start.
Want the raw numbers behind the table? Read the independent Composio study
SpringBrand is independent, not affiliated with DeepSeek. DeepSeek Harness (dsh) is an open-source project by DeepSeek AI; community desktop builds belong to their own maintainers.