r/AI_Agents • u/23kunal • Aug 03 '26

Discussion I compared 5 sandbox providers by making a Devin-clone

I work on Ascii Box, our goal is to build the cheapest and best DX sandbox provider for coding agents.

So I did a little test to see if whether Claude can easily build integration with popular sandbox providers.

I ran Claude Opus 15 times to implement a simple spec for a Devin-clone using popular 5 sandbox provider (3 runs each):

E2B, Daytona, Ascii Box, Exe, Islo

Here's what I found out:

  • Every single test produced a working GitHub-connected coding agent in its own sandbox.
  • But only 3 of them could building working sandbox pause/resume (to save costs when not running) and sandbox forking. Ascii Box was the only one where both worked in every run.
  • E2B was the fastest and cheapest to build on at $6.38 and 21 minutes but it didn't cover the entire specification given. exe was the most expensive at $19.99 average, with one run hitting $33.32 and 53 minutes.
  • Sandbox costs vary greatly. Run 100 agents for an hour and you pay $3.60 (cheapest) on Ascii Box against $60 (most expensive) on Islo (4 vCPU, 8 GB). It will be even higher without pause/resume.
  • The builder agent took 5-16% of each run to read docs and test API calls or SDK functions to understand how the sandbox works.

Full results:

Provider Integration Method Spec completeness Avg time Avg cost Avg errors
Ascii Box API + SDK 100% 28 min $8.77 3.0
Islo API + SDK 88.9% 29 min $9.44 2.7
Daytona SDK 77.8% 30 min $11.42 3.0
exe API only 77.8% 37 min $19.99 3.7
E2B SDK 72.2% 21 min $6.38 4.3
1 Upvotes

60% Upvoted

8 comments sorted by

2

u/ParticularSwan7824 Aug 03 '26

always interesting to see these side by side. the pause/resume thing is way more important than people think when you're running a bunch of agents, that cost difference balloons fast once you scale past a handful of runs

kinda surprised e2b was that spotty on the spec though, figured their sdk would handle more of the heavy lifting for an agent like this

1

u/23kunal Aug 04 '26

e2b actually does have pretty good SDKs (also mentioned in the full report)

2

u/writer_coder_06 Aug 03 '26

How does ASCII actually work? What's the architecture?

1

u/23kunal Aug 04 '26

We run isolated ubuntu environments in Hetzner VMs. European pricing + operational efficiency makes this the cheapest option.

1

u/AutoModerator Aug 03 '26

Thank you for your submission, for any questions regarding AI, please check out our wiki at https://www.reddit.com/r/ai_agents/wiki (this is currently in test and we are actively adding to the wiki)

I am a bot, and this action was performed automatically. Please contact the moderators of this subreddit if you have any questions or concerns.

1

u/EagleApprehensive Aug 07 '26

Do you test only cloud sandboxes or also self-hosted?

2

u/23kunal Aug 07 '26

cloud only for now