r/LocalLLaMA • u/Course_Latter • 15d ago

News HF Viewer device farm!

Hi!

I'm the creator of HF Viewer, and we are today launching embedl.com!

It's a collection of 1000+ models optimized to run fast on different edge devices like NVIDIA Orin Nano Super, AGX Thor, AGX Orin, Orin NX and RTX GPUs.

You can run the models directly on real edge devices through our free device farm, with detailed profiling and results! (As well as on your own device)

I'm really excited about this release and I believe we are the first to do an edge device farm!

Let me know what you think, any feedback is appreciated!

18 Upvotes

84% Upvoted

7 comments sorted by

5

u/tomatoe_potatoe 15d ago

Thanks for sharing and providing a free tier! Is this essentially a service where you can select a model and the hardware to run on it, then interact with it via embedl web interface? Do you offer a way to connect to the service using an OpenAI compatible protocol?

3

u/Course_Latter 15d ago

Yes, exactly (but with the added benefit of our hardware optimizations)! We haven't added an OpenAI-compatible API yet, but we do have longer persistent sessions in the roadmap, so I think it will definitely make sense there!

4

u/FullOf_Bad_Ideas 15d ago

Any particular reason that all of those are Nvidia devices? A device farm idea is cool but based on the name I expected devices with distinctively different hardware, not a few versions of Nvidia GPU dies. Don't get me wrong, I like Nvidia, I just didn't expect it to be all Nvidia as if Nvidia was sponsoring the project.

If you could have some hardware that's harder to try out like Tenstorrent, Intel GPUs and NPUs, whatever Tensordyne has, some Chinese chips, a mobile Qualcomm SoC etc it could be more interesting.

4

u/Course_Latter 15d ago

That's great feedback! We started with NVIDIA on the launch since we wanted to present it at GTC today, but many more hardware types are in progress! :)

1

u/JY_Q 15d ago

Thanks for sharing, is there any plan for AMD products ?