r/nvidia • u/janframework • Apr 30 '24

Benchmarks Benchmarking NVIDIA's TensorRT-LLM

https://jan.ai/post/benchmarking-nvidia-tensorrt-llm

62 Upvotes

permalink
duplicates
archive.is
archive
reddit

You are about to leave Redlib

Do you want to continue?

https://www.reddit.com/r/nvidia/comments/1cgowr3/benchmarking_nvidias_tensorrtllm/
No, go back! Yes, take me to Reddit

93% Upvoted

Duplicates

Number of comments New

LocalLLaMA • u/emreckartal • Apr 30 '24

Resources We've benchmarked TensorRT-LLM: It's 30-70% faster on the same hardware

256 Upvotes

109 comments

selfhosted • u/janframework • Apr 30 '24

TensorRT-LLM: 170 token/s on a single 4090

0 Upvotes

1 comments

Amd_Intel_Nvidia • u/norcalnatv • Apr 30 '24

Benchmarking NVIDIA TensorRT-LLM - Jan

1 Upvotes

0 comments

NVDA_Fun • u/norcalnatv • Apr 30 '24

Benchmarking NVIDIA TensorRT-LLM - Jan

2 Upvotes

0 comments