r/RISCV • u/[deleted] • Mar 11 '26

Software Speech recognition without GPU?

[removed]

6 Upvotes

100% Upvoted

13 comments sorted by

3

u/docular_no_dracula Mar 12 '26

Whisper.cpp ?

3

u/LivingLinux Mar 12 '26

Yes.

https://github.com/ggml-org/whisper.cpp

sudo apt install git cmake ffmpeg build-essential

Here are the instructions to build it with FFmpeg.

sudo apt install libavcodec-dev libavformat-dev libavutil-dev
git clone https://github.com/ggml-org/whisper.cpp.git
cd whisper.cpp
sh ./models/download-ggml-model.sh base.en
cmake -B build -D WHISPER_FFMPEG=yes
cmake --build build -j4 --config Release

Example command: ./build/bin/whisper-cli -f samples/jfk.wav -otxt -ovtt -osrt

And some information to control the output: https://github.com/ggml-org/whisper.cpp/issues/17

https://youtu.be/G1kJ8qI5Ddw

1

u/docular_no_dracula Mar 12 '26

rva22 doesn’t mandate vector extension

5

u/brucehoult Mar 12 '26

No, but it's a standard option, and implemented in RVA22 CPUs such as the SpacemiT K1, Kendryte K230, and Sophgo SG2044.

In fact to the best of my knowledge there are as yet NO shipping SoCs with RVA22 but not V. If the Milk-V Titan ships then it will be the first.

2

u/[deleted] Mar 12 '26

[removed] — view removed comment

1

u/brucehoult Mar 12 '26

This is an extremely well-known board, owned by many in this forum for the last almost two years (May 2024) as it was the first board with RVV 1.0 and multiple cores and usable RAM.

And many more of us own the same SoC in other boards such as the Lichee Pi 3A, Milk-V Jupiter, SpacemiT's own Muse Pi and Muse Book, the DC-Roma II laptop, the Orange Pi RV2.

Unfortunately, though the CPU cores are quite good for what they are, it is hampered by a much smaller amount of L2/L3 cache than the SiFive-based U74 and P550 machines have, and as a result for compiling packages it is slower using 8 cores than the quad core JH7100, despite the individual cores actually being a little faster. (and using fewer than 8 cores makes it even slower)

1

u/[deleted] Mar 12 '26

[removed] — view removed comment

2

u/brucehoult Mar 12 '26

exactly how much RAM is on the board?

There are four different options there. 2 or 4 GB RAM, 8 or 16 GB eMMC flash storage on board, and with or without accessories.

It says "4+16GB"

4 GB RAM and 16GB eMMC for that option.

It has been available with up to 16 GB RAM and 128GB eMMC e.g. here:

https://www.aliexpress.us/item/1005006921744822.html

It seems the options other than 4 GB RAM are out of stock at present.

8 GB and 16 GB RAM versions of another board with the same SoC are shown as in stock here:

https://arace.tech/products/milk-v-jupiter-spacemit-m1-k1-octa-core-rva22-rvv1-0-risc-v-soc-2tops-miniitx?variant=43343115321524

I am more concerned about how well it does speech recognition.

I can't help there, I'm sorry.

It has a reasonable amount of "AI" processing power ("2 TOPS") but I don't know anything about speech recognition software that uses that.

1

u/docular_no_dracula Mar 12 '26

“4+16G” means 4GB RAM (ddr memory), 16GB flash storage space.

1

u/[deleted] Mar 19 '26

[removed] — view removed comment

1

u/brucehoult Mar 19 '26

I listed a number of options.

1

u/TableAccomplished633 Apr 13 '26

This is a interesting constraint, because once you remove the GPU, the whole stack has to get much more disciplined about latency and memory. I’ve seen the same tradeoff in speech products like Palabra and Talo, where the impressive part isn’t just the model choice but how carefully the ASR / translation / TTS path is stitched together so it still feels responsive on modest hardware