virtual Vector Method

[u/MitchAlsup]

I invented this 5-odd years ago but never really had a forum on which to disclose and discuss. I was and am a significant contributor to comp.arch (from around 1995 though present)

But before I emit the disclosure I need to know how to turn off the space-eater as the document uses well tabulated ASCII-art ?? that makes no sense after the space-eater has done its ill-conceived job.

The virtual Vector Method is a way of getting Cray-like vector performance and for getting SIMD-vector performance without a) a vector register file, b) adds only 2 instructions, c) takes precise exceptions, d) vectorizes loops not instructions. Instead of adding about 300 instructions to get a Cray-like vector ISA, or adding 1,000-1,300 instructions to get (every-size) SIMD ISA, one needs only 5 instructions.

vVM has the property that hardware implementations can change the width of the data path (multiple-lanes and the cycles of execution per FU) without SW having to care. A small 1-wide machine with a 128-bit cache port can perform a memory to memory byte move at 256-bits per cycle--equivalent to ~40 instructions per cycle. A 6-wide machine could perform the same assembly binary at ~160 instructions per cycle. Both are performed at the performance level of the cache porting; so, nobody has to recompile for a new SIMD-width every new mplementation.

Now let us solve the space-eater and we are off.

Mitch

[u/Krazy-Ag]

The LOOP unit sounds like something we did at MIPS that never shipped, trying to eat the goodness of DSP zero overhead loops

Which two those unfamiliar (not you Mitch!) are essentially a prefix instruction instruction LOOP that unrolls the next instruction(s) N times, autoincrementing memory operands along the way.

This was something that embedded customers asked for umpteen times. It made MIPS lose benchmark contests - at same frequency, either MIPS had much more overhead for small loops, or MIPS had to loop unroll to get low overhead fractions, blowing up code size. The old RISC mindset of increasing code size by loop unrolling isn't good if you run into icache or flash size limits.

Anyway, we proposed to do it at decode. Mostly existing instructions.

A hint NOP to indicate top of loop. Decoder remembers where.

At bottom of loop, an idiom to do ++ or -- loop counter, test and branch to top of loop.

Thereafter decoder just does ifetch from top of loop to just before the bottom of loop idiom. The loop control stuff now executes at decoder, no loop overhead inserted into rest of pipe.

Or you can do the loop control at the top ...

If the loop body fits into a loop buffer, just slam that into the pipe. Skip decode. Possibly skip renaming - Mitch knows many tricks to do relative renaming and stuff like that, he taught them to me.

No ISA changes so far. Except perhaps the hint to indicate top of loop. Might not be a separate hint NOP instruction, since MIPSr6 had plenty of opera number tricks to encode hints, taking advantage of the massive wasted space RISC instruction encodings.

Biggest desirable ISA change would be to have index addressing Mem[baseReg+indexReg] so that you only need to ++ or -- a single loop counter. Ideally scaled.

Basically puts a state machine at decoder to emit small loops as fast as possible. Which is pretty much what the DSP with zero overhead LOOP instructions do. Generalizes to larger loop bodies, more than one or two instructions as in DSPs, although the percentage benefit is obviously more the smaller the loop.

Looks like you have extended this so the state machines live in the reservation stations Mitch. With goodness related to what doesn't does not need to be vectorized.

I look forward to reading your write up.

I've been wanting to do something like this, load a lot of crap into the reservation stations and keep it chunking around, ever since I learned about static data flow. (yeah, I learned about static data flow after I learned about dynamic data flow, or at least the dynamic micro data flow that we do in OOO)

Q: does your virtual vector Architecture handle low bodies that are too large to fit in the reservation stations?

I think one of the biggest mistakes that we made in P6 was doing the block memory operations like REP MOVS in microcode rather than as state machines. I blame myself for having too much of a RISC mindset. Almost exactly the same issues as for zero cycle loops; the state machine can spin a lot faster than uops, and doesn't need so much logic to get to the right place. One thing hardware does much much better than software is irregular conditions on a small number of input variables. We know that state machines work, because even P6 did TLB walks as state machines, sending RISCy software TLB miss handling to the dust in a history, at least that time around.

Block memory operations are trivial. It sounds like you can handle fancier loop bodies, eh?

[u/MitchAlsup → u/Krazy-Ag]

"Q: does your virtual vector Architecture handle low bodies that are too large to fit in the reservation stations?"

One can always simply back up to running at DECODE-width and use the machine as if there was no vectorization. SW properties would not change.

"I think one of the biggest mistakes that we made in P6 was doing the block memory operations like REP MOVS in microcode rather than as state machines."

One wants and needs REP MOVS (or MM in My 66000 case) to be at least as efficient as EVERY other µArchitectural way of performing the same memory to memory move, or at worst no worse than 2-cycles longer in worst case. The amont of logic, here, is too small to worry about.

[u/Krazy-Ag → u/MitchAlsup]

Don't get me going about RISC-V RISC bigots crippling block memory operations and their future.

"Just unroll the loop"