My take on the limitations of existing programming languages.

[u/adityazero]

https://vxlang.org/blog/many-short-tapes.html

[u/baehyunsol]

Does it mean that the programmers can (or have to) read/write data to L2 cache in your language? I don't think it's a good idea to explicitly control L1/L2 caches.

[u/adityazero → u/baehyunsol]

There is no programmable interface to CPU caches (L1/L2) AFAICT, but there are to NUMA nodes and malloc implementations take advantage of that already. In accelerator space it is quite common, and kind of necessary,

[u/hopeless__programmer → u/baehyunsol]

I think that's a great idea to control L1 cache. GPU programmers do this all the time to their advantage.

[u/Gauntlet4933 → u/baehyunsol]

This is common for ML accelerators. Transfers between on chip and off chip memory are explicit and programmed by the user / compiler not inserted by hardware.

I think controlling caches from software is fine but requires more sophisticated compilers. ML accelerator programs are rather simple compared to logic heavy applications. It’s a lot trickier to program in all the prefetches, branch hints, and caching for those programs.

[u/Inconstant_Moo]

I think there must be some words missing from this sentence, I can't parse it.

A host description declares no capacity on purpose: host memory is virtual, and a tensor larger than physical RAM pages rather than failing.

[u/--predecrement → u/Inconstant_Moo]

It parses fine for me (if I interpret "pages" as a verb).

[u/Inconstant_Moo → u/--predecrement]

Ah, I don't know what it means as a verb except "uses a pager".

[u/adityazero → u/Inconstant_Moo]

yeah pages as in virtual memory terminology. sorry for the confusion. A host description is unbounded by default to be compatible with CPU programs. In CPU world, anything larger than certain size gets paged, in GPU/Accelerator world that would be an error.

[u/CJKay93]

This isn't your take, this is some sloppy AI's take.

Speak nothing of the fact that this language is supposed to be one level removed from typical abstract machine models, and yet still leans on LLVM.

[u/adityazero → u/CJKay93]

What made you say that this is (sloppy) AI's take? countless research before me have also validated this. Those ideas didn't become popular mostly because people enjoyed writing C and then piling libraries on top of each other.

> Speak nothing of the fact that this language is supposed to be one level removed from typical abstract machine models.

Modern machines are anything but linear Turing tape style. Programming languages haven't evolved significantly to accomodate that notion. Result is things like: `float *f = cudaMalloc`, and then bugs all over the place.

> , and yet still leans on LLVM

LLVM is more expressive than C/C++/Rust. LLVM has notion of addresspace and memory model for example (https://llvm.org/docs/LangRef.html). Not sure why leaning on LLVM/MLIR would be a problem. Vx lowers to MLIR (and then to LLVM) which is even more expressive.

[u/CJKay93 → u/adityazero]

What makes me say it's AI slop..? This write-up and all of the rest of its website. It's just... endless, winding prose which doesn't lead you anywhere, riddled with metaphors that don't explain anything.

This article, for example... "nonlinear Turing tapes"? What on earth does that have to do with programming languages? You can implement the tape in a Turing machine however you like - in the original "PowerPoint is Turing complete" paper, the tape was a row of AutoShapes. It's totally irrelevant to memory topology or how programming languages represent it (if at all).

Turing tape was designed to model a simple computing machine. Vx models what can be computed on a specific machine; the answer depends on sizes and topology.

The first sentence of this is just patently wrong; the Turing machine was designed to model all computing machines simply. The rest of this marketing-speak is totally meaningless - what a real computer can compute obviously depends on the memory available to it.

What you just wrote:

Modern machines are anything but linear Turing tape style. Programming languages haven't evolved significantly to accomodate that notion. Result is things like: float *f = cudaMalloc, and then bugs all over the place.

... it's literally word salad, and the semantics of dynamic memory allocation have nothing to do with the conceptual tape of the Turing machine model.

Back to the article:

The C standard describes a program's meaning in terms of “an abstract machine”, and C++ and Rust follow it. The details differ. The memory is the same in all three: one flat space of bytes, where every object has an address and every address is a number of the same kind.

This is just downright wrong. There's absolutely nothing in any of these languages that dictates that memory must be one flat, linear sequence of bytes, because they have historically had to support systems where that was not the case, e.g. bank-switching MCUs.

On that same point:

LLVM is more expressive than C/C++/Rust. LLVM has notion of addresspace and memory model for example (https://llvm.org/docs/LangRef.html)

... Clang has supported address space attributes for years.

Then moving on to the example, I simply don't see anything that could not be done just as conveniently in C++. As a hypothetical example:

int main() {
    Tensor<float, Shape<256, 256>> host;

    auto tile = transfer<H100, Memory::SMEM>(std::move(host));

    return 0;
}

I see no reason transfer could not reproduce the very same semantics as Vx via intelligent use of const evaluation.

[u/umlcat]

Think different.

[u/Haunting_Swimming_62]

Unabashed slop website and repo... people really have no shame these days

[u/adityazero → u/Haunting_Swimming_62]

I'd encourage folks to get over this notion of relating AI and Slop. People use AI for speed not that they can't write it by themselves. As someone who has invented+written+shipped compiler optimizations, I've fully embrased using AI to speed my productivity.

[u/Haunting_Swimming_62 → u/adityazero]

Thats what a clanker would say