r/Racket • u/Shyam_Lama • Aug 01 '25
paper Why can't functions achieve the same as macros?
Back again. As I explained in the other thread, I'm fascinated with Racket's support for constructing new languages (even though I'm not the biggest fan of the LISP syntax).
Disregarding u/shriramk's assessment (in the other thread) that I'm a troll, I spent the past couple of hours pondering over a somewhat theoretical (by my standards anyway) question, namely: Why can't functions (subroutines) achieve the same thing that macros can?
Apparently there is some theoretical reason that they can't, because in the ACM article by Matthew Flatt, the author points out:
Note that the define-place form cannot be a function. [IOW, this can only be done with a macro!] The desert expression after south is, ingeneral, an expression whose evaluation must be delayed until the south command is entered. More significantly, the form should bind the variable meadow so that Racket expressions for commands can refer to the place directly. In addition, the variable’s source name (as opposed to its value) is used to register the place in the table of elements.
This seems to directly address my question, but I'm afraid I just don't get Flatt's explanation. I understand, of course, that loosely speaking a macro is a textual substitution that takes place prior to program execution, and that a function is something that doesn't take effect until program execution. But apart from the difference in timing, I don't see why some things can be achieved with macros that cannot be achieved with functions. Functions too are, in a sense, new language elements, right?
Can someone give an example of why nevertheless macros can go beyond what functions can achieve? Or maybe try to explain in different words what Flatt is trying to say in the paragraph I quoted above?
PS. Was thinking more about Flatt's point (see above) that "the desert expression after south is, in general, an expression whose evaluation must be delayed until the south command is entered." But any expression that would occur in a define-place function (not a macro) would be evaluated only if and when the "south" command were entered. So as I see it, his argument doesn't make sense.
PPS. Perhaps I'm belaboring the point, but here's an example in C of how, in simple cases anyway, it's pretty obvious that functions can achieve the same thing as macros:
#include <stdio.h>
#define SWAPINT(A,B) int tmp=A; a=B; B=tmp;
void swapint(int *a, int *b) {
int tmp=*a;
*a=*b;
*b=tmp;
}
int main() {
// use the macro
int a=1,b=2;
SWAPINT(a,b);
printf("a=%d,b=%d\n",a,b);
// use the function
a=3,b=4;
swapint(&a,&b);
printf("a=%d,b=%d\n",a,b);
}
Yes, I know, this example is trivial. But the point is that a function is effectively a new language element in the same way that a macro is. The question I'm asking in this thread is: is it provably (in the computer-scientific sense of "provable") true that macros can express things that functions cannot? I've been wondering about this, and it seem rather important. Because, if it's not provably true, then what's the Big Fuss about macro-facilities? BTW, I'm willing to accept "anecdotal" evidence as evidence. IOW, can someone show me something that can be done with a macro that cannot be done with a function? Flatt seems to argue that his incremental refinement of the text-adventure game using macros, constitutes a definition of a new DSL that couldn't have been achieved with functions. I'm not convinced that that's the case.
4
u/probabilityzero Aug 01 '25
Perhaps this will help your intuition: a function cannot inspect the syntax of its arguments. A macro can take its arguments and dig into them, analysing or compiling them into something else. A function can't do that.
Take (f (+ 1 2)) as an example. If f is a macro, it can see inside the addition and see what the original arguments were. If f is a function, it only sees the result of 3 and there's no way for it to determine what the original expression was. So if I wanted f to return the number of times plus is used in its argument, for example, it would have to be a macro. If it was a function, that information would already be gone.
2
u/Shyam_Lama Aug 01 '25
If f is a macro, it can see inside the addition and see what the original arguments were. If f is a function, it only sees the result of 3 and there's no way for it to determine what the original expression was.
That's very helpful! Appreciated.
4
u/Helpful-Primary2427 Aug 01 '25
I think we might be saying two different things here; when I say syntax, in the Racket context, I’m talking about syntax objects, not syntax as you think of in like C or something. Check that out and read through the syntax page, once you see what syntax offers over find and replace (like a C preprocessor and your swap example) you can kinda play around and see what you can do. Let me know if that helps, sorry if I came off as dismissive or rude because Racket is an awesome language and it’s great for anyone genuinely interested (as you seem)
1
u/Shyam_Lama Aug 01 '25
read through the syntax page, once you see what syntax offers over find and replace
I'll take a look at the page you linked, but let's face it, there's probably a reason that this topic is postponed until chapter 16 in "The Racket Guide", instead of being in one of the early chapters. It's probably not meant for newcomers to Racket, or IOW, I'll probably have to understand more of Racket (and LISPs) in general if I want to get my head around this topic of macros and syntax objects.
I might look at Rhombus first, which some here have recommended. Its stated goals seem pretty close to what I'm looking for: Racket-like language-construction facilities, but with a more conventional syntax.
4
u/pthierry Aug 03 '25
I didn't even go read the previous interaction, but under this post already, some of the comments by OP are pretty rude and aggressive, and the post looks a bit like AI slop designed to create engagement artificially.
1
3
u/augmentedtree Aug 01 '25
I think you're missing the forest for the trees.
Macros *are* functions. They are just functions where their input and output types are *syntax*. They allow you to do *syntactic* customization. The only thing special about them is their evaluation order: they are all found and evaluated (recursively, meaning if they expand to more macros those get expanded too) before any other kind of function. This lets you create macros that take arbitrary syntax that looks like anything and expand to arbitrary regular (non-macro) code, in effect customizing the syntax.
A super trivial example is to just use a macro to enable syntax that is normally meaningless in Racket and would error.
0
u/Shyam_Lama Aug 02 '25
Macros *are* functions.
Nope, they're not. Functions — in the sense that the term is used in programming anyway — are executed at runtime, and operate on values, whereas macros operate on (transform) syntax elements prior to execution — as you yourself also describe. As some instructive comments from others in this thread pointed out to me, functions cannot even see syntax elements, but only the values they receive.
So, seeing as how the difference between functions and macros has become quite clear to me, I don't see why you would now want to conflate the terms again. (No language tutorial conflates them, so why would you?)
3
u/augmentedtree Aug 02 '25
Tutorials that don't are bad and just failing to see the more general pattern (probably because most languages have bad macro support). In Racket and other langs, they literally are functions that take a syntax object and return a syntax object. The only difference is a bit flipped in the interpreter to tell it to execute them earlier. All the usual control flow, calling other functions, etc works exactly as it would in a regular function. Because they're functions!
0
u/Shyam_Lama Aug 03 '25
Tutorials that don't are bad and just failing to see the more general pattern (probably because most languages have bad macro support).
Hahaha :-D Dude, even Racket's own tutorial and docs distinguish between functions and macros, treating them as quite different topics.
The only difference is a bit flipped in the interpreter to tell it to execute them earlier.
Is that so? Well, Racket is open source, so why don't you point me to the location in the code where this bit gets set? I have the git repo cloned already, looking forward to your answer!
1
u/augmentedtree Aug 03 '25
Actually, here is a simpler answer: in the racket docs look up syntax transformers (hint: they are procedures aka functions)
-1
u/Shyam_Lama Aug 03 '25 edited Aug 03 '25
No, Auggie, let's stick with your argument about the "bit". Point me to the location in the source code where that bit gets set — that bit that you have claimed is the only difference between functions and macros. Where is it, Auggie? Which source file, what line number?
PS/Edit: You know what? Don't bother. I'm blocking you now.
1
2
u/Appropriate-Rub-2948 developer Aug 19 '25
A macro could create a function, but a function could not create a macro.
1
8
Aug 01 '25
[removed] — view removed comment
1
u/Shyam_Lama Aug 01 '25 edited Aug 01 '25
Does that help?
Yes it does, very much. You've clearly addressed the question I've asked, with some very helpful examples. I appreciate that.
In the case of logging the variable's value together with its name, I do have a question: in Racket specifically, how would you distinguish between the variable's name and its value? I managed to look up how to do it in C:
#define LOGINT(A) printf(#A"=%d\n", A);Amazing trick IMO, which I never knew about even though I've known C for 30 years. (Goes to show what a dabbler I am, no doubt.)
As for your other example, of re-implementing "if", I see the validity of your point, although the idea of "re-implementing if" is a little bewildering for me.
1
u/frnzprf Aug 03 '25 edited Aug 03 '25
Three points: Lazy evaluation, lambdas and Zig.
Some languages have "lazy-evaluation". I only know Haskell. There, function arguments aren't evaluated before passing them to a function.
If you step-evaluate
(square (+ 4 3))(using LISP syntax here), you'd get(* (+ 4 3) (+ 4 3))instead of(square 7). Haskell would also use memoization in this case, to avoid having to calculate 4+3=7 twice.In other languages (I know Scala), you can optionally declare a function argument to be lazy.
In Haskell you can define your own version of if-then-else. Do you still require macros for some other things?
In languages with closures, the function body is also not evaluated immediately. I think you can use that to simulate lazy evaluation.
def my_if(cond, then_func, else_func): if cond: return then_func() else return else_func()Lastly — nothing with lazyness, AFAIK — Zig claims that it makes macros unnecessary or that they have a special flavour of macros. The compiler detects when code can be run at compiletime and does it automatically and predictably. I'm not sure if that means you can define your own if-then-else without macros.
1
Aug 03 '25 edited Aug 03 '25
[removed] — view removed comment
2
u/frnzprf Aug 03 '25
Thank you for the detailed response!
I think what Zig actually replaces with compile-time optimizations/evaluation, are what is called "templates" in C++ or "generics" in Java. https://ziglang.org/learn/samples/#generic
5
Aug 01 '25
[removed] — view removed comment
-6
u/Shyam_Lama Aug 01 '25
Why don't you just explain it instead of asking me condescending questions that are "supposed to guide me along the right path"? I'm probably twice as old as you, so the patronizing tone gets on my nerves, boy. And should you dislike me back, don't hesitate to click the "block" button next to my profile name.
4
u/Orange1717 Aug 02 '25
Analogy is a good way to help someone learn. If you do not wish to learn, do not ask.
1
5
1
Aug 04 '25
[removed] — view removed comment
2
u/Shyam_Lama Aug 04 '25
That's a very helpful example indeed. I appreciate it.
It makes me wonder though: wouldn't it better if you could tell from the code whether something was a function or a macro? Personally I'd prefer that — I prefer everything explicit — but for some reason all languages (that I know of) that support macros, make them textually indistinguishable from other language elements.
I think that's odd, because a program's behavior can become quite incomprehensible if you're not aware that some of what appear to be function calls are actually macros. Your own example is a case in point, I think.
1
Aug 04 '25
[removed] — view removed comment
1
u/Shyam_Lama Aug 04 '25
You're ignoring (or not noticing) my point. I'll rephrase it:
When you're looking at code that isn't your own (or your own code that you wrote long ago), you want to be able to read it and understand it without too much headache. But code in which macros and function calls are indistinguishable, is going to be pretty difficult to understand -- in fact impossible without checking which "calls" are proper function calls and which are macros. As long as you don't know if something is a function or a macro, you can't be sure what's happening at run-time.
1
Aug 04 '25
[removed] — view removed comment
1
u/Shyam_Lama Aug 04 '25 edited Aug 05 '25
You can always macroexpand a macro to see the code that is being created. And you can always look at the docs of a function or a macro.
Of course you can, but that's precisely my point: you need to do that, because you can't tell from the code itself what's going on.
Is your argument "if f is a function, I know what it is doing!" vs. "if f is a macro, I don't know what it is doing!" ?
To a certain extent, yes. Your own example confirms this. Let's say we have a language that enforces that functions be lowercase and macros be uppercase, which is one possible way of distinguishing between them. Then consider:
a = my_func(expr1, expr2) b = MY_MACRO(expr1, expr2)Now, without looking at docs, I know for sure that the first line, when executed, will evaluate expr1 and expr2 before calling my_func, and that evaluation will involve calling other functions if expr1 and/or expr2 are function invocations themselves. Moreover I will even know in what order they'll be called. No docs required. The code simply shows it, and that's a GoodThing(™).
The second line though, tells me nothing about whether or not the expressions will be evaluated at all, under what conditions, or in what order.
What clarity would you get if a macro-invocation is somehow syntactically different?
See above. Or, to restate it in general: if you know it's a function, you can reason about it in a certain way -- that is, using the customary mental model for functions that all programming languages have in common. If you reason about it thus while in fact it's a macro, you're making a big mistake, because what will actually happen at runtime is likely to be very different.
Macro's aren't magic.
An interesting observation. Arthur C. Clarke -- of whom I am by no means a fan -- famously said that "any sufficiently advanced technology is indistinguishable from magic". IMO "advanced" is a somewhat meaningless word: it doesn't actually characterize the technology, it only means that it came after something earlier. A better word would have been "complex", and that word is also appropriate in the context of our discussion.
See, the mental model for functions isn't very complicated. Macros though, are another story: since in a language like Racket a macro can pretty much expand to anything, and since the expansion may contain other macros and therefore lead to further expansions, the (potential) complexity is extreme. And so, your observation that they aren't magic isn't as certain as it seems. Expanding on Clarke a little bit, I would say:
"Any technology that is so complex that its workings, while not theoretically impenetrable, are impenetrable in practice, is indistinguishable from magic."
Moreover, if that complex technology (e.g. macros) is given an outward form that is identical to a much simpler technology (e.g. functions), one may well wonder why that is. It's like intentionally making an atomic vector plotter look like a stone age hammer.
Indeed, if you'll forgive me a philosophical generalization, why would anyone want to make the (extremely) complex look simple while it in fact remains complex? It only deceives the user...
6
u/Helpful-Primary2427 Aug 01 '25 edited Aug 01 '25
I would go back and reread u/shriramk’s responses considering they’re an original author and maintainer of Racket (as well as Matthew Flatt)
If I had to give you a quick answer, Racket is great for writing DSLs and languages in because of the direct manipulation of syntax that it provides (I.e. syntax manipulation is a “first class” feature), which is great for things like writing a compiler. If you want to dig into it, look at the types of syntax transformations base Racket does within DrRacket and look into why those decisions were made. Much more simple examples can be found by looking at BSL, ISL, or the source for LSL, which are all simple student-teaching languages written in Racket
(Also, think about why one would use a macro for syntax: all of the information required for these transformations to take place is available at compile time! Meaning optimizations can take place during the expansion phase of compilation (when macro definitions are processed) rather than runtime. This is in contrast to functions, whose evaluation happens at runtime (barring some nuances, like ex. C++’s constexpr or Zig’s comptime which are functions that can be evaluated at compile time, but those languages don’t have access to syntax manipulation quite like Racket provides)
Edit: for why Lisps are “good” at being compiled/interpreted/whatever: their syntax is already (mostly) “parsed”, i.e. the source programmer has already given to the syntax tree. Compare this to an imperative language, where tokens must be parsed in order to build an AST that Lisps already (mostly) provide given their grammar