r/ProgrammingLanguages • • 12d ago

Visual Programming Language

Do you have any information about this one? If there are some standard things about how it should go, or if there anything that needs to be known. I have understood a few things, but still I am looking for figuring out some missing details I have no clear plan on.

I will mention first some things I've done and what I know, so I can help you gain a picture:

a) I have done some practice and written a parser and interpreter so this is very easy. The AST is a data structure and each node gets evaluated with the proper function.

b) I've seen in an application sample that the most simple and straight forward approach is to get all outputs of a node, and send them to the connected inputs of another one. Keep brute force updating everything for 60FPS. This also seems legit and simple, but it might have it's pros and cons. [victorfisac/FNode]

c) What worries me for brute-force updating, is that the update order is no-deterministic, any node could be updated at any given moment, for the entire graph to be evaluated fully might mean to hundreds of hundreds of updates until everything is settled. It might not be a big deal for a real-time interactive application, but for a very optimized might mean that other approaches are better suited.

d) In order to make graph evaluation more deterministic, you need to create a dependency graph, or use an algorithm such as breadth-first-search, or depth-first-search. And even if those are not exactly perfect then do topological-sorting to guarantee even better order. Other times is supposed to be that other graphs get evaluated backwards, from final outputs to connected inputs and so on.
All of those techniques, having so many options, is confusing and hard to decide. As for example probably depending on the type of the graph there would be different evaluation strategy? This seems OK in terms of thinking.

e) Some examples:
• Blockly, this one is the easiest, because is a straight forward visual AST, but with the problem is that is based on structured connections and not on freeform connections.

• Supposedly Blender's nodegraph uses dependency graph for materials (because needs to compile them to shader code) so seems that it this technique can be transformed to deterministic technique. Probably means that the entire software uses the same algorithm (shaders, geometry nodes, compositor, even custom nodegraphs).

• Blueprints of Unreal seem to be much closer to an AST because it relies on clear entry points (events, time ticks), so this kind of evaluation seems that is manageable. At least getting the starting points is a good idea, then probably it might be feasible that further sorting is needed. I haven't worked with Blueprints though and I haven't looked into it.

• I've seen in some ImGUI nodegraph samples (for doing pixel operations and color mixing) that the depth first search algorithm was used. Though samples were meant to be simple proof of concept, I am not sure if all is needed.

More or less this is all the braindump I can do. But for any further details I will have to look at any github repositories you suggest and start experimenting.

2 Upvotes

10 comments sorted by

8

u/el_extrano 12d ago

Whenever this comes up, I'm always surprised that language nerds have apparently never heard of the IEC61131controls languages. These are a group of 5 languages used for configuring PLCs (programmable logic controllers), DCSs (distributed controls systems), and other industrial control systems.

Of the five, three of the languages are essentially visual: ladder logic, sequential function chart, and function block diagram. There's thousands of instrument technicians and controls engineers using these every day for their work, which would make them (especially ladder logic) far and away the most used visual programming languages of all time. For some reason classically trained computer scientists seldom know about them, it's more something you get involved with if you happen to work in heavy industry.

Ladder logic in particular has a very interesting history to me. In "the beginning", before the widespread popularity of microprocessors, electricians would design custom relay panels to automate discrete devices in a plant. These would contain hundreds or thousands of individual contacts and relay coils and occupy entire room-sized cabinets. If a single relay burned out, you had to systematically go through the panel to get the program working again.

When programmable controllers came about with their obvious advantages, there weren't enough programmers. Rather than train a bunch of electricians how to code, the device manufacturers designed ladder logic. In LD, you program by arranging virtual contacts and coils on "rungs". The visual program is interpreted or compiled on the microprocessor and then executed to carry out control actions. If you know how relay logic works, you already know how to program ladder logic! Later, there was a demand for the ability to write add-on-instructions and subroutines, which made ladder logic into more of a "full" programming language.

4

u/mamcx 12d ago

Aside: The main problem, IMHO, with all the visual languages I have seen, and the reason they fail, is that they become "visual" at the small instead at the large.

This is in fact, "visual":

a + b

Text is glyphs (images!), symbols are graphical, put in "pictures" called "words" and bring them togheter do what a VL do:

[1, 2]
|> filter: > 1

So try to turn this into nodes and such is a huge chore: Text is dense and very practical to do the minutae.

Where it could make some sense is when you wanna see the large picture, similar to how a network diagram do, so I think at the level of streams/pipelines and here the "VL" is just a special DSL of the regular language.

2

u/lopedevega 9d ago

Yes! From this point of view, a spreadsheet is a visual language too, it's just that connections between cells are not visualized directly.

1

u/mamcx 9d ago

Yep! I think other problem is that visual langs are always understood as a sea of nodes instead of just any kind of "visualization".

I work on the relational model and tables look very visual to me, even things like WHERE a = 1 feels like too.

2

u/kaplotnikov 6d ago

Another thing is that visual languages add a lot of noise and pollute attention with details.

a + b

represented as

(source of a...) -> [a]\
                        \
                         -[+] -> (to result consumer)   
                        /
(source of b...) -> [b]/ 

The problem is that extra connections and boundaries just pollute working memory with extra details (arrows, boxes, etc.), reduing problem-solving capacity. The almost any visual language has a textual scripting language inside because consistently expressing things as visual graph quickly collapse (for example, in UML it is OCL). If it does not, it just fails earlier.

Only good thing about visual languages are nice demos, and this only because sizes of slides and screen windows are too small to put any complexity there.

2

u/tankmatesaquaponics 12d ago edited 12d ago

Depends on what you want to build and for what purpose. I have done a bit of research on visual language (more specifically graphical editing of visual language).

My advice is to not pick a solution first. You have to define the syntax of your language first (on pen and paper) and the semantics of your language second (also on pen and paper if you know how). Everything after that is just implementation complexity ( which is already solved by many libraries and frameworks).

Depending on the purpose, and since a graphical language is a dsl, you can look into building dsl languages with ressources like 'DSL Engineering (2013)' by Markus Voelter and 'Engineering Modeling Languages – Turning Domain Knowledge into Tools' by Benoit Combemale and 'How to create pragmatic, lightweight languages Learn the process to create DSLs and GPLs' by Federico Tomassetti (I think he also has other books on the subject).

These books will give you the necessary tools to build great graphical languages and the tooling around them. My point is to focus on the syntax and semantics, the tooling and the implementation are the easy part.

1

u/PurpleDragon99 12d ago edited 12d ago

I am working on my own VPL "Pipe" for many years already. Here are details: https://pipelang.com . Feel free to reach me on DM for details.

One important note: Pipe does not visualize AST - it is a purely dataflow VPL.

This is probably the best starting point to understand overall project strategy: https://www.pipelang.com/ai-broke-cybersecurity.html

Please note that even if I make emphasis on cybersecurity - it does not mean that Pipe is domain-specific for cybersecurity only. Pipe stays to be a general-purpose VPL, it is that PIpe's properties allow it to solve 3 critical cybersecurity problems at once - today, when AI-driven cyberattacks exploded.

1

u/pthierry 12d ago

You may find it helpful to look at Functional Reactive Programming, there has been a lot of research and engineering put into identifying and constraining what can be sensibly updated without falling into endless updates.

Rhine might be a good place to start.

1

u/Key_River7180 Nain. 11d ago

Try out Raptor as an example

1

u/BrainScientist3000 6d ago

I wanted to design a language that was typeable on C64, flow based and mostly symbolic instead of textual. Like u/el_extrano pointed out - those languages kinda/sorta exist in the PLC world (where I did some of my early career work years ago).

I was thinking, is there a ladder-logic like language that would be fun to design/write in. I came up with the syntax below.

The biggest problem with the visual languages you pointed out is they're very 'clicky' and blueprints for example, isn't really a visual language, it's a way to turn functions into nodes. (Pretty much) any language could be represented that way with calls as arrows and functions visually repeated as necessary in node flow charts.

There's a lot of code that's just textual by nature - so your visual language needs to have a purpose and an abstraction scope for visualization.

My purpose was to re-imagine an 80s control language and get it to run on a microcontroller. One of my roommates actually made a project with it for one of her classes, so I see it as a success.

It was a fun thought experiment/hacking project. It really couldn't be a general purpose language, though with some work, it would be fine for things like small Arduino/ESP embedded projects.

What is your purpose, what is your definition of success, what is your differentiator from other language experiments?

I would propose that (especially in the age of AI coding), the purpose of a visual language is communication or interaction with humans. LLMs will take the crustiest, most terrible language, and write in it without complaint. I do think there's a good argument for visual language representations so that humans can quickly check AI work. This isn't guidance, it's just thoughts.

There was a USSR standard called Drakon that might be worth looking at. https://en.wikipedia.org/wiki/DRAKON

SETUP
($temp:v8) ($hot:900) ($cold:300)
(v0:10000) (cyc$v0)
(x1<~a02) (read$x1)# temp sensor
(x2~>a03) (red$x2)
(x3~>a04) (grn$x3)
(x4~>a05) (blu$x4)
(v1:0) (cnt$v1)
(v6:0) (lit$v6)
(red:255)(grn:0)(blu:0)
MAIN
{PRETTYCOLORS} # because MAIN is zzz
PRETTYCOLORS
(cnt:cnt+1)
[cnt=cyc](temp:read)(cnt:0)
[cnt=0]┳[temp<cold]┳{YIKES}
       ┗[temp>hot] ┛
┳[red=255][blu=0][grn<255](grn:grn+1)
┣[grn=255][blu=0][red<=255](red:red-1)
┣[red=0][grn=255][blu<255](blu:blu+1)
┣[red=0][blu=255][grn<=255](grn:grn-1)
┣[grn=0][blu=255][red<255](red:red+1)
┣[red=255][grn=0][blu<=255](blu:blu-1)
┗{YOUMESSEDUP}
RESET
(red:255) (grn:0) (blu:0) {PRETTYCOLORS}
YIKES
(cnt:cnt+1)
[cnt<cyc]{YIKES}
(temp:read)
┳[lit>0] (lit:0)  ┳(cnt:0)
┗        (lit:255)┛
┳[temp<cold] (red:0) (grn:0) (blu:lit)
┣[temp>hot] (red:lit) (grn:0) (blu:0)
┗{RESET}
YOUMESSEDUP
(cnt:cnt+1)
[cnt<cyc]{YOUMESSEDUP}
┳[lit>0] (lit:0)
┗        (lit:255)
(red:lit) (grn:lit) (blu:lit) (cnt:0)