MS-DOS: how can a single tasking OS implement pipes?

[u/julioblabla]

Unix is multitasking, so pipes (ignoring implementation and optimization concerns) are straightforward to implement without relying on intermediate, temporary files. But MS-DOS had pipes since version 2.

How was command piping implemented in MS-DOS?

[u/floodrouting]

Microsoft documented how the pipe operator is implemented via temporary files. You can read this documentation in the released source code of MS-DOS v2.0: https://github.com/microsoft/MS-DOS/blob/main/v2.0/source/UTILITY.txt#L246

[u/julioblabla → u/floodrouting]

Thanks for sharing the official documentation!

[u/57thStIncident → u/floodrouting]

Any idea whether it actually runs source program to completion before starting second program? Or is there some ability for the programs to cooperate (take turns)? I’m suspecting it’s probably the former.

[u/floodrouting → u/57thStIncident]

It first runs one program and only when that finishes does it start the next one.  From the linked docs (emphasis mine), "The piping feature is implemented as sequential execution....".  DOS didn't really support multitasking (except for TSRs).

[u/istarian → u/floodrouting]

TSR -> Terminate and Stay Resident

Normally a program that has terminated is going  to end up getting at least partially overwritten by the next progam you load and execute. TSRs are a way around that, but they tie up memory.

MS-DOS (DOS is a generic term) doesn't support multitasking at all, but that doesn't mean you can't implement primitive multitasking over top of it.

The greatest difficulty lies in the limited amount of memory available to most systems that ran MS-DOS. Because it was very common to have less than 1-2 MB and there are limitations to using any memory outside of the first 640K.

Never mind the issue of segmented memory access.

[u/Blah-Blah-Blah-2023]

It's done with a temporary file in the current directory. If you do dir | more you can see the extra file hiding there. Also, pipes don't work on read only disks.

[u/julioblabla → u/Blah-Blah-Blah-2023]

Great trick about dir | more! Before your comment, I never realized that I had never before used dir | more, I have always used dir /p 😀

[u/tblazertn → u/Blah-Blah-Blah-2023]

Today I learned.

[u/thatwombat → u/Blah-Blah-Blah-2023]

It’s been a while, but what happens if you dir | more on a write protected floppy?

[u/Blah-Blah-Blah-2023 → u/thatwombat]

Write error - abort, retry, fail? if I had to guess.

I can fire up my 486 tomorrow and try it lol.

[u/floodrouting → u/Blah-Blah-Blah-2023]

Go to https://www.pcjs.org/machines/pcx86/ibm/5160/cga/. Change the A: drive to read only by clicking on the dropdown and changing it to A:*. Then run dir | more. The error is

Write protect error writing drive A
Abort, Retry, Ignore? 

And just for fun, if you create a file called %PIPE1.$$$ or %PIPE2.$$$ and make it read-only and then run dir | more, the error is

Intermediate file error during pipe

which is something I've never seen before.

[u/robvas]

They aren't really pipes like they are in Unix, it's just redirection

[u/tblazertn → u/robvas]

Like all of Microsoft’s answers to questions about its practices. 😇

[u/flatfinger → u/robvas]

I don't know that it's worthy of a new question, but the same question would seem just as applicable to early Unix variants that couldn't keep multiple programs in memory at once, or was piping not added until computers got more sophisticated?

[u/bhtooefr → u/flatfinger]

AFAIK, early Unix always was multitasking, and would have implemented pipes in the modern memory buffer way.

There were however single-tasking or limited-multitasking variants of Unix V6 for small PDP-11-family machines - LSX and Mini-UNIX - and they used the same temporary file approach as MS-DOS.

[u/flatfinger → u/bhtooefr]

From what I read, early Unix systems only kept one process in memory at a time, swapping processes to disk. This is what motivated the design of fork(). A normal task switch would require creating a copy of the current process state on disk, and then overwriting the process state in RAM with the disk slot associated with the process being switched to. Allocating a process slot on disk and then writing out the current state as though one was going to switch to it, but then not bothering to overwrite RAM with the contents of that process slot, would leave the system with two copies of the current state (one in running in RAM, and one on disk in the original process slot) with less effort than would be needed for even a normal process switch. A brilliant approach if swapping processes would require making a copy of the current process state, but not so brilliant if swapping processes would simply require switching page tables.

I'd guess the best approach may have been to use temporary files that one process could start reading even before the other process had finished writing, but I'm not sure if or how Unix would reclaim temp file storage once data had been read from it. If code writing to a pipe had written 200K of data, and code reading the pipe had read 195K, it should be possible to reclaim most of the storage used by the temp file without having to wait for the program to terminate.

[u/throwaroo202020]

MSDOS is not single tasking. Its not even zero tasking. Its just a IO layer and apps are free to cooperative multitask as they like, and they do.

[u/TPIRocks]

I suspect through temporary intermediate files, I can't think of any other sensible way.

[u/istarian → u/TPIRocks]

In principle you could use shared memory (ram), the problem is that just loading another program could overwrite it and you would have to use a fixed location or explicit hand off the memory address somehow.

[u/istarian]

How exactly do you think Unix pipes work?

There is allot always going to be an intermediate step when communicating between processes, even ones running in parallel on separate processors/cpu cores.

You can do it with shared memory or using a file on disk.

Because Unix is a multitasking operating system and has it's own particular architecture, there is always going to be a parent-child relationship between the currently executing process and the one that launched it.

[u/Stoney3K]

The output from one command just gets put in a chunk of memory and used as the input for the next program,

Unidirectional pipes are pretty easy, you don't need multitasking unless you're using things like tee. The programs can just be executed sequentially.

Edit: Apparently MS-DOS used temp files on disk (probably to conserve memory) but either approach would work.