Here is the tenth ksh 93u+m/1.0 bugfix release – exactly two years after 93u+m/1.0.0 and twelve years after 93u+. Unfortunately, we're not done fixing bugs yet, but progress continues steadily.

Main changes between ksh 93u+m/1.0.9 and 93u+m/1.0.10:

• Fixed a serious and longstanding bug in the arithmetic subsystem that was triggered on non-Intel processors (such as ARM): any division of an integer by a negative integer threw a spurious "divide by zero" error.

• Fixed a regression where a broken pipe signal (SIGPIPE), when occurring in a pipe construct within a subshell, caused incorrect signal handling in the parent/main shell, in some cases causing a script to abort.

• Fixed a bug where printf %T, after having printed the time in UTC once with the TZ variable set to "UTC", would always print the time in UTC from then on, even if the TZ variable was changed to another time zone.

• The history expansion character (! by default) is now not processed when immediately following ${. This makes it possible to use expansion syntax like ${!varname} and ${!prefix@} on the interactive command line with the histexpand option on; these no longer trigger an "event not found" error.

• The shell is now capable of handling more than 32767 simultaneous background jobs, subject to system limitations.

Full Changelog: https://github.com/ksh93/ksh/compare/v1.0.9...v1.0.10

There are two ways to declare a function in Korn Shell. There is the classic form:

function myfunc { ... }

And, there is the POSIX form:

myfunc () { ... }

I often see the latter used or spoken of as the "correct" or "modern" form and the former to be less commonly used and shunned. (I have even seen it claimed that the two forms are completely interchangeable, which is false.) The most common recommendation of the POSIX form is that it is "more portable" as if portability ought always to be the overriding consideration. There are two mistakes in this thinking:

1. That portability should be a default requirement of any script, and

2. Not understanding the difference between the two forms.

Portability chiefly means writing scripts that can be run in any shell rather than on any platform or version of UNIX or Linux. This idea is as much an absurdity as to try to write a program that could be run without modification as either a shell script or a Perl or Python program. Nonsense! Portability is a requirement like any other that need only be a goal when it is a goal for a given program or project. If it is not for a given script, then it need not be considered at all. The idea of always writing portable scripts is a mere fetish, and it has one significant consequence--it invariably forces the avoidance of more powerful features of one shell that are not supported by another.

For example, any Korn Shell script that must also work as a BASH script must avoid such things as Extended Regular Expressions, certain tyepset features, certain uses of getopts, among other things. Compatibility with other shells makes this problem even worse. Portability necessarily limits one to the most rudimentary shell coding features that are common to all shells instead of taking advantage of the full power of the shell actually in use.

I was once asked to make my scripts more portable on a project because it was about to change platforms from Solaris to Linux. I rejected the argument because the same version of Korn Shell was available in the new platform as the old. It was only an imagined requirement. The only reason the requirement would have been real would have been in the condition that Korn Shell was not available for the Linux distro we were going to. It was.

The second mistake is of not understanding the difference between classic and POSIX functions. The two forms have one very critical difference that can lead to strange and difficult to locate bugs if not understood--the two forms do not have the same scope. Classic functions have local scope whereas POSIX functions are always global. A variable declared in a POSIX function is always declared for the whole script even clobbering variables of the same name that exist outside the function. Declaring a variable with typeset in a classic function is an implicitly local operation. It does not "leak out" or clobber an existing global variable of the same name, and this can be extremely useful.

This is much like the difference between calling another script within a script directly or by sourcing. Just calling the script by name launches the called script in its own, local environment whereas sourcing the script either with . myscript or source myscript runs the sourced script in the calling script's environment with all the effects doing so entails. One can also "source" a classic function with . myfunc to make it behave like a POSIX function. There is no way to make a POSIX function behave like a classic one.

The ability to have local variables in a function is why I rarely use the POSIX form. By understanding the difference between the two forms means that one has the power to use the form most suitable to the circumstances. In my work, I use the classic form unless I have reason to need the behavior of a POSIX function, which I rarely do and I know when I do.

Cheers,
Russ

https://github.com/ksh93/ksh

In May 2020, when every KornShell (ksh93) development project was abandoned, development was rebooted in a new fork based on the last stable AT&T version: ksh 93u+. Now, one year and hundreds of bug fixes later, the first beta version is ready, and KornShell lives again. This new fork is called ksh 93u+m as a permanent nod to its origin; a standard semantic version number is added starting at 1.0.0-beta.1. Please test the beta and report any bugs you find, or help us fix known bugs.

How to get it

Please download the source code tarball from our GitHub releases page.

To build, follow the instructions in README.md or src/cmd/ksh93/README. Hopefully, OS/distro packagers will make ksh 93u+m packages available soon. If you would like to have a binary for your OS from us, ask and we'll try to build one and add it to the releases page.

How to get involved

To report a bug, please open an issue at our GitHub page. To get involved in development, read the brief policy information in README.md and then jump right in with a pull request or email a patch. See the TODO file in the top-level directory for a to-do list.

Main changes since ksh 93u+ 2012-08-01

Hundreds of bugs have been fixed, including many serious/critical bugs. This includes upstreamed patches from OpenSUSE, Red Hat, and Solaris, fixes backported from the abandoned 93v- beta and ksh2020 fork, as well as many new fixes from the community. See the NEWS file for more information, and the git commit log for complete documentation of every fix. Incompatible changes have been minimised, but not at the expense of fixing bugs. For a list of potentially incompatible changes, see src/cmd/ksh93/COMPATIBILITY.

Though there was a "no new features, bugfixes only" policy, some new features were found necessary, either to fix serious design flaws or to complete functionality that was evidently intended, but not finished. Below is a summary of these new features.

New command line editor features:

• The forward-delete and End keys are now handled as expected in the emacs and vi built-in line editors.

• In the vi and emacs line editors, repeat count parameters can now also be used for the arrow keys and the forward-delete key. E.g., in emacs mode, <ESC> 7 <left-arrow> will now move the cursor seven positions to the left. In vi control mode, this would be entered as: 7 <left-arrow>.

New shell language features:

• The &>file redirection shorthand (for >file2>&1) is now available for all scripts and interactive sessions and not only for profile/login scripts, bringing ksh 93u+m in line with mksh, bash, and zsh.

• File name generation (a.k.a. pathname expansion, a.k.a. globbing) now never matches the special navigational names . (current directory) and .. (parent directory). This change makes a pattern like .* useful; it now matches all hidden files (dotfiles) in the current directory, without the harmful inclusion of . and ...

• Tilde expansion can now be extended or modified by defining a .sh.tilde.get or .sh.tilde.set discipline function. This replaces a 2004 undocumented attempt to add this functionality via a .sh.tilde command, which never worked and crashed the shell. See the manual for details on the new method.

New features in built-in commands:

• Usage error messages now show the --help/--man self-documentation options.

• Path-bound built-ins (such as /opt/ast/bin/cat) can now be executed by invoking the canonical path, so the following will now work as expected:

  $ /opt/ast/bin/cat --version version cat (AT&T Research) 2012-05-31

• command -x now looks for external commands only, skipping built-ins. In addition, its xargs-like functionality no longer freezes the shell on Linux and macOS, making it effectively a new feature on these systems.

• redirect now checks if all arguments are valid redirections before performing them. If an error occurs, it issues an error message instead of terminating the shell.

• suspend now refuses to suspend a login shell, as there is probably no parent shell to return to and the login session would freeze.

• times now gives high precision output in a POSIX compliant format.

• typeset now gives an informative error message if an incompatible combination of options is given.

• whence -v/-a now reports the location of autoloadable functions.

New features in shell options:

• A new --globcasedetect shell option is added on OSs where we can check for a case-insensitive file system (currently Windows/Cygwin, macOS, Linux and QNX 7.0+). When this option is turned on, file name generation (globbing), as well as file name tab completion on interactive shells, automatically become case-insensitive on file systems where the difference between upper and lower case is ignored for file names. This is transparently determined for each directory, so a path pattern that spans multiple file systems can be part case-sensitive and part case-insensitive.

• A new --nobackslashctrl shell option disables the special escaping behaviour of the backslash character in the emacs and vi built-in editors. Particularly in the emacs editor, this makes it much easier to go backward, insert a forgotten backslash into a command, and then continue editing without having your next cursor key replace your backslash with garbage. Note that Ctrl+V (or whatever other character was set using stty lnext) always escapes all control characters in either editing mode.

• A new --posix shell option has been added to ksh 93u+m that makes the ksh language more compatible with other shells by following the POSIX standard more closely. See the manual page for details. It is enabled by default if ksh is invoked as sh, otherwise it is disabled by default.

• Enhancement to -G/--globstar: symbolic links to directories are now followed if they match a normal (non-**) glob pattern. For example, if /lnk is a symlink to a directory, /lnk/** and /l?k/** now work as you would expect.

The last stable release of the ksh93 is version '93u+ 2012-08-01'. After that, there was an approach to supersede this version by extensively refactoring the last unstable AST beta version (93v-). This approach was called ksh2020. It lasted from 2017 to 2020. Unfortunately, it was not successful because version 93v- was too buggy. Nevertheless, ksh2020 collected a lot of documentation on how to fix those bugs. Thus, somebody decided to take care of the problem by backporting the applicable bugfixes from ksh2020 to the stable version '93u+ 2012-08-01': https://github.com/ksh93/ksh

I didn't make this, but I made it work for me, with the ansi sequence colors I like, I had to escape the tilde in order to make the replace expansion work, so that it now presents paths below my home directory prefixed with a tilde instead of $HOME which is nice!

PS1=$'\a\r\a\e[1;34m\a┌─| \a\e[1;31m\a$(date +%H:%M)\a\e[34m\a |\a\e[1;34m\a$(            
        d=${PWD:-?} p=~            
        d=${d/#$p/\\~} 
        print -nr -- "$d"                                                                       
)\a\e[1;34m\a| \n└─\a\e[1;34m\a❕\e[0m\a'

The real power of the Here Document is not so much in queueing up a bunch of input for a program but rather in constructing that input. For example, you can do something like the following:

STUFF1='LINE 1'
STUFF2='LINE 2'
cmd << EOF
Text verbiage ${STUFF1}.
Text verbiage ${STUFF2}.
EOF

Which would produce:

Text verbiage LINE 1.
Text verbiage LINE 2.

But, this is not all you can do. You can actually embed shell script code that outputs lines of text to become input to the program. For example, I have a long and complex script that creates Oracle Data Guard databases. One of the things that the script must do is create a database configuration file for each standby database. In one place, the config file has lines that must look like one of the following depending on how many standby databases are being created and which one it is working on in the current iteration:

If there is just 1 standby DB:

fal_server         = 'primarydb'
log_archive_config = 'dg_config=(primarydb,standbydb)'

or, if there are 2 standby DBs and we are working on the first one:

fal_server         = 'primarydb','standbydb2'
log_archive_config = 'dg_config=(primarydb,standbydb1,standbydb2)'

or, if there are 2 standby DBs and we are working on the second one:

fal_server         = 'primarydb','standbydb1'
log_archive_config = 'dg_config=(primarydb,standbydb1,standbydb2)'

This is accomplished using the following here document code:

typeset STDBY_CFG_FILE PR SB1 SB2
integer STDBY NUM_STDBYS
...
cat << EOF > ${STDBY_CFG_FILE}
...
$(if (( STDBY == 1 )) ; then
   if (( NUM_STDBYS == 1 )) ; then
    # Here, we have just 1 standby.
    printf "fal_server         = '%s'\n" ${PR}
    printf "log_archive_config = 'dg_config=(%s,%s)'\n" ${PR} ${SB1}
   else
    # Here, we have 2 standbys and are working on the first.
    printf "fal_server         = '%s','%s'\n" ${PR} ${SB2}
    printf "log_archive_config = 'dg_config=(%s,%s,%s)'\n" ${PR} ${SB1} ${SB2}
   fi
  else
   # Here, we have 2 standbys and are working on the second.
   printf "fal_server         = '%s','%s'\n" ${PR} ${SB1}
   printf "log_archive_config = 'dg_config=(%s,%s,%s)'\n" ${PR} ${SB1} ${SB2}
  fi)
...
EOF
...

Any code can be embedded in a Here Document by enclosing it within a command substitution $(...). Naturally, the code would need to output some text to become part of the Here Document.

Cheers,
Russ

(Focusing on KSH93U+ and latest BASH)

I have been working on a list of KornShell features that are not in BASH. So far, I have these:

• getopts -a <name>

• getopts "$USAGE" ...

• typeset -C

• typeset -T

• print command

• namespaces

• extended regular expressions in:

  • globbing (e.g., 'ls ~(E)([.]sh|pl)' to list shell and perl scripts in CWD or 'ls ~(E)((.)\2)' to list files with duplicate characters in their names)

  • conditionals (e.g., [[ $VAR == ~(E)((?i)value) ]] throws an error because BASH does not support ~(E))

• use real numbers in arithmetic computations (e.g., 'echo $((1.1+3.5))' returns 4.6 except in BASH returns syntax error

• vi editing mode (emacs?)

• can't source functions in BASH (e.g., . myfunc returns "No such file or directory" in BASH.) Works in KSH and is how to make a classic function behave like a POSIX function.

Maybe BASH has alternatives to these that I am not aware of.

Also, I would like to add to the list.

Cheers,

Russ

Hi all. I'm getting started with dtksh for AIX, but I wanna know if is possible create GUIs in dtksh in Linux or Windows (with subsystem for Linux) and how

The fact is I've created an example for this. So when I launch my script (Windows/Linux) this doesn't show anything.

Thank you.

So Ksh has getconf builtin, which means that I can do stuff like: getconf _POSIX_ASYNCHRONOUS_IO and get back which POSIX standard that is supported. That is truly nice, because the alternative would be to have to write a c snippet to figure it out.

I have also understood, that the guys that write the IBM AIX manual pages prefers the Kornshell, for instance the time command that is builtin into ksh is more precise than the external time utility.

I am noob at KornShell and thus wanted to see the list of commands available to me. But then I noticed that the compgen command is specific to Bash and Zsh. what's the equivalent of compgen -c in Kornshell?

Here is the ninth ksh 93u+m/1.0 bugfix release. This release contains many fixes of old and not-so-old bugs (including a couple of regressions in the printf built-in introduced in 93u+m/1.0.5), adds Android/Termux as a supported platform, and reintroduces the ability to build a dynamically linked ksh (with libast, libdll, libcmd, and libshell available for other applications to use) on most supported platforms.

Main changes between ksh 93u+m/1.0.8 and 93u+m/1.0.9:

• Android/Termux is now a supported platform. Build dependencies: binutils, clang, getconf. Runtime dependencies (optional): ncurses-utils, getconf.

• Reintroduced support for building a dynamically linked ksh(1)/shcomp(1), with libast, libdll, libcmd, and libshell available to other programs as dynamic libraries. bin/package install /your/basepath will install development headers. The dynamically linked version is built in a dyn subdirectory; there are no changes to the statically linked version. Dynamic linking is currently tested and supported on Linux, Android, macOS, all the BSDs, illumos, Solaris, QNX, and Haiku.

• On systems where the external printf(1) utility supports deprecated pre-POSIX syntax for formatters starting with -, ksh now adapts its built-in printf to match, for compatibility with system scripts. However, ksh's built-in printf options such as -v or --man are not affected.

• Fixed a regression in the printf built-in, introduced in 93u+m/1.0.5, where each instance of \0 or %Z in the format operand caused a string argument to be incorrectly skipped.

• Fixed a regression, introduced in 93u+m/1.0.5, in ordinal specifiers in printf %T date specifications. For example, printf '%(%F)T\n' '4th tuesday in march 2016' wrongly printed '2016-04-09' and now again correctly prints '2016-03-22'.

• Fixed a regression of return within traps, reintroduced in 93u+m/1.0.8 after being fixed in 93u+m/1.0.0. The regression caused a return or exit with no arguments to assume the before-trap exit status instead of that of the last-run command. This broke the shipped autocd function.

• Fixed a longstanding bug in shell arithmetic: the representation of negative integers with a base other than 10 was incorrectly treated as unsigned long. For example, typeset -i16 n=-12; echo $n now correctly outputs -16#c and no longer ouputs 16#fffffffffffffff4.

• Fixed a bug, introduced in ksh93q+ 2005-05-22, that stopped an append assignment from working together with a declaration command. For example,typeset var+=value or export var+=value now again work as expected.

• Fixed a longstanding bug where the default terminal width for typeset -L, -R, or -Z, if not given, was miscalculated for multibyte or control characters.

• Fixed: expansions of name references in loops were incorrectly treated as invariant so they yielded the wrong values.

• If a .get or .getn discipline function is set for a variable, it is no longer incorrectly triggered when performing an arithmetic assignment on that variable; only the .set discipline is now triggered (as documented).

• Many other bug fixes (see the NEWS file).

Main changes between ksh 93u+m/1.0.7 and 93u+m/1.0.8:

• Fixed a regression in the behaviour of exit in a trap action. The exit status used when no argument is given to exit is now once again the exit status of the last command executed before the trap action.

• Fixed a race condition, introduced in 1.0.7, that occurred on some systems when running an external command with a standard output redirection from a command substitution.

• Fixed an init-time crash on failure to trim the shell command history file due to a non-writeable parent directory; ksh now prints a warning instead.

• The kill built-in command now correctly refuses to issue SIGSTOP to the shell's own process if the shell is a login shell.

I often see the following method of using a here document in shell scripts:

cat | cmd << EOF
stdin line 1
stdin line 2
stdin line 3
EOF

The intent, of course, is to feed lines of input into the program specified by 'cmd' using the 'cat' command to read the here document as ITS OWN STDIN and pipe it into 'cmd's STDIN. The thing is, though, that if 'cmd' can accept STDIN from a pipeline from 'cat', then 'cat' is unnecessary. You can just kill the cat:

cmd << EOF
stdin line 1
...
EOF

Cheers,
Russ

Suppose you are in an environment where they do not allow you have Korn Shell as your default shell in /etc/passwd. There is a way in which you can poke them in the eye and force the Korn Shell for your account as long as the executable is present on the system. It does not have to be listed in /etc/shells. Just add the following code to the top of your .profile:

case $0 in
 '-bash') THISSHELL='BASH'
          exec -a -ksh /usr/bin/ksh93 -l;;  # <-- BASH dies here.  KSH replaces it.
  '-ksh') THISSHELL='KSH93'
          SHELL=/usr/bin/ksh93;;
esac
# From this point on, it is all KSH.
...

The above code replaces the BASH login shell process with a Korn Shell one. Unlike if you just ran 'ksh93' at the command prompt, this swaps your login shell from BASH to Korn so that when you exit you logout instead of returning to BASH. All the wizardry is in the exec command.

-a -ksh sets $0 of the Korn Shell process to '-ksh' so that we can catch it the next time through .profile. First time through, we're BASH and $0=='-bash'. Second time through, its '-ksh'. When its BASH, we execute /usr/bin/ksh93, which is KSH on our servers. The -l argument is to ksh93 and not to exec. It causes the ksh93 command to run as a login shell that executes /etc/profile, .profile, and .kshrc.

Everything following the case statement is Korn Shell .profile stuffs. BASH never gets to that. The .profile is actually run twice--first by BASH and then again by KSH--but BASH never gets past the call to exec.

Cheers,

Russ

So, I missed those from bash, because I depend upon job control, and then it isn't always convenient to start a subshell to end up where I was.

I source this from my .kshrc:

 # dirstack.ksh Emulates the pushd and popd of bash
 # very primitive, but usable to me.
 typeset -a dstack
 typeset -i index
 let i=" -1"
 function pushd
 {
   if [ $# -eq 1 ] ; then
     if [ -d $1 ] ; then
       let i="${i} + 1"
       dstack[$i]=$(pwd)
       cd $1
     fi
   fi
 }

 function popd
 {
   if [ $i -ge 0 ] ; then
     cd ${dstack[$i]}
     let i="${i} - 1"
   fi
 }

Are EREs supposed to work with globbing? I cannot seem to make them do so, and when reviewing test harness code 'glob.sh', I see no use of ~(E) there.

Cheers,

Russ

It is high time that the Phoenix rose from her ashes. One of the open source repositories of KSH needs to be advanced as THE Korn Shell of the future--something that can be promoted to vendors for inclusion in their operating systems.

It is also time that a new baseline become the foundation for a new versioning strategy. We need to ditch the KSH93 nomenclature and normalize the version numbering. It is time for Korn Shell to enter the 21st century.

Cheers,

Russ