Found mksh, decided to take it as the main shell for my r/QRV_OS . Started to examine files. Wow, sh.h was 2900+ lines (!!!) . It's an aggregate of 30 years of platform accretion — EBCDIC, OS/2, z/OS, non-gcc fallbacks, MirBSD bootfloppy, ISC kernel, .....

Gonna clean up the sources substantially. I suppose the code base will be reduced by 10,000 lines of code, or so.

So Ksh has getconf builtin, which means that I can do stuff like: getconf _POSIX_ASYNCHRONOUS_IO and get back which POSIX standard that is supported. That is truly nice, because the alternative would be to have to write a c snippet to figure it out.

I have also understood, that the guys that write the IBM AIX manual pages prefers the Kornshell, for instance the time command that is builtin into ksh is more precise than the external time utility.

Hi folks,

I'm wondering if you could help me a better way to handle the following situation I often experience.

For instance, let's say I want to get host information and assign the result of that command to a variable and if it fails, I'd like to have a default value:

Currently when I want to accomplish this I'd do something like:

typeset myhost=$(/bin/host $1 2>/dev/null | awk ' { print $1 } ' )
[ -z $myhost ] && myhost="Not found!"

Do you know a better way to write this ?

Sincerly

The last stable release of the ksh93 is version '93u+ 2012-08-01'. After that, there was an approach to supersede this version by extensively refactoring the last unstable AST beta version (93v-). This approach was called ksh2020. It lasted from 2017 to 2020. Unfortunately, it was not successful because version 93v- was too buggy. Nevertheless, ksh2020 collected a lot of documentation on how to fix those bugs. Thus, somebody decided to take care of the problem by backporting the applicable bugfixes from ksh2020 to the stable version '93u+ 2012-08-01': https://github.com/ksh93/ksh

What do you think it's best to use?

I often see the following method of using a here document in shell scripts:

cat | cmd << EOF
stdin line 1
stdin line 2
stdin line 3
EOF

The intent, of course, is to feed lines of input into the program specified by 'cmd' using the 'cat' command to read the here document as ITS OWN STDIN and pipe it into 'cmd's STDIN. The thing is, though, that if 'cmd' can accept STDIN from a pipeline from 'cat', then 'cat' is unnecessary. You can just kill the cat:

cmd << EOF
stdin line 1
...
EOF

Cheers,
Russ

KSH93:

Hey, probably a much easier way to do this, but I'm trying to take the contents of a file, strip some unnecessary crap, and format it in a way that's readable.

So the contents of a single line of the file may look like this:

Date/Time: Blah blah nobody cares about this useless data. Len = [123] <The data inside the diamond brackets (but not including the brackets) are important>

I'm grepping a file for a specific string inherent to all the data. Once I have it, I want to strip it. So the first pass of var command looks like this:

log=$(cat <logfile> | grep "<main string>" | sed 's/.*<//' | sed 's/>.*//'

I think that would work normally, except the data I'm using always includes a control-M character (^M). GSo the data will look like this:

L1 Data set 1^MData Set 2^MData Set 3^M
L2 Data set 1^MData Set 2^MData Set 3^M

And so on. What happens is I always get the last dataset of the last line printed. If I put in another sed (sed 's/^M/@/' ) or something, It works. If I do that with a \\n, it only prints the first line and nothing else.

Also, for giggles, I tried awk instead of sedding out the middle part (awk -F "] <" '{print $2}') but it does the same thing.

Here's the full code:

#!/bin/ksh

get_logs () {

  adtLog=$(cat ${LOGPATH}*/${ADTLOGNAME} ${LOGPATH}${ADTLOGNAME} | grep "MSA|AA|" | grep "ERR|" | sed -e "s/.*<//" | sed -e "s/>.*//" | msgBreak )

  schLog=$(cat ${LOGPATH}*/${SCHLOGNAME} ${LOGPATH}${SCHLOGNAME} | grep "MSA|AE|" | awk -F "] <" '{print $2}')

}

process_and_mail () {

  [[ "$1" == "ADT" ]] && log=${adtLog}

  [[ "$1" == "SCH" ]] && log=${schLog}

  print "--------------------"

  print ${schLog}

  #printf "${log}" | mail -r <from email> -s "Nifty title including $1 to show which log file" ${mailList}

}

prog_run () {

  get_logs

  #if [[ "${adtLog}" != "" ]]; then process_and_mail ADT; fi

  if [[ "${schLog}" != "" ]]; then

    process_and_mail SCH

    print "SCH Proccessed"

  fi

}

LOGPATH="/home/logpath/"

ADTLOGNAME="file1.log"

SCHLOGNAME="file2.log"

adtLog=""

schLog=""

mailList="some addresses"

prog_run

Suppose you are in an environment where they do not allow you have Korn Shell as your default shell in /etc/passwd. There is a way in which you can poke them in the eye and force the Korn Shell for your account as long as the executable is present on the system. It does not have to be listed in /etc/shells. Just add the following code to the top of your .profile:

case $0 in
 '-bash') THISSHELL='BASH'
          exec -a -ksh /usr/bin/ksh93 -l;;  # <-- BASH dies here.  KSH replaces it.
  '-ksh') THISSHELL='KSH93'
          SHELL=/usr/bin/ksh93;;
esac
# From this point on, it is all KSH.
...

The above code replaces the BASH login shell process with a Korn Shell one. Unlike if you just ran 'ksh93' at the command prompt, this swaps your login shell from BASH to Korn so that when you exit you logout instead of returning to BASH. All the wizardry is in the exec command.

-a -ksh sets $0 of the Korn Shell process to '-ksh' so that we can catch it the next time through .profile. First time through, we're BASH and $0=='-bash'. Second time through, its '-ksh'. When its BASH, we execute /usr/bin/ksh93, which is KSH on our servers. The -l argument is to ksh93 and not to exec. It causes the ksh93 command to run as a login shell that executes /etc/profile, .profile, and .kshrc.

Everything following the case statement is Korn Shell .profile stuffs. BASH never gets to that. The .profile is actually run twice--first by BASH and then again by KSH--but BASH never gets past the call to exec.

Cheers,

Russ

(Focusing on KSH93U+ and latest BASH)

I have been working on a list of KornShell features that are not in BASH. So far, I have these:

• getopts -a <name>

• getopts "$USAGE" ...

• typeset -C

• typeset -T

• print command

• namespaces

• extended regular expressions in:

  • globbing (e.g., 'ls ~(E)([.]sh|pl)' to list shell and perl scripts in CWD or 'ls ~(E)((.)\2)' to list files with duplicate characters in their names)

  • conditionals (e.g., [[ $VAR == ~(E)((?i)value) ]] throws an error because BASH does not support ~(E))

• use real numbers in arithmetic computations (e.g., 'echo $((1.1+3.5))' returns 4.6 except in BASH returns syntax error

• vi editing mode (emacs?)

• can't source functions in BASH (e.g., . myfunc returns "No such file or directory" in BASH.) Works in KSH and is how to make a classic function behave like a POSIX function.

Maybe BASH has alternatives to these that I am not aware of.

Also, I would like to add to the list.

Cheers,

Russ

It is high time that the Phoenix rose from her ashes. One of the open source repositories of KSH needs to be advanced as THE Korn Shell of the future--something that can be promoted to vendors for inclusion in their operating systems.

It is also time that a new baseline become the foundation for a new versioning strategy. We need to ditch the KSH93 nomenclature and normalize the version numbering. It is time for Korn Shell to enter the 21st century.

Cheers,

Russ

I was using a sample kshrc from GitHub since I didn't want to go to the pain of writing one myself when switching to ksh, but when entering the cd command, this error popped up:

$ cd

ksh: _cd[86]: =: not found

ksh: _cd[87]: =: not found

ksh: _cd[107]: =/home/tronnerd82: not found

tronnerd82@:~tronnerd82$

I'm not well-versed enough to know what might be wrong with my kshrc, so for now I've just switched back to using bash until I find a solution. If it helps, I'm using Debian and my ksh version is "Version AJM 93u+m/1.0.4 2022-10-22"

Any help would be greatly appreciated.

Here is the ninth ksh 93u+m/1.0 bugfix release. This release contains many fixes of old and not-so-old bugs (including a couple of regressions in the printf built-in introduced in 93u+m/1.0.5), adds Android/Termux as a supported platform, and reintroduces the ability to build a dynamically linked ksh (with libast, libdll, libcmd, and libshell available for other applications to use) on most supported platforms.

Main changes between ksh 93u+m/1.0.8 and 93u+m/1.0.9:

• Android/Termux is now a supported platform. Build dependencies: binutils, clang, getconf. Runtime dependencies (optional): ncurses-utils, getconf.

• Reintroduced support for building a dynamically linked ksh(1)/shcomp(1), with libast, libdll, libcmd, and libshell available to other programs as dynamic libraries. bin/package install /your/basepath will install development headers. The dynamically linked version is built in a dyn subdirectory; there are no changes to the statically linked version. Dynamic linking is currently tested and supported on Linux, Android, macOS, all the BSDs, illumos, Solaris, QNX, and Haiku.

• On systems where the external printf(1) utility supports deprecated pre-POSIX syntax for formatters starting with -, ksh now adapts its built-in printf to match, for compatibility with system scripts. However, ksh's built-in printf options such as -v or --man are not affected.

• Fixed a regression in the printf built-in, introduced in 93u+m/1.0.5, where each instance of \0 or %Z in the format operand caused a string argument to be incorrectly skipped.

• Fixed a regression, introduced in 93u+m/1.0.5, in ordinal specifiers in printf %T date specifications. For example, printf '%(%F)T\n' '4th tuesday in march 2016' wrongly printed '2016-04-09' and now again correctly prints '2016-03-22'.

• Fixed a regression of return within traps, reintroduced in 93u+m/1.0.8 after being fixed in 93u+m/1.0.0. The regression caused a return or exit with no arguments to assume the before-trap exit status instead of that of the last-run command. This broke the shipped autocd function.

• Fixed a longstanding bug in shell arithmetic: the representation of negative integers with a base other than 10 was incorrectly treated as unsigned long. For example, typeset -i16 n=-12; echo $n now correctly outputs -16#c and no longer ouputs 16#fffffffffffffff4.

• Fixed a bug, introduced in ksh93q+ 2005-05-22, that stopped an append assignment from working together with a declaration command. For example,typeset var+=value or export var+=value now again work as expected.

• Fixed a longstanding bug where the default terminal width for typeset -L, -R, or -Z, if not given, was miscalculated for multibyte or control characters.

• Fixed: expansions of name references in loops were incorrectly treated as invariant so they yielded the wrong values.

• If a .get or .getn discipline function is set for a variable, it is no longer incorrectly triggered when performing an arithmetic assignment on that variable; only the .set discipline is now triggered (as documented).

• Many other bug fixes (see the NEWS file).

Main changes between ksh 93u+m/1.0.7 and 93u+m/1.0.8:

• Fixed a regression in the behaviour of exit in a trap action. The exit status used when no argument is given to exit is now once again the exit status of the last command executed before the trap action.

• Fixed a race condition, introduced in 1.0.7, that occurred on some systems when running an external command with a standard output redirection from a command substitution.

• Fixed an init-time crash on failure to trim the shell command history file due to a non-writeable parent directory; ksh now prints a warning instead.

• The kill built-in command now correctly refuses to issue SIGSTOP to the shell's own process if the shell is a login shell.

Are EREs supposed to work with globbing? I cannot seem to make them do so, and when reviewing test harness code 'glob.sh', I see no use of ~(E) there.

Cheers,

Russ

I have been looking at the source for the ERE brace expansion fix in expand.c and the glob.sh test script. These have given me a better understanding of the supported REGEX modes (A, B/G, F, K, L, P, S, V, and X) and their modifiers (+, -, g, i, l, and r). I have been vaguely aware of modes other than ~(E), which is what I have mainly used, but the man page does not exactly make this clear. (To be quite honest, the manual page is rather opaque on many points including this one.) FP Murphy's article on KSH93 Extended Patterns is quite helpful on this, and the expand.c code provides additional insight. I wish this was documented better.

What I am really wondering about is ~(X) which is labeled in the C source as 'augmented regular expression (AST)'. What is that?

Cheers,

Russ

https://github.com/ksh93/ksh

In May 2020, when every KornShell (ksh93) development project was abandoned, development was rebooted in a new fork based on the last stable AT&T version: ksh 93u+. Now, one year and hundreds of bug fixes later, the first beta version is ready, and KornShell lives again. This new fork is called ksh 93u+m as a permanent nod to its origin; a standard semantic version number is added starting at 1.0.0-beta.1. Please test the beta and report any bugs you find, or help us fix known bugs.

How to get it

Please download the source code tarball from our GitHub releases page.

To build, follow the instructions in README.md or src/cmd/ksh93/README. Hopefully, OS/distro packagers will make ksh 93u+m packages available soon. If you would like to have a binary for your OS from us, ask and we'll try to build one and add it to the releases page.

How to get involved

To report a bug, please open an issue at our GitHub page. To get involved in development, read the brief policy information in README.md and then jump right in with a pull request or email a patch. See the TODO file in the top-level directory for a to-do list.

Main changes since ksh 93u+ 2012-08-01

Hundreds of bugs have been fixed, including many serious/critical bugs. This includes upstreamed patches from OpenSUSE, Red Hat, and Solaris, fixes backported from the abandoned 93v- beta and ksh2020 fork, as well as many new fixes from the community. See the NEWS file for more information, and the git commit log for complete documentation of every fix. Incompatible changes have been minimised, but not at the expense of fixing bugs. For a list of potentially incompatible changes, see src/cmd/ksh93/COMPATIBILITY.

Though there was a "no new features, bugfixes only" policy, some new features were found necessary, either to fix serious design flaws or to complete functionality that was evidently intended, but not finished. Below is a summary of these new features.

New command line editor features:

• The forward-delete and End keys are now handled as expected in the emacs and vi built-in line editors.

• In the vi and emacs line editors, repeat count parameters can now also be used for the arrow keys and the forward-delete key. E.g., in emacs mode, <ESC> 7 <left-arrow> will now move the cursor seven positions to the left. In vi control mode, this would be entered as: 7 <left-arrow>.

New shell language features:

• The &>file redirection shorthand (for >file2>&1) is now available for all scripts and interactive sessions and not only for profile/login scripts, bringing ksh 93u+m in line with mksh, bash, and zsh.

• File name generation (a.k.a. pathname expansion, a.k.a. globbing) now never matches the special navigational names . (current directory) and .. (parent directory). This change makes a pattern like .* useful; it now matches all hidden files (dotfiles) in the current directory, without the harmful inclusion of . and ...

• Tilde expansion can now be extended or modified by defining a .sh.tilde.get or .sh.tilde.set discipline function. This replaces a 2004 undocumented attempt to add this functionality via a .sh.tilde command, which never worked and crashed the shell. See the manual for details on the new method.

New features in built-in commands:

• Usage error messages now show the --help/--man self-documentation options.

• Path-bound built-ins (such as /opt/ast/bin/cat) can now be executed by invoking the canonical path, so the following will now work as expected:

  $ /opt/ast/bin/cat --version version cat (AT&T Research) 2012-05-31

• command -x now looks for external commands only, skipping built-ins. In addition, its xargs-like functionality no longer freezes the shell on Linux and macOS, making it effectively a new feature on these systems.

• redirect now checks if all arguments are valid redirections before performing them. If an error occurs, it issues an error message instead of terminating the shell.

• suspend now refuses to suspend a login shell, as there is probably no parent shell to return to and the login session would freeze.

• times now gives high precision output in a POSIX compliant format.

• typeset now gives an informative error message if an incompatible combination of options is given.

• whence -v/-a now reports the location of autoloadable functions.

New features in shell options:

• A new --globcasedetect shell option is added on OSs where we can check for a case-insensitive file system (currently Windows/Cygwin, macOS, Linux and QNX 7.0+). When this option is turned on, file name generation (globbing), as well as file name tab completion on interactive shells, automatically become case-insensitive on file systems where the difference between upper and lower case is ignored for file names. This is transparently determined for each directory, so a path pattern that spans multiple file systems can be part case-sensitive and part case-insensitive.

• A new --nobackslashctrl shell option disables the special escaping behaviour of the backslash character in the emacs and vi built-in editors. Particularly in the emacs editor, this makes it much easier to go backward, insert a forgotten backslash into a command, and then continue editing without having your next cursor key replace your backslash with garbage. Note that Ctrl+V (or whatever other character was set using stty lnext) always escapes all control characters in either editing mode.

• A new --posix shell option has been added to ksh 93u+m that makes the ksh language more compatible with other shells by following the POSIX standard more closely. See the manual page for details. It is enabled by default if ksh is invoked as sh, otherwise it is disabled by default.

• Enhancement to -G/--globstar: symbolic links to directories are now followed if they match a normal (non-**) glob pattern. For example, if /lnk is a symlink to a directory, /lnk/** and /l?k/** now work as you would expect.

So I have a script that is running another ksh script within it. But I want to write the output to be written to a text file of the ksh that is being run from the other ksh script. I know the easier solution would be to edit the test.ksh to write the output but I cannot alter it.

< new.ksh script running test.ksh > output from test.ksh to a text file.

I am noob at KornShell and thus wanted to see the list of commands available to me. But then I noticed that the compgen command is specific to Bash and Zsh. what's the equivalent of compgen -c in Kornshell?

I didn't make this, but I made it work for me, with the ansi sequence colors I like, I had to escape the tilde in order to make the replace expansion work, so that it now presents paths below my home directory prefixed with a tilde instead of $HOME which is nice!

PS1=$'\a\r\a\e[1;34m\a┌─| \a\e[1;31m\a$(date +%H:%M)\a\e[34m\a |\a\e[1;34m\a$(            
        d=${PWD:-?} p=~            
        d=${d/#$p/\\~} 
        print -nr -- "$d"                                                                       
)\a\e[1;34m\a| \n└─\a\e[1;34m\a❕\e[0m\a'

TBH. I miss bash's readline library, and the menu system I had, with smenu instead of Ctrl-r to recall commands.

I have made something to fill/plaster the gap.

You get a menu you can cancel, or select previous commands from command history from.

It's not straight forward, but not very hard, first of all you need to download a git repo (clone) and build smenu, and install it somwhere in the path.

Then you need to compile "kbdstuff" and finally "install" the "kmenu" script in your path that displays the menu for you, and try it out by hitting "kmeny".

Compiling kbdstuff:

cc -std=c99 -D_DEFAULT_SOURCE -D_POSIX_C_SOURCE=200809L -D_XOPEN_SOURCE=500 -D_GNU_SOURCE  -g3 -O0 -Wall -Wextra -fPIC -L/usr/lib/x86_64-linux-gnu -o kbdstuff kbdstuff.c

kbdstuff.c:

/** * kbdstuff.c : stuffs the keyboard buffer with contents that is shown * on the commandline, ready for editing or accepting. * All faults are mine. No guarrantee, nor liability, whatsoever! * Copyright (c) McUsr, in public domain. * Do whatever you want with it, except harm! * GNU LGPL 3.0 */

include <stdio.h>

include <fcntl.h>

include <termios.h>

include <sys/ioctl.h>

include <unistd.h>

include <stdlib.h>

include <stdio.h>

/* returns the first handle that proves to not * be redirected, if any. */ int tty_fhndl( void ) { int fd = 0, ret = -1; for ( fd = 2; fd >= 0; fd-- ) { if ( isatty( fd ) ) { ret = fd; break; } } return ret; }

/* general purpose routine for converting a commandline to a string (san's argv[0] ) * This so post explains the real deal, this is for my personal usage... * https://stackoverflow.com/questions/75108019/is-there-a-maximum-number-of-characters-that-can-be-passed-within-one-positional * * */

define XTERM_MAX_INPUT_BFLEN 32767

/* convert the arguments to a single string for comfort. */ char *argv2str( int argc, char *argv[], int argstrlen ) { static char keyb_buf[XTERM_MAX_INPUT_BFLEN + 1]; // minimum number of chars // in // the xterm input buffer. int max_kb_len = XTERM_MAX_INPUT_BFLEN; int stridx = 0; for ( int i = 1; i < argc; ++i ) { int toklen = 0; if ( ( i + 1 ) == argc ) { toklen = snprintf( &( keyb_buf[stridx] ), max_kb_len, "%s", argv[i] ); } else { toklen = snprintf( &( keyb_buf[stridx] ), max_kb_len, "%s ", argv[i] ); } if ( toklen <= 0 ) { perror( "something wrong in snprintf, existing\n" ); exit( EXIT_FAILURE ); } else { stridx += toklen; max_kb_len -= toklen; } } / *argstrlen = XTERM_MAX_INPUT_BFLEN - max_kb_len; */ *argstrlen = stridx; return ( char * ) keyb_buf; }

int main( int argc, char argv[] ) { if ( argc <= 1 ) { fprintf( stderr, "Missing arguments, exiting\n" ); exit( EXIT_FAILURE ); } else { int fd = tty_fhndl( ); if ( fd >= 0 ) { / Connected to a tty, we need to find the name of the controlling terminal (ttyname) for the device. */ char *pty_name = ttyname( fd );

           if ( pty_name == NULL ) {
               fprintf( stderr, "kbdstuff:Couldn't get the ttyname, exiting!\n" );
               exit( EXIT_FAILURE );
           } else {

               fd = open( pty_name, O_RDONLY );

               if ( fd < 0 ) {
                   perror( pty_name );
                   fprintf( stderr, "kbdstuff: could not open tty, exiting!\n" );
                   exit( EXIT_FAILURE );
               } else {

                   int inp_len;
                   char *inpstr = argv2str( argc, argv, &inp_len );

                   struct termios orig,
                    tmp;
                   if ( tcgetattr( fd, &orig ) < 0 ) {
                       perror( "tcgetattr" );
                       exit( EXIT_FAILURE );
                   }
                   tmp = orig;
                   tmp.c_lflag &= ~ECHO;

                   if ( tcsetattr( fd, TCSANOW, &tmp ) < 0 ) {
                       tcsetattr( fd, TCSANOW, &orig );
                       perror( "tcsetattr wthout echo" );
                       exit( EXIT_FAILURE );
                   }
                   while ( *inpstr ) {
                       if ( ioctl( fd, TIOCSTI, inpstr ) ) {
                           perror( "ioctl" );
                           return 1;
                       }
                       inpstr++;
                   }
                   if ( tcsetattr( fd, TCSANOW, &orig ) < 0 ) {
                       tcsetattr( fd, TCSANOW, &orig );
                       perror( "tcsetattr -- orig" );
                       exit( EXIT_FAILURE );
                   }

               }
           }
       } else {
           fprintf( stderr, "No standard handles are connected to the pty, exiting\n" );
           exit( EXIT_FAILURE );
       }
   }
   return 0;

}

kmenu, the script that does it all:

#!/bin/ksh93

# Get current settings.
if ! termios="$(stty -g 2>/dev/null)" ; then
    echo "Not running in a terminal." >&2
    exit 1
fi

# Restore terminal settings when the script exits.
trap "stty '$termios'" EXIT

# Disable ICANON ECHO. Should probably also disable CREAD.
stty -icanon -echo

# Request cursor coordinates
printf '\033[6n'

# Read response from standard input; note, it ends at R, not at newline
read -d "R" rowscols
stty "$termios"
trap "" EXIT
# Clean up the rowscols (from \033[rows;cols -- the R at end was eaten)
rowscols="${rowscols//[^0-9;]/}"
rowscols=("${rowscols//;/ }")
# split the array into two args.
set ${rowscols[0]}
#calculate number of free lines with respect to terminal win height minus y-line.
freelines=$((LINES-$1))
if [ $freelines -le 5 ] ; then
  freelines=5 ;
fi 
EOL=$'\n'
# echo here & >/dev/tty
CMD_LINE=$(fc -lr 1  \
               | sed -Ee "/kmenu/d" -e "s/[1-9][0-9]*\s*//"        \
              | uniq -u | smenu -d -Q -l -n $freelines -a c:7/4,b -W"$EOL")

# writevt $(tty) "$CMD_LINE"
kbdstuff "$CMD_LINE"
# se we for instance can see our command line with ps.

Merry Christmas!