TFM:Getting to Know Your Shell

From ProgSoc Wiki

Jump to: navigation, search


Getting to Know Your Shell

Christopher "Aubergine" Fraser


When you're entering UNIX commands, you are in fact interacting with the shell. The shell is so named because it puts a layer between you and the gooey, pulsating splatter that is at the heart of UNIX. Taking the time to become accustomed to the ways of UNIX shells can save you from much tedium and frustration, and also make you incredibly smart and 50% more attractive.

This chapter attempts to demystify the interactive aspects of shell usage and gives an overview of the major features that are common to more recent UNIX shells. It assumes readers are accustomed to basic UNIX commands.

A Brief Description

A UNIX shell works in a very simple way:

  1. Display a prompt.
  2. Read a command line from the terminal.
  3. Execute commands as indicated by command line.
  4. Repeat.

When you log into a UNIX machine, the login program automatically starts up a shell for you, if X is not automatically loaded or unavailable on that machine (shells can, of course, be accessed from within an X session via a terminal window). There are a number of different shells around. The type of shell started up is referred to as your default shell. This is determined by a field in your password file entry. You can use the finger command to look up someone's default shell. The following command should report that I use bash as my default shell.

% finger cfraser
Login name: cfraser    			In real life: Chris 'Aubergine' Fraser
Directory: /home/cfraser              	Shell: /usr/bin/bash
On since Feb 17 14:04:55 on console 

A Forest of Shells

There are two major lineages of UNIX shells, the original Bourne shell sh(1) and its bastard love child the C shell csh(1). From these a number of bloated feature-ridden shells have sprogged[1]. This document will deal with four of the shells available on the ProgSoc machines and in the Linux labs at UTS:

  • csh. This is a standard shell provided by most flavours of UNIX . It has some useful interactive features, and unlike some other shells remains reasonably small and efficient. csh is better for interactive use than plain sh, but not by much. It has a great many bugs, limitations and inconsistencies. It should never be used for writing shell scripts. [2].
  • tcsh. Basically an improved version of csh, featuring additional Nice Things like interactive line editing, but unfortunately retaining csh's limitations and inconsistencies. Anything that applies to csh should apply to tcsh.
  • bash. Maintains sh compatible syntax but with lots of added Nice Things. It has line editing, aliases, shell functions, online help, features, features, features. Nothing has been spared. As a result, this is a large program, and it has quite a large drain on machine resources[3]. bash and tcsh are mostly comparable in terms of features (useful or otherwise). bash is now the default shell almost anywhere Linux is installed, ProgSoc machines included.
  • zsh. A shell that aims to be similar to bash but cooler

and funkier.

Most, if not all, of the features discussed are available in the other major UNIX shells.

Try Before You Buy

As mentioned above, a shell is simply an interactive UNIX program, so to play around with a shell, try running it. For example, to run bash, enter the command:[4]

% bash

Running the shell in this manner should be sufficient for trying out most of the examples presented here. Eventually, after playing around with a shell you may decide you want to use it all the time. See the section on Changing Your Shell for further information.

Shell Startup

All of the shells check to see if a startup file exists in the user's home directory. This file contains a list of commands that should be executed every time the shell starts up. Much of this document will talk about commands you can use to customise aspects of the shell. If you enter them at the prompt, they will only affect that shell session. If you put them in your startup file they will affect all new shell sessions. The name of the startup file varies from shell to shell:[5]

Shell Startup File
csh/tcsh .cshrc
sh .profile
bash .bashrc
zsh .zshrc

History Substitution

This probably the most used interactive shell feature. First implemented in csh, history substitution is also available in tcsh, bash and zsh with slight flavour variations. It is most useful for correcting typos in a previous command or reusing text from a previous command line, without having to use your mouse to cut-and-paste.

The basic idea is that every command line entered is stored as an entry in a history list. The number of entries in the history buffer is determined by the environment variable HISTSIZE for bash and history for csh and tcsh. To view the current history list, use the history command. After a few dozen commands this list will become somewhat excessive, so it's typical to pipe the output through a command like tail. Eg:

% history | tail -20

Or perhaps something more adventurous like:

% history | sed -e "s/^[ ]*/ /" -e "s/   / /" | /usr/bin/pr -t -w80 -2

Most people define their own command h to print out the last couple of history commands. See the section on Functions and Aliases further on.

After reading in your command line, the shell scans it for history phrases. A history phrase begins with a bang character (!) and usually extends to next whitespace character[6]. If the shell understands your phrase, it will replace the entire phrase with text from the command buffer. The following sections explain how to specify exactly what text you want from the history buffer.

The manual page for csh has a fairly succinct section on all of the bells and whistles associated with history substitution. bash and tcsh both offer obscure extensions.

Selecting History Entries

Following the bang is text to indicate exactly which history entry you want to reference. There are three basic forms:

  • History Number. The shell assigns sequential history numbers to all history entries. You can reference number n by !n. This method is most useful if you print out the current history number in the prompt. See the section on Prompts for how to do this.
  • Relative. The !-n construct refers to the n--th most recent history entry, relative to the current command line. !! is a more convenient form of !-1, meaning the previous command line.
  • Key Word. You can refer to a history entry via part of its text. The constructs !str and !?str? refers to the most recent entry beginning with or containing the text str respectively.

For example:

% echo !50            # print out history entry 50
% !?login             # execute the last entry referring to login
% !! > temp           # execute the last entry and redirect output
% echo "The last ls command was: !ls"

Filtering Selections

If you only want part of the command line, you can select specific words[7] by appending a colon and a word designator. The most commonly used designators are:

  • $ The last word of the entry.
  • * The whole history entry except for the first word (the command). You can omit the colon for this designator if you like. This is effectively the same as 1-$.
  • n The n--th word. 0 is the first word.
  • x--y The range of words from x to y.

Here are some quick examples. The command executed is printed under the history phrase.

% axe birthdays
% !!:0 
% vi !?bir:$ calendar
vi birthdays calendar
% emacs !!:*
emacs birthdays calendar

Modifying Selections

You can append additional colons and designators to modify the substituted text. The most useful of these designators is :s/x/y which substitutes occurrences of x with y. The following example selects the previous entry and substitutes cc with hh.

% !!:s/cc/hh/

A short form of this designator is available via the caret ^ character. Thus, an alternate formulation of the above is:

% ^cc^hh^

Note that the caret modifier only works on the previous history entry. Caret is especially useful for fixing typos like transposed characters. In bash at least there is also a :p modifier which, if it appears at the end of a line, will result in the modified line been printed but not executed. This is useful for checking to see if the modifications were performed as intended. If acceptable, the command line can then be executed with !!.


Interactive shells display a prompt to signify they are ready for the next line of input. You can set your prompt to be any text string that takes your fancy.[8] In sh and bash you simply assign to the PS1 variable:

$ PS1="I Honk For Unicorns $ "

And in csh and tcsh you use the prompt environment variable:

% set prompt = "I Honk For Unicorns % "


The text string will be evaluated before being set as the prompt. This means you can refer to the output of commands and other environment variables in the string. For example, to use the output of the hostname command and the value of the LOGNAME environment variable in your prompt you can use

$ PS1="$LOGNAME@`hostname`$ "

for bash and

% set prompt ="$LOGNAME@`hostname`% "

for csh and tcsh. Note that because the prompt string is only evaluated once, the above prompts are static (ie: they do not change). This can lead to unexpected results. For example:

% set prompt = "`pwd`% "

will display the current working directory you were in when you changed the prompt, rather than the directory you're currently in. If you change directory the prompt will not change.

More Usefulness

Prompts are good place to print useful information. Suggestions include the name of the machine you are logged into (the hostname), current working directory and the current history number. Note that the latter two can change as you execute commands. Now csh is pretty poor with regard to useful prompts. The only way you can have the prompt updated after changing the current working directory is to redefine the cd command. The only really useful prompt information csh easily provides is the current history number, which is displayed by a ! in the prompt string.

bash and tcsh support all manner of prompt fanciness. The following table summaries the special codes you can put into the prompt string and what they do;

tcsh bash Function
 %! \! History number
 %m \h Host name
 %n \u User name
 %/ \w Working Directory

tcsh and bash both have loads more special prompt codes, most of which are just plain silly. Consult the manual page if you're interested.

Less Usefulness

You can use different typefaces for your prompt. Most terminals can handle VT100 control codes. As a quick example, the following bash style prompt will be printed in bold:

$ PS1="\033[1m\w\033[0m\u@\h[\!] "

Control characters can confuse line editing. Normally you can get around this by splitting your prompt across two lines. A example of such a two-line prompt is:[9]

$ PS1="\033[1m\w\033[0m\n\u@\h[\!] " 

The "\033" inserts an escape character. "\e" can be used in tcsh. You will probably have to use an editor which allows you to insert control characters to get an escape character in the csh prompt.

Job Control

The usefulness of Job Control has mostly been superseded by graphical user interfaces. Job control evolved out of the desire to run multiple commands (or "jobs") from the one terminal. With the advent of windowing systems, you just start up another window. Still, you may not have that luxury if you're remotely logged-in somehere.[10]

Commands started by the shell normally run in the foreground. This means that the shell waits for them to finish before prompting for another command. You can put a command into the background by putting an ampersand (&) at the end of the command line.

Doing so will print out the job number the command was allocated (and the process ID) and return you to the prompt. Each job the shell starts has a job number associated with it. If you only ever run commands in the foreground the shell will reuse job number 1 for all of them. You can place a background job in the foreground using the fg command, which accepts an optional argument of the job number. If you omit the job number, it will foreground the most recently backgrounded job.

You can suspend the foreground job using Control-Z, then put it in background with the bg command. This is otherwise similar to the fg command.

You can refer to jobs via the percent (%) character. For example, to kill job number 2:

% kill %2

You can also get a list of the current jobs and their status, at least in bash, by using the jobs built-in:

$ jobs
[1]+  Stopped                 jstar .bashrc
[2]-  Running                 mpg123 mckc.mp3 &

Shells differ in what they do about backgrounded jobs when you log out of the shell. csh and tcsh will typically kill them unless you run them via the nohup(1) command. bash leaves backgrounded processes alone, but will warn about suspended jobs. Often, system administrators get antsy about background processes which stay running after their owner has logged out. Check first.

Word Expansion

You have already met several characters, such as ! and &, which are treated in a special way. This section deals with special characters typically used by shells. If you want to use these characters unmolested by the shell, precede them with a back-slash (\) or quotes, a'la:

% echo 'Hello World!'    # bang is quoted
% vi swallow\&hurl       # ampersand is escaped

Tilde Expansion

The tilde (~) character can be used to refer to the home directories of other users. It must be at the start of a word---the text following the tilde is interpreted as a username. The shell will find their home directory using the password file. If you follow the tilde with a slash - or nothing at all - it assumes you mean your own home directory.

% ls ~bigted    # list the files in Big Ted's home directory
% ls ~/bin      # list the files in your own bin directory

Pathname Expansion

Pathname expansion (or globbing as it is usually called) attempts to interpret a word as a pattern. The word is replaced with a list of pathnames - files and directories - which successfully match the pattern. Note that in UNIX (unlike DOS), the shell actually performs the globbing. This means that UNIX console programs all support globbing automatically. You signify you want a word to be globbed by one of the following characters:

  • * Matches any number of characters, including none.
  • ? Matches any single character.
  • [...] Matches any one of the characters enclosed by the square brackets. Character pairs separated by a dash denote a range of valid characters. If the first character following the [ is a ! or a ^ then the match is inverted.

As with most things presented here, the best way to get used to globbing is to have a play with it. I recommend you initially use the echo command to see exactly what the glob pattern is expanded to.

% echo *.[ch]   # show names of .c and .h files
% ls -l ../*    # list files in parent directory
% vi */Makefile # edit the Makefile files in immediate subdirectories
% rm x?         # remove all two letter files beginning with x
% rm [A-Z]*     # remove all files starting with upper case letters

Note that file names beginning with dot (.) are not normally globbed, unless you explicitly include the dot yourself. There is a very good reasons for this. The following command could potentially be disastrous if .. was globbed.

% rm -rf *

In most flavours of UNIX you cannot' undelete[11].

Brace Expansion

The brace characters ({ and }) are also special characters (in bash at least). The following should illustrate their function.

% echo aa{1,2,3}bb
aa1bb aa2bb aa3bb
% ls -d /usr/local/{bin,lib}
/usr/local/bin	/usr/local/lib

Filename Completion

Filename completion allows you to partially type the name of a file or command and have the shell complete the rest.

To complete a filename, type the first few letters and hit tab or \ctrl{I}. If you've used enough characters to distinguish your file from all others in the directory, the shell will complete the filename for you, otherwise it will ring the keyboard bell.

bash uses the GNU readline library to do its filename completion. In the standard setup the tab key works as above, but hitting it a second time lists options. Readline can be configured by a .inputrc file in your home directory. Apparently, you can set tab to cycle through your options, rather than listing them. There is also a word (rather than pathname) completion option, based on the last few hundred words you've typed in. Consult the bash manual page for further information.

Filename completion is turned on in csh via:

% set filec

The author is ignorant of filename completion in tcsh, but assumes it's something similar to csh.

Most shells allow you to ignore certain file extensions. For bash this is set by the FIGNORE variable. The following will make bash ignore files ending in .o and .out. Note that they can still be globbed -- only filename completion will ignore them.

$ FIGNORE=".o:.out"

Command Line Editing

Both bash and tcsh have interactive command line editing. This means you can use control keys (like the arrow keys) to edit a command as you type it in. The default "mode" in bash and tcsh is for emacs-style editing keys. However, both shells also have a vi mode. To enable this in bash put the following command in your .bashrc.

set -o vi

For tcsh the following should be placed in your .cshrc.

bind vi

Functions and Aliases

Aliases are a rather simplistic attempt to allow you to define new commands or modify the behaviour of existing ones.

In csh and tcsh the alias command is also used to define new aliases.

% alias rm 'rm -i'
% alias list 'ls -la \!* | more'

The first alias redefines the rm command to prompt for confirmation before deleting. The second defines a command list which uses ls and more. The !* expands to the list of arguments given to the command. Thus, if you then entered:

% list /class/usr /users

the shell would effectively execute the command:

% ls -la /class/usr /users | more

bash uses a slightly different syntax for aliases. The above rm example would look like:

$ alias rm="rm -i"

Note that bash does not support !*. Instead, bash has functions. These are similar to aliases but vastly more powerful. Written as a function list looks like:

$ list() { ls -la $* | more }

The builtin command type can be used to view the definitions of aliases and functions defined in bash. The manual page for bash contains complete information.

Changing Your Shell

There are two ways of doing this. Firstly, leave your default shell as-is, but have it automatically start a different shell after logging in. This is achieved by adding the appropriate lines to the end of your shell startup file. To automatically start up bash in this manner if your default shell is csh:

% cat >> ~/.cshrc
exec /usr/local/bin/bash

The same idea can be used to start up tcsh from csh, but it's a little bit more tricky since tcsh also uses the .cshrc file. If tcsh itself tries to startup tcsh you'll end up in an infinite loop. This will not only prevent you from logging onto the machine and fixing the problem, it will probably bring the machine to its knees. It's worth noting that doing this during assignment time is potentially life threatening[12].

The second option is to change your default shell. This gives you quicker logins than the first option, since csh no longer has to be loaded, but it means that environment variables set in your current shell's login files are not be passed to the new shell. You have to find the important environment variables and set them yourself in your own login file[13]. You can use the chsh command to change your default shell, however it is really only worth doing if your are seriously going to use the additional features of the new shell.

  1. Including: ash, bash, clam, itcsh, ksh, tcsh and zsh. A notable, relative newcomer is rc, a shell originally written for the Bell Labs Plan 9 Operating System, with an emphasis on minimal features and clean design. Well worth checking out.
  2. See the document Csh Programming Considered Harmful by Tom Christensen for a brief overview of the most bogus aspects of csh and an explanation of why writing shell scripts in csh is a bad idea. Google it.
  3. Relatively speaking -- can you see how much this chapter has aged!?
  4. If you then type in help at the bash$ prompt, you will get the online help. Useful for finding out whether the shell you are running is in fact bash.
  5. A shell which has been started because the user has just logged in is called a login shell. Some shells consider this special, and will check for a login startup file. If they don't find one they will use the normal startup file. Putting all your commands in normal startup file will suffice for the normal user.
  6. Whitespace characters are spaces, tabs etc.
  7. In most shells, the term word refers to a whitespace separated sequence of characters, or a sequence of characters enclosed in double quotes.
  8. There is a rough convention that csh and its derivatives have % at the end of their prompts, and the Bourne shells have $. In either case, the super user usually has a # prompt. In the following text % will be used for csh and tcsh examples, and $ for bash ones.
  9. \n generated the new line.
  10. Although you could always start a screen.
  11. If you rely on this PCism, define your own rm command. Consult the UNIX FAQ for hints.
  12. You can apparently differentiate between tcsh and csh in your .cshrc with something like if ($?tcsh) then ... endif. Again, the author of this chapter professes ignorance in these matters.
  13. This isn't really discussed in this document, but running the env should be sufficient.
Personal tools