# Reset example files and folders
cd ${HOME}
mkdir -p exampledirectory1 exampledirectory2
rm -f exampledirectory?/*
rm -f examplefile2
echo "File1" > exampledirectory1/file1
echo "File2" > exampledirectory1/file2
echo "File3" > exampledirectory1/file3
echo -e "This Is Line1\nThis Is Line2\nThis Is Line3\nThis Is Line4" > examplefile
chmod 755 exampledirectory1 exampledirectory2
chmod 644 examplefile
Confused?
Maybe this looks more like what you expected:
But is that a shell?
Yes and no. Let's have a look at the definition of a shell:
In computing, a shell is a user interface for access to an operating system's services. In general, operating system shells use either a command-line interface (CLI) or graphical user interface (GUI), depending on a computer's role and particular operation. Wikipedia
Maybe you wouldn't expect it, but the Windows Desktop, including the Windows Explorer, is a shell (and Microsoft actually calls it so).
On Linux systems we usually mean CLIs (command-line interfaces) when we're talking about shells. That doesn't mean that there aren't graphical shells. The X window system + your desktop environment is also a shell.
Back to our Linux “shell”…
What we see here are actually two things:
The terminal window is basically a software version of this little fellow here: a very modern VT100 terminal (therefore the name “terminal emulator”).
Linux systems (and many other UNIX-like systems) follow a filesystem structure standard called the “Filesystem Hierarchy Standard” (FHS).
Basic principle: All folders fall into these two orthogonal categories: shareable vs. unshareable – static vs. variable
“Shareable” files are those that can be stored on one host and used on others. “Unshareable” files are those that are not shareable. For example, the files in user home directories are shareable whereas device lock files are not.
“Static” files include binaries, libraries, documentation files and other files that do not change without system administrator intervention. “Variable” files are files that are not static.
Furthermore, the standard defines a basic folder tree.
The most important directories:
/
– The root filesystem/bin
– Essential user command binaries/boot
– Static files for the boot loader/dev
– Device files/etc
- Host-specific system configuration/home
– User home directories/lib
(/lib64
) – Essential shared libraries and kernel modules/media
– Mount point for removable media/mnt
– Mount point for temporarily mounted filesystems/opt
– Add-on application software packages/root
– Home directory of the root user/run
– Run-time variable data/sbin
– System binaries/srv
– Data for services provided by this system/tmp
– Temporary filesBesides these top-level directories FHS defines two secondary filesystem trees:
/usr
– Contains shareable, read-only data used by all users of the system/var
– Contains variable (shareable and unshareable) data files (e.g. mail spools, log files, temporary files)Both directories contain a subtree similar to the top-level root filesystem tree.
/usr
:¶/usr/bin
– Most user command binaries/usr/lib
– Libraries and object files for user programs/usr/share
– Static, architecture-independent resource files (definitions, icons, media files, *.desktop files, static game data etc.)Besides the preceding list of directories, there are also some very important configuration files which you should know about.
/etc/passwd
– usernames and home directories, readable by everyone (usually no passwords included, despite the name)/etc/shadow
– passwords for users, only readable by root/etc/hosts
– mapping from 127.0.0.1 and [::1] to localhost, additional hostname configurations/etc/fstab
– system-wide mount points for storage devices/etc/profile
, /etc/profile.d/*
– global shell settingsNow that we have a basic understanding of the system we're working on, we can start with actual command-line fu. \o/
Once logged into a system, you'll be placed in an initial directory, usually your home directory. The folder you're currently residing in is your current working directory (CWD).
To find out where we are, type
pwd
/home/cmdline
Great! we know where we are. I guess you're curious whether there are files in this directory…
ls
exampledirectory1 examplefile seminar exampledirectory2 My Example File.txt somedirectory
Is that all? Probably not.
ls -a
. .cache .hiddenfile .profile .w3m .. exampledirectory1 .ipython seminar .bash_history exampledirectory2 .jupyter somedirectory .bash_logout examplefile .local .ssh .bashrc .gitconfig My Example File.txt .viminfo
Files that start with a dot are hidden files. Use the -a
flag to list hidden files, too.
If you want more information than just the file names, use -l
. This will also tell you the owner of a file, its size, permissions and last modification date. -h
makes the filesize more human-readable (instead of printing bytes).
ls -lh
total 20K drwxr-xr-x 2 cmdline cmdline 4,0K Oct 26 18:04 exampledirectory1 drwxr-xr-x 2 cmdline cmdline 4,0K Oct 26 18:28 exampledirectory2 -rw-r--r-- 1 cmdline cmdline 56 Oct 26 18:28 examplefile -rw-r--r-- 1 cmdline cmdline 0 Oct 26 18:27 My Example File.txt drwxr-xr-x 6 cmdline cmdline 4,0K Oct 26 18:28 seminar drwxr-xr-x 2 cmdline cmdline 4,0K Oct 26 18:26 somedirectory
You can of course also combine it with the -a
flag to show hidden files.
The CWD can be changed with the cd
command.
cd exampledirectory1
Verify that we're in a different directory now:
pwd
/home/cmdline/exampledirectory1
The shortcuts .
and ..
represent the current and the parent directory. It's easy to go one level back up:
cd ..
pwd
/home/cmdline
As already seen before, each file and directory has a basic set of permissions assigned to it.
Permissions are either represented as characters (e.g. -rw-rw-r--
) or octal numbers (e.g. 644
).
Recall our file listing from before:
ls -l
total 20 drwxr-xr-x 2 cmdline cmdline 4096 Oct 26 18:04 exampledirectory1 drwxr-xr-x 2 cmdline cmdline 4096 Oct 26 18:28 exampledirectory2 -rw-r--r-- 1 cmdline cmdline 56 Oct 26 18:28 examplefile -rw-r--r-- 1 cmdline cmdline 0 Oct 26 18:27 My Example File.txt drwxr-xr-x 6 cmdline cmdline 4096 Oct 26 18:28 seminar drwxr-xr-x 2 cmdline cmdline 4096 Oct 26 18:26 somedirectory
The first character simply shows the type of the file (d
for directory, -
for normal file).
The other nine (3x3) characters represent the permissions for the owning user, the owning group and others (i.e. everyone else). These are readable (r
), writable (w
) and executable ( x
).
User and group of a file are shown in the third and fourth column of the file listing.
File permissions can be changed by the chmod
command. The command takes the new permissions and the filename as parameter:
chmod a=rwx examplefile
ls -l
total 20 drwxr-xr-x 2 cmdline cmdline 4096 Oct 26 18:04 exampledirectory1 drwxr-xr-x 2 cmdline cmdline 4096 Oct 26 18:28 exampledirectory2 -rwxrwxrwx 1 cmdline cmdline 56 Oct 26 18:28 examplefile -rw-r--r-- 1 cmdline cmdline 0 Oct 26 18:27 My Example File.txt drwxr-xr-x 6 cmdline cmdline 4096 Oct 26 18:28 seminar drwxr-xr-x 2 cmdline cmdline 4096 Oct 26 18:26 somedirectory
This will set all (a
) permissions to rwx
(full permissions). Permissions can also be set for only the user (u
), the group (g
) or everyone else (o
). Multiple definitions are separated by commas.
chmod u=rwx,g=rw,o=r examplefile
ls -l
total 20 drwxr-xr-x 2 cmdline cmdline 4096 Oct 26 18:04 exampledirectory1 drwxr-xr-x 2 cmdline cmdline 4096 Oct 26 18:28 exampledirectory2 -rwxrw-r-- 1 cmdline cmdline 56 Oct 26 18:28 examplefile -rw-r--r-- 1 cmdline cmdline 0 Oct 26 18:27 My Example File.txt drwxr-xr-x 6 cmdline cmdline 4096 Oct 26 18:28 seminar drwxr-xr-x 2 cmdline cmdline 4096 Oct 26 18:26 somedirectory
Instead of setting the full permission bitmask with =
, individual bits can be added or subtracted using +
and -
.
The easiest way to add the executable bit for user, group and others is
chmod +x examplefile
ls -l
total 20 drwxr-xr-x 2 cmdline cmdline 4096 Oct 26 18:04 exampledirectory1 drwxr-xr-x 2 cmdline cmdline 4096 Oct 26 18:28 exampledirectory2 -rwxrwxr-x 1 cmdline cmdline 56 Oct 26 18:28 examplefile -rw-r--r-- 1 cmdline cmdline 0 Oct 26 18:27 My Example File.txt drwxr-xr-x 6 cmdline cmdline 4096 Oct 26 18:28 seminar drwxr-xr-x 2 cmdline cmdline 4096 Oct 26 18:26 somedirectory
(note the implied a
in front of the +
)
In the same way we can remove write permissions for just the group:
chmod g-w examplefile
ls -l
total 20 drwxr-xr-x 2 cmdline cmdline 4096 Oct 26 18:04 exampledirectory1 drwxr-xr-x 2 cmdline cmdline 4096 Oct 26 18:28 exampledirectory2 -rwxr-xr-x 1 cmdline cmdline 56 Oct 26 18:28 examplefile -rw-r--r-- 1 cmdline cmdline 0 Oct 26 18:27 My Example File.txt drwxr-xr-x 6 cmdline cmdline 4096 Oct 26 18:28 seminar drwxr-xr-x 2 cmdline cmdline 4096 Oct 26 18:26 somedirectory
Read and write permissions for folders work basically the same as for files. They define whether you can list a folder's contents (using the ls
command) and whether you can add or delete files in this directory.
But what do execute permissions mean on a folder?
Execution rights on a folder allow the user to
cd
command# Make sure command exits with clean exit code 0
# (necessary to continue execution of further cells)
alias cd="echo bash: cd: exampledirectory1: Permission denied; true"
chmod -x exampledirectory1
cd exampledirectory1
bash: cd: exampledirectory1: Permission denied
unalias cd
chmod +x exampledirectory1
A shorter (yet slightly harder to understand) notation is using octal numbers.
Octal permissions are defined as follows:
r := 4
w := 2
x := 1
To get to the desired permission set, add these numbers for each of user, group and others.
The resulting three-digit octal number represents the full set of permissions.
Express the permission set -rwxrw-r--
as octal number.
r+w+x = 4+2+1 = 7
r+w = 4+2 = 6
r = 4
Building a 3-digit number from it, we end up with 764
.
chmod 764 examplefile
ls -l
total 20 drwxr-xr-x 2 cmdline cmdline 4096 Oct 26 18:04 exampledirectory1 drwxr-xr-x 2 cmdline cmdline 4096 Oct 26 18:28 exampledirectory2 -rwxrw-r-- 1 cmdline cmdline 56 Oct 26 18:28 examplefile -rw-r--r-- 1 cmdline cmdline 0 Oct 26 18:27 My Example File.txt drwxr-xr-x 6 cmdline cmdline 4096 Oct 26 18:28 seminar drwxr-xr-x 2 cmdline cmdline 4096 Oct 26 18:26 somedirectory
Although numeric permissions seem to be complicated, in almost all cases you only need to remember five numbers.
Besides changing file permissions and listing directories you can of course also create, copy, rename, delete and create files and folders.
Copying files can be done with the cp
command:
cp [OPTION]... SOURCE DEST
cp [OPTION]... SOURCE... DIRECTORY
Common options are:
-R, -r, --recursive copy directories recursively
-a, --archive preserve most attributes if possible
(such as owner, permissions), implies `-R`
-v, --verbose explain what is being done
Renaming files can be done with the mv
command:
mv [OPTION]... SOURCE DEST
mv [OPTION]... SOURCE... DIRECTORY
Common options are:
-f, --force do not prompt before overwriting
-i, --interactive prompt before overwriting
-v, --verbose explain what is being done
Removing/deleting files can be done with the rm
command:
rm [OPTION]... FILE...
Common options are:
-f, --force ignore nonexistent files and arguments, never prompt
-i prompt before every removal
-r, -R, --recursive remove directories and their contents recursively
(DANGEROUS!!!)
-v, --verbose explain what is being done
Removing/deleting empty directories can be done with the rmdir
command:
rmdir [OPTION]... DIRECTORY
Common options are:
-v, --verbose output a diagnostic for every directory processed
Money question: why would you need rmdir
, when you can use rm -r
?
Counter question: Why do you cut your hair with scissors when you could use the lawn mower?
rmdir
will fail when the directory is not empty. Similarly, rm
(without -r
) will fail if you try to delete a directory instead of a file. This is simply a sane precaution. Use rm -r
sparsely and only if needed and…
Creating new directories can be done with the mkdir
command:
mkdir [OPTION]... DIRECTORY
Common options are:
-p, --parents no error if existing, make parent directories as needed
-v, --verbose print a message for each created directory
Working with and editing (text) files on the command line is worth a full seminar. There are lots of tools you can use to work very effectively with text files on the command line.
We will cover some of these tools in future lessons, but some are also out of scope of this seminar.
In this lesson I will only present three basic tools: cat
, tail
and head
.
The cat
command reads one or multiple files (or standard input), concatenate them and print them on the standard output.
cat [OPTION]... [FILE]...
Common options are:
-n, --number number all output lines
-s, --squeeze-blank suppress repeated empty output lines
cat examplefile
This Is Line1 This Is Line2 This Is Line3 This Is Line4
Printing only the last n
(default: 10) lines of one or multiple files can be accomplished with the tail
command.
tail [OPTION]... [FILE]..
Common options are:
-n, --lines=K output the last K lines, instead of the last 10;
or use -n +K to output starting with the Kth
tail -n2 examplefile
This Is Line3 This Is Line4
tail -n+2 examplefile
This Is Line2 This Is Line3 This Is Line4
Similar to tail
, printing the first n
(default: 10) lines of one or multiple files can be accomplished with the head
command.
head [OPTION]... [FILE]..
Common options are:
-n, --lines=[-]K print the first K lines instead of the
first 10; with the leading '-', print
all but the last K lines of each file
head -n2 examplefile
This Is Line1 This Is Line2
head -n-1 examplefile
This Is Line1 This Is Line2 This Is Line3
Printing text files is neat, but most of the time we want to edit them.
Using input and output redirection (which we'll discuss in a later chapter) you can actually use cat
, head
and tail
to edit files. But most of the time it is much more convenient to use a text editor.
There are a number of text editors for the command line. The most common are
nano
(very easy to use, but not very powerful)vim
(very powerful, but takes a long time to learn)emacs
(as powerful as vim
with higher RMS factor)nano
¶nano
is very simple to use, but also doesn't provide many editing features.
All keyboard shortcuts are shown at the bottom. ^
means Ctrl
. So ^X
(the exit command) means Ctrl+X
.
If you try to exit the program but changed your file, you will be asked if you want to save.
Correctly configured, vim
provides all features of a modern text editor and most of certain development IDEs. However, it takes a lot of time to learn.
There are four modes in vim
:
When you start vim
, you are in normal mode. Tap i
to go into edit mode. <ESC>
to go back to normal mode.
Type :w<ENTER>
in normal mode to go temporarily to command mode and save your file.
Type :q<ENTER>
to exit vim
. If you changed your file but want to exit without saving, type :q!<ENTER>
There are some great web ressources about learning vim
. A small selection:
Additionally, there is also the command vimtutor
which comes with vim
. It will interactively run you through all the basic editing features, vim
provides.
Keep calm, none of us can.
If you ever forget how a command works, try running it with --help
or, if that doesn't work, with -h
(but be careful, in certain cases -h
may also mean something else than help). Most commands print basic usage instructions when provided with these flags.
For example:
mkdir --help
Usage: mkdir [OPTION]... DIRECTORY... Create the DIRECTORY(ies), if they do not already exist. Mandatory arguments to long options are mandatory for short options too. -m, --mode=MODE set file mode (as in chmod), not a=rwx - umask -p, --parents no error if existing, make parent directories as needed -v, --verbose print a message for each created directory -Z, --context=CTX set the SELinux security context of each created directory to CTX --help display this help and exit --version output version information and exit Report mkdir bugs to bug-coreutils@gnu.org GNU coreutils home page: <http://www.gnu.org/software/coreutils/> General help using GNU software: <http://www.gnu.org/gethelp/> For complete documentation, run: info coreutils 'mkdir invocation'
If that is not enough, there is almost always a Manual page (short: Manpage), accessible through the man
command:
man passwd
Note the (1)
behind the title. This tells you that you are currently looking at the manpage for the command passwd
. If you want information about the /etc/passwd
file, type:
man 5 passwd
There is a total of 9 possible categories for Manpages. An extract from the man
Manpage (man man
):
1 Executable programs or shell commands
2 System calls (functions provided by the kernel)
3 Library calls (functions within program libraries)
4 Special files (usually found in /dev)
5 File formats and conventions eg /etc/passwd
6 Games
7 Miscellaneous (including macro packages and conventions), e.g. man(7), groff(7)
8 System administration commands (usually only for root)
9 Kernel routines [Non standard]
Don't worry, if you can't remember them all. I cannot remember anything except for 1
, 5
and 8
either.
Most of the time you'll get the right Manpage without explicitly specifying the number anyway. And if not, you know where too look it up.
That marks the end of lesson 1. Next time we'll focus on some more advanced command line techniques. We'll cover:
Thanks for listening.