.. _unix:
====================================
UNIX/LINUX operating system
====================================
Shells
++++++
A shell is a program that allows you to interact with the operating
system and sofftwares through typing commands. As opposed to a
graphical user interface (or GUI), which involves
point-and-click, a shell is a command line interface (or CLI), where
the user types in commands to interact with the computer.
On a Mac machine there is the option of using a Unix shell in a "terminal
window" (go to Applications –> Untilities –> Terminal to open a
terminal window). The Mac operating system (macOS) is a Unix-based
graphical operating system. The latest version of macOS is Catalina. A
few previous versions include Mojave, Sierra, Leopard, Lion, Yosemite, and El
Capitan.
Two common Unix shells are `bash`__ and `csh`__.
In this class you will mainly use the `bash` shell. In the shell (when
you open a terminal window) you will have a ``prompt`` usually denoted
by a single ``$`` where you can type in commands. If you think your shell is different, you can type:
.. code-block:: none
$ bash
This will start a new bash shell and give you the bash prompt to type
in commands. For example, type the command ``whoami`` at the
prompt and hit return (or enter) to see what response you get. I get
the following response:
.. code-block:: none
$ whoami
motamed
Now that our computer knows who we are, let's see what else the
computer can do.
**Remark**: Two other shells that you may already be familiar with include Matlab and Python. Most of you are already familiar with the Matlab shell (or "Matlab
command window" if you are using a GUI version of Matlab). A Matlab
shell uses the prompt ``>>``, instead of ``$`` used by bash. A Python
sheel uses the prompt ``>>>``. To invoke and use Pyhton, simply type
``python``:
.. code-block:: none
$ python
Python 2.7.13 (default, Apr 23 2017, 16:50:35)
[GCC 4.2.1 Compatible Apple LLVM 8.0.0 (clang-800.0.42.1)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> x=4
>>> y=2*x
>>> y
>>> 8
Type ``exit()`` to exit python and return to the shell.
pwd, cd, and ls
+++++++++++++++++++++++++++++++
The command ``pwd`` means "prints the working directory". The result
will be the full path to where your prompt is located in the directory
structure. Directories are called “folders†on windows systems.
Another useful command is ``cd`` that is used to change working directories. For example if I am
in the directory ``/home/motamed/`` and want to change to
``/home/motamed/TEACHING/MATH471``, where my files for the course Math
471 are stored, I would just type ``$ cd TEACHING/MATH471``, or alternatively ``$
cd TEACHING`` and then ``$ cd MATH471`` to go two levels down, one
level at a time. If you want to go up (towards the top levels), you can use ``$ cd ..`` which brings you up one level.
If you need to make a new directory use ``$ mkdir dir_name``, where
``dir_name`` is the name of the new directory chosen by you.
The command ``ls`` lists the contents (i.e. files and directories) of
the current working directory. It can be executed
with different "options" (or flags); see
``man ls``. For example, the ``-lt`` option provides a long listing
sorted by time modified. For this you just need to type:
.. code-block:: none
$ ls -lt
The ``-l`` option tells ``ls`` to display a long listing that contains additional
information about each file, such as how large it is, who owns it,
when it was last modified, etc. The first 10 characters indicate the
file type and read, write, and executing permission classes. See `here`__
for more information. Another example is the ``-a`` option, which
displays also the hidden files, such as those started with a period (or dot).
The command ``ls`` also takes "arguments", that is, a list of files to
apply ``ls`` to. For example, if we want to list the information
about a specific file (e.g. `git.rst` in my directory), we type:
.. code-block:: none
$ ls -l git.rst
We can also use the ``*`` character to list more than one file:
.. code-block:: none
$ ls -l *.rst
Displaying the contents of a file
+++++++++++++++++++++++++++++++++++++++++++
The commands ``less``, ``more``, ``head``, ``tail``, and ``cat`` can be used to
display the contents of files. The command ``less`` is somewhat more
than ``more``, as it can scroll both up and down. The commands ``head`` and ``tail``
display the first and last n lines of a file, respectively. The default number of
lines is n=10, but it can be adjusted to, say, 15 by typing ``$
tail -15 filename``.
The command ``cat`` is also used to read files: ``$ cat filename``. We
can also use ``cat`` to put its output (here "filename") to a new file
or to over-write an existing file: ``$ cat filename
> newfile`` or to append to an existing file ``$ cat filename >> existingfile``.
Copying, moving, and deleting files
+++++++++++++++++++++++++++++++++++++++++++
The commands ``cp``, ``mv``, and ``rm`` are used to copy, move, and
delete files and directories, respectively.
To copy a file named `original_file`, you can type:
.. code-block:: none
$ cp original_file copy_file
To copy an entire directory structure named `original_dir` (i.e. copying all files and subdirectories in
it), type:
.. code-block:: none
$ cp -r original_dir copy_dir
To rename a file or move to a different place (e.g. a different
directory), you can use the command ``mv``. You can for example type:
.. code-block:: none
$ mv oldfile newfile
If you want to delete a file named `filename`, you can type:
.. code-block:: none
$ rm -i filename
remove filename?
The ``-i`` flag forces ``rm`` to ask before removing the file. Other
useful flags for ``rm`` include ``-f`` for force to remove and ``-r`` for descending into sub-directories.
Environment variables
+++++++++++++++++++++
The bash shell uses environment variables to keep track of various
information. The command ``printenv`` will display any environment
variables you have set. Typing this command will give an output that looks something like this:
.. code-block:: none
[appelo@mizar ~]$ printenv
HOSTNAME=mizar.unm.edu
SHELL=/bin/bash
USER=motamed
PATH=/Users/motamed/mpich/bin:/usr/sbin:/sbin:/usr/local/bin:/sw/bin:/usr/bin
PWD=/Users/motamed/TEACHING/MATH471/webpage
HOME=/Users/motamed
LOGNAME=motamed
The environment variables can be accessed through the dollar sign, for
example ``$ ls $PWD`` would list the content of the current working
directory. OR ``$ cd $HOME/TEACHING/MATH579`` would take me to the
subdirectory of the course Math 579 no matter where your current
working directory is. You can print just one variable for example by:
.. code-block:: none
$ printenv PWD
/Users/motamed/TEACHING/MATH471/webpage
or
.. code-block:: none
$ echo $PWD
/Users/motamed/TEACHING/MATH471/webpage
You can also define new environment variables. For example, I can
generate a new variable `HPSC` and define it to be the path to my
course webpage directory:
.. code-block:: none
$ export HPSC=$HOME/TEACHING/MATH471/webpage
Now try ``$ echo $HPSC``.
PATH and the search path
++++++++++++++++++++++++++++
Whenever you type a command at the Unix prompt, the shell looks for a
program to run. This is true when you execute a built-in command or run a program that you have installed. To figure out where to
look for such programs, the shell searches through the directories
specified by the ``PATH`` variable, which in my case is
.. code-block:: none
$ printenv PATH
PATH=/Users/motamed/mpich/bin:/usr/sbin:/sbin:/usr/local/bin:/sw/bin:/usr/bin
So if you install some software in a new location, say ``HOME/newbin``, you may want to append ``PATH`` by using the ``export`` command
.. code-block:: none
export PATH=$PATH:$HOME/newbin
The search paths are separated by a colon. If you want the added directory to be searched first you can prepend the ``PATH`` instead
.. code-block:: none
export PATH=$HOME/newbin:$PATH
The .bashrc or .profile files
+++++++++++++++++++++++++++++++
Whenever you open a new terminal the file ``$HOME/.bashrc`` or
``$HOME/.profile`` (depending on your OS) is executed as a bash script. In this file you can put anything that you want
to be executed on startup. For example, if you want to do some
permanent changes (like exporting a new environment variable, such as
``HPSC`` above) or to always append the ``PATH`` variable (like we did
above), you can add them as new lines to your ``.bashrc`` or ``.profile`` file. You can also premanenetly redefine the biult-in commands. For example, you can
redefine ``ls`` to color the output by adding the line ``alias ls='ls
-G'`` to your ``.bashrc`` or ``.profile`` file.
which
+++++++++++++++++++++++++++++++
The ``which`` command displays the full path to the program that is being executed:
.. code-block:: none
$ which python
/opt/local/bin/python
$ which gfortran
/usr/local/bin/gfortran
top
+++++++++++++++++++++++++++++++
The ``top`` command displays the programs that are currently running
on the system with general information about memory and processor
usage. It is particularly useful to check for other users running
large programs before running one yourself on a shared computer. You
can also use it to monitor your own programs. To exit ``top`` simply
press ``q``. If would like to kill a process, find the PID
(process ID) number from ``top``, and then use the ``kill`` command in terminal window by typing ``$ kill PID_number``.
Logging in to a remote computer
+++++++++++++++++++++++++++++++
**ssh:** To login to a remote computer you can use the `secure shell`__, for example to login to ``linux.unm.edu`` I would type:
.. code-block:: none
ssh -X motamed@linux.unm.edu
Here the ``-X`` starts the X-server so you can open windows locally on
your machine. The result of the above command is the following message
(you will need to type your UNM password, the same as the one you use
to login to `my.unm`):
.. code-block:: none
========================================================================
= WARNING NOTICE TO USERS =
= Authorized uses only. =
= All activity may be monitored and reported. =
========================================================================
motamed@linux.unm.edu's password:
Last login: Thu Aug 13 15:42:19 2020 from MY.COM.PU.TER
---------------------------------
linux.unm.edu or ftp.unm.edu
is Red Hat Enterprise 7
---------------------------------
See available software and learn
about this environment at:
http://LinuxShellServices.unm.edu
[motamed@comet ~]$
I am then ready to type commands after the prompt ``$`` on the remote computer. To exit the
remote computer, simply type ``exit``.
**scp:** The ``scp`` command is used to transfer files between two
computers. For example, to copy a file named `filename1` in my local
directory to a directory `dir_name` on the remote computer, I would type:
.. code-block:: none
$ scp filename1 motamed@linux.unm.edu:dir_name
And to copy a remote file named `filename2` to my local machine, I
would type:
.. code-block:: none
$ scp motamed@linux.unm.edu:dir_name/dir_name .
where ``.`` in the end of the above command line means "`this
directory`". I could instead give the path to a different local directory.
-------------------
A good resource for reading more about shells and Unix/Linux is `software-carpentry`__.
For a list of Unix commands with details see `here`__.
__ http://en.wikipedia.org/wiki/Bash_(Unix_shell)
__ https://en.wikipedia.org/wiki/C_shell
__ https://en.wikipedia.org/wiki/File_system_permissions
__ http://en.wikipedia.org/wiki/Secure_Shell
__ https://software-carpentry.org/lessons/
__ https://en.wikipedia.org/wiki/List_of_Unix_commands