Linux Server

Goals

Concepts

Language

Library

Dependencies

Preview

Preparation

If you are using Windows, download and install either the PuTTY or the KiTTY SSH client.

Lesson

Linux is an open-source operating system and is one of the most popular server platforms. Since then then it has forked into a multitude of variations, each known as a distribution or distro. Linux even underlies Google's Android operating system on mobile devices.

Of the most popular Linux distributions there are two main divisions, based upon the package management system they use (explained later in this lesson). Many distributions have variations targeted for server or for personal desktop use.

Red Hat
A commercial Linux distribution by the company Red Hat, primarily through their Red Hat Enterprise Linux (RHEL) product. CentOS is a free, community based version of RHEL and is the most popular Linux distribution for web. CentOS is the distribution we will be using in this lesson. See The most popular Linux for Web servers is ....
Debian
One of the earliest Linux distributions. The company Canonical Ltd. produces another very popular Debian-based distribution named Ubuntu.

Logging In

Effecting a login to a Linux machine requires a username and a password. Upon login a user is placed in the user's home directory, which by by default is /home/username. The tilde ~ character is a shortcut way to represent the current user's home directory. The home directory is also stored in the $HOME environment variable.

SSH

Remote login is usually performed over a secure connection using the Secure Shell (SSH) protocol, which uses port 22 by default. SSH communication is based upon the public cryptography key shared during the first connection with a server. Future connections use this key not only to secure the connection but also to verify the identity of the server, which is why many SSH clients will ask if the server's public key should be stored for later use. The initial transfer of the public key is performed before the server has been identified, so in a very high security situation the public key should itself be transferred in a secure manner before the first connection.

Bash

Communicating with Linux is traditionally performed using a command-line interpreter called a shell. Although you can configure Linux to use your shell of choice, the most popular shell is Bash, which comes as the default on many Linux installations.

Built-In Commands

Bash as with other shells comes with several built-in commands which do not require the installation of a separate program. Here are a few of the most useful ones:

cd
Changes the current directory. Using cd by itself withouth indicating a directory name will change to the user's home directory.
echo
Prints something to the screen; e.g. echo Hello, World!
exit
Exits the script or the shell. If you have switched to another user during a session, exit will return to the session of the previous user.
export
Sets an environment variable for the current session; e.g. export FOO=bar For the variable to be available the next time you log in, you will need to add it to a login script, explained below.
help
Prints help information for a built-in command.
kill
Sends a signal to end a running process. TODO see below
pushd/popd
Changes to a new directory while placing the old one on an internal stack; and returns to the last pushed directory and removes it from the stack, respectively.
pwd
Prints the working directory.
source
Executes a script. The full stop or dot . character is a synonym for source.

TODO add command-line arguments to all commands above

Scripts

When you first log in, Bash executes several scripts in order:

  1. /etc/profile
    1. /etc/profile calls all scripts in /etc/profile.d/
  2. ~/.bash_profile
    1. ~/.bash_profile calls ~/.bashrc
      1. ~/.bashrc calls /etc/bashrc

Thus if you want to add an environment variable for a single user, add the appropriate export command to ~/.bashrc. If you want to add an environment variable for all users, the best practices is to create an appropriately named file inside /etc/profile.d/ (e.g. /etc/profile.d/maven.sh for Maven environment variables such as M2_HOME). If you add a new script file you must make it executable (e.g. chmod +x /etc/profile.d/maveh.sh), explained in Permissions below.

TODO tip about using source to reload profile

TODO discuss creating a script to be called by a user, specifying the shell

Environment Variables

Bash allows environment variables to be set using export, as shown above. TODO talk about placing in .bashrc Here are some of the most common environment variables:

_
The most recently executed command.
HOME
The home directory of the logged-in user.
HOSTNAME
The hostname of the machine.
PATH
The paths for searching for commands, separated by the colon : character.
SHELL
The path to the shell.
USER
The username of the logged-in user.

To retrieve the value of an environment variable, prefix it with the dollar $ sign, such as $HOME as you saw above.

Essential Commands

To get around in the shell and navigate the file system you need to be familiar with the following commands, along with some important command-line arguments:

cat [<option>]... [<file>]...
Prints the contents of files to the standard output. So named because it allows multiple files to be concatenated.
ls [<option>]... [<file>]...
Lists the contents of a directory.
--all (-a)
Lists all directories, even those starting with the full stop . character. TODO see below about hidden files
-l
Use a long listing format.
mkdir [<option>]... [<directory>]...
Creates directories.
more
Filters output to display information a screenful at a time.

TODO include rm, and note rm -rf

TODO include ln, add note about not putting link in directory form

Users and Groups

Linux controls access to resources more tightly than has traditionally been the case on desktop Windows systems. This is especially desirable in a server environment, where multiple people may enter the system but only certain of them should have access to administrative tasks.

As with most other platforms, the people who access the system are referred to as users. Users may be placed in one or more groups. Linux permissions may be specified for an individual user or for an entire group.

root

Linux, following the tradition of Unix, has a special user named root which is a superuser—it has permissions to modify anything in the system. The use of the root user is necessary for initial setup of the system, but best practices recommend that separate users be set up immediately for continued access and system maintenance.

sudo

The program sudo allows individual users to perform tasks that would normally only be accessible to root. Simply by prefixing a command with sudo, a user will be able to perform a task using root privileges. The user will need to provide his/her password (not the root password) to gain these privileges. The sudo command by default performs the command as root, but using the -u flag and specifying a username will perform the command as any specified user.

Adding and configuring users on CentOS and Ubuntu.
CentOS Ubuntu
Add User
  1. sudo adduser username
  2. sudo passwd username
sudo adduser username
Change Password sudo passwd username sudo passwd username
Give sudo Access sudo gpasswd -a username wheel sudo gpasswd -a username sudo
See Also How To Create a Sudo User on CentOS How To Create a Sudo User on Ubuntu

TODO: note about changing user settings after creation with usermod; see http://linuxg.net/how-to-change-a-user-login-shell-uid-or-homedir-with-usermod/

Files

All files in Linux are placed at some level under a single root / directory. The Filesystem Hierarchy Standard (FHS) provides a standard set of directories for Linux distributions to follow, although different distributions are at various levels of compliance of different levels of the standard. Here are some of the most important Linux top-level directories; most of them have specialized subdirectories for different types of executable files and data. TODO note about files beginning with a dot

/
Root directory of the entire file system hierarchy.
/etc/
System configuration files.
/home/
User home directories.
/lib/
Libraries.
/mnt/
Mounted file such as hard drives and DVD drives..
/opt/
Optional software packages.
/root/
Home directory for the root user.
/usr/
Contains shareable, read-only data. TODO note /usr/local/bin/ as a location for installing add-on executables
/var/
Place for variable files which are expected to change.

TODO note difference between /usr/local and /opt; see http://unix.stackexchange.com/a/11552

Example sections in long directory ln -l for directory foo and file bar.
Permissions Links Owner Group Size Modified Timestamp Filename
drwxrwxr-x 9 jdoe jdoe 4096 Oct 29 15:38 foo
-rw-rw-r-- 1 jdoe jdoe 1526374 Oct 2 20:15 bar.txt

Permissions

Every file (including directories) has three classes of permissions: for file owner user (u), for the file group (g), and for other users (o). For each class there are three permissions: read (r), write (w), and execute (x). A long directory listing will show file permissions in the first column, as in the following figure:

Example long directory listing with file permissions for directory foo and file bar.

drwxrwxr-x  9 jdoe jdoe    4096 Oct 29 15:38 foo
-rw-rw-r--  1 jdoe jdoe 1526374 Oct  2 20:15 bar.txt
File Type User (u) Group (g) Others (o)
Read Write Execute Read Write Execute Read Write Execute
foo d r w x r w x r - x
bar.txt - r w - r w - r - -

There are several commands available to change file permissions. Each of these commands allow the --recursive (-R) flag indicating that permissions should be changed recursively for all files in all subdirectories.

chown [<option>]... [<owner>][:[<group>]] <file>...
Changes the owner of a file; e.g. chown jsmith bar.txt sets jsmith as the owner of bar.txt.
chgrp [<option>]... <group> <file>
Changes the group of a file; e.g. chgrp students bar.txt sets the group of bar.txt to students.
chmod [<option>]... <mode>[,<mode>]... <file>...
Changes the file mode bits, that is, the permissions. For the mode, specify the symbol indicating the class: user (u), group (g), others (o), and/or all (a); followed a symbol to add permissions (+), to remove permissions (-), or to set permissions (=); followed by the permissions read (r), write (w), and/or execute (x). For example chmod g+rw foo --recursive adds read and write permissions for each file's group in the entire foo directory hierarchy. If no class is indicated, the default is all (a).

Processes

As you learned when studying Java threads, each application runs in an operating system process. In Linux each one is given a process ID (PID), which you can use for indicating a process in commands.

Listing Processes

Here are common commands you can use for listing processes.

top [<option>]...
Shows a dynamically updated list of all running processes. For a more user-friendly dynamically-updating display of processes, you can use the htop command, but usually it must be installed separately. See Packages below. On CentOS htop is available from the EPEL, also explained below.
ps [<option>]...
Produces a static list of processes in various formats. The ps switches with the hyphen - character are different than those without. For example u shows detailed information, while -u restricts the processes to those of a user.
pgrep [<option>]...
Returns the PID for a process.

Killing a Process

Stopping a process is done by the built-in kill command, already noted in a section above. The command kill -TERM foobar (or simply the kill foobar) sends a signal to the process to terminate. If the process refuses to shut down, you can use -KILL option which bypasses the application, routing signal to to operating system.

Commands

TODO wget, curl, more

TODO tar, especially with -J for xz

TODO pwd

TODO ln

TODO ln -al; pushd/popd; source (.)

TODO nano, screen

systemd

In addition to the processes you may run manually and that eventually end, for Linux to function there must be many processes that stay running, or run periodically. For example, if it were not for the SSH daemon sshd running, there would be nothing listening when you try to connect to a Linux server using an SSH client. For these sort of services, the major Linux distributions have migrated to a system of programs and daemons called systemd.

Like Maven, systemd is declarative. The services to run are defined as targets, and individual services declare dependencies. When it comes time to run a service, systemd can examine the dependencies to determine in which order services must be started. The individual jobs that systemd controls are called units.

Common systemd unit types
service unit
Performs some services.
mount unit
Attaches a filesystem.
timer unit
Defines a recurring event to run some service.
target unit
Groups other units.

systemctl

The primary program to control systemd is called systemctl. It has several commands for managing units:

list-units
Lists all configured units. This is the default command; you can simply enter systemctl to see the available units.
status <unit>
Gives the status of some unit.
start <unit>
Starts a unit.
stop <unit>
Starts a unit.
restart <unit>
Stops and then restarts a unit.
enable <unit>
Configures a unit to start automatically after reboot.
disable <unit>
Stop a unit from starting automatically after reboot.
reload <unit>
Reloads a unit's configuration without stopping the unit.
daemon-reload
Reloads the configurations for all systemd units.

Unit Configuration

Adding a new unit (or configuring an existing one) is accomplished by adding a unit file to the directory /etc/systemd/system. The base name of the unit file indicates the name of the unit, while its extension indicates the type of unit. For example to configure a service for the Tomcat server, you would create a file named /etc/systemd/system/tomcat.service. (See the section on Tomcat for more details and a sample service unit file.)

Defining, starting, and enabling a service thus includes the following steps:

  1. Create the unit file /etc/systemd/system/example.service.
  2. Reload the systemd configurations using systemctl daemon-reload.
  3. Start the service using systemctl start example.service.
  4. Enable the service for automatic starting with systemctl enable example.service.
  5. User systemctl status example.service to ensure the service is started and enabled.

Packages

TODO prose

TODO mention groups

CentOS Ubuntu
Update Package Lists yum check-update sudo apt update
Upgrade Installed Packages sudo yum update sudo apt upgrade
Find a Package yum search string apt search string
View Package Information yum info package apt show package
Install Package sudo yum install package sudo apt install package
Install Package Group sudo yum groupinstall "group" sudo apt-get install group
Remove Installed Package sudo yum remove package sudo apt remove package
Add Repository sudo yum install repository sudo add-apt-repository ppa:repository/ppa
See Also Package Management Basics: apt, yum, dnf, pkg

TODO note about how to list users

TODO list some common packages to install, e.g. nano (not available by default on CentOS 7)

Applications

Linux applications are installed using the package system for the distribution. Here are some popular applications, along with special installation and configuration details.

Git

The installed version of Git on both CentOS and Ubuntu is out of date. You'll need to either install it manually or pull it from an alternate repository. Don't forget that each user must still set his/her Git username and email.

CentOS
wget https://www.kernel.org/pub/software/scm/git/git-2.10.1.tar.xz
tar -xf git-2.10.1.tar.xz
cd git-2.10.1
make configure
./configure --prefix=/usr #Necessary for CentOS.
sudo make install

See How To Install Git on CentOS 7.

Ubuntu

You can install the latest version of Git directly from the PPA ppa:git-core/ppa.

sudo add-apt-repository ppa:git-core/ppa
sudo apt-get install git

Java

CentOS

Although it not too difficult install Oracle's version of Java on Linux, the installation packages require a procedure outside the standard package-management command, including a manual download, cookie support, and acceptance of a license agreement. More straightforward is installing the OpenJDK, an open-source implementation of Java. OpenJDK comes in two variations: java-1.8.0-openjdk which only includes the JRE, and java-1.8.0-openjdk-devel which includes the entire JDK.

sudo yum install java-1.8.0-openjdk-devel

See How To Install Java on CentOS and Fedora.

Ubuntu

TODO; see https://www.digitalocean.com/community/tutorials/how-to-install-java-with-apt-get-on-ubuntu-16-04

TODO note alternate PPA for getting Java 8 on 14.04, e.g. WSL; see http://askubuntu.com/q/464755

TODO maybe simplify by creating a table with package names

Maven

The installed version of Maven on both CentOS and Ubuntu is out of date. It is straightforward to install it manually, however, and gives a good example of installing standalone applications that do not need to be built (as Maven uses Java).

Install Maven Files

Download and install Maven in a separate /opt/apache-maven-x.x.x/ directory, and create a symbolic link from /opt/maven/ to this directory. This allows us to easily upgrade Maven by pointing /opt/tomcat/ to a different directory in the future.

wget http://mirrors.ibiblio.org/apache/maven/maven-3/3.3.9/binaries/apache-maven-3.3.9-bin.tar.gz
sudo tar -xf apache-maven-3.3.9-bin.tar.gz --directory /opt/ #sudo needed for writing to /opt
sudo ln -s /opt/apache-maven-3.3.9/ /opt/maven #create symbolic link for home directory
sudo ln -s /opt/maven/bin/mvn /usr/local/bin/mvn #add system path link to Maven executable
Define Maven Environment Variables

Use nano or another editor to create the /etc/profile.d/maven.sh file below, which will set the Maven environment variables for the entire system:

sudo nano /etc/profile.d/maven.sh
export M2_HOME=/opt/maven/
source /etc/profile #reload profile to set environment variables

Tomcat

The version of Tomcat available from the repositories is several versions behind, so you'll want to install Tomcat manually to get the latest version. This will also give you experience with creating users, setting permissions, and configuring a systemd unit.

TODO prose about /opt/tomcat

Create Tomcat User and Group

Create a system user tomcat with group tomcat, with no login privileges and a home directory in /opt/tomcat/. The actual directory will be created later. See Creating Group and User for Tomcat RHEL/CentOS for a discussion of installation variations.

sudo useradd -r -s /sbin/nologin -d /opt/tomcat/ tomcat
Install Tomcat Files

Download and install Tomcat in a separate /opt/apache-tomcat-x.x.x/ directory, and create a symbolic link from /opt/tomcat/ to this directory. This allows us to easily upgrade Tomcat by pointing /opt/tomcat/ to a different directory in the future.

wget http://download.nextag.com/apache/tomcat/tomcat-8/v8.5.6/bin/apache-tomcat-8.5.6.tar.gz
sudo tar -xf apache-tomcat-8.5.6.tar.gz --directory /opt/ #sudo needed for writing to /opt
sudo ln -s /opt/apache-tomcat-8.5.6/ /opt/tomcat #create symbolic link for home directory
Configure Tomcat Permissions

Give the tomcat group access to the entire Tomcat installation, but only give the tomcat user write access to certain files in case there is a security breach. See Apache Tomcat 8 Security Considerations for more details on Tomcat security.

sudo chgrp --recursive tomcat /opt/tomcat/ #give tomcat group access to Tomcat installation
sudo chmod g+r --recursive /opt/tomcat/ #give tomcat group read access to Tomcat installation
sudo chmod g+rwx /opt/tomcat/conf/ #allow tomcat group to add files to the configuration directory
sudo chown --recursive tomcat /opt/tomcat/logs/ /opt/tomcat/temp/ \
    /opt/tomcat/webapps/ /opt/tomcat/work/ #make tomcat user owner of working directories
Set Up Tomcat systemd Unit

Use nano or another editor to create the /etc/systemd/system/tomcat.service file below:

sudo nano /etc/systemd/system/tomcat.service
# systemd unit file for Tomcat
[Unit]
Description=Apache Tomcat Web Application Container
After=syslog.target network.target

[Service]
Type=forking
SuccessExitStatus=143

Environment=JAVA_HOME=/usr/lib/jvm/jre
Environment=CATALINA_PID=/opt/tomcat/temp/tomcat.pid
Environment=CATALINA_HOME=/opt/tomcat
Environment=CATALINA_BASE=/opt/tomcat
Environment='JAVA_OPTS=-Djava.awt.headless=true -Djava.security.egd=file:/dev/./urandom'

ExecStart=/opt/tomcat/bin/startup.sh
ExecStop=/bin/kill -TERM $MAINPID

User=tomcat
Group=tomcat

[Install]
WantedBy=multi-user.target
Define Tomcat Manager User

If you plan on deploying via the Tomcat Manager App or as part of a Maven build, define a Tomcat user in /opt/tomcat/tomcat-users.xml having the manager-gui and/or manager-script roles, respectively. (See Migration Guide - Tomcat 7.0.x: Manager Application for a list of default supported roles.) This allows the identified user to deploy applications via the web UI, or via automated build scripts (such as within Maven), respectively. Set an appropriate manager username and password.

sudo nano /opt/tomcat/conf/tomcat-users.xml
<?xml version="1.0" encoding="UTF-8"?>
<tomcat-users xmlns="http://tomcat.apache.org/xml"
              xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
              xsi:schemaLocation="http://tomcat.apache.org/xml tomcat-users.xsd"
              version="1.0">
  …
  <user username="manager username" password="manager password" roles="manager-gui,manager-script"/>
</tomcat-users>

TODO mention how to change port number

Start Tomcat Service

Now that you've defined a service for Tomcat, tell systemd to reload the service files and then start Tomcat. You may also want to enable the Tomcat service so that it starts automatically when the system boots.

sudo systemctl daemon-reload #refresh systemd
sudo systemctl start tomcat #start Tomcat
sudo systemctl enable tomcat #configure the Tomcat service to start on boot (optional)
Test Tomcat

If you were successful, you can access the default Tomcat splash page in a web browser by accessing port 8080, e.g. http://localhost:8080/.

Deploying to Tomcat

Autodeploy a WAR via webapps/

The easiest way to deploy a web application to Tomcat is simply to copy the WAR file to the Tomcat webapps/ directory. Tomcat will automatically open the WAR file (which by default will be readable by Tomcat even if owned by root) and install the web application in its own context, using the WAR base filename as the context root. For example adding a foo.war file would use /foo/ as the context path for any servlets in the application; if a servlet in the web application used the annotation @WebServlet("/bar"), on the servlet would be accessible at /foo/bar.

cd foo #change to the foo project directory
sudo cp target/foo-1.0.0-SNAPSHOT.war /opt/tomcat/webapps/foo.war

Deploying via the Manager App

TODO warning about unsecure connection

TODO

TODO note about being able to use the web management interface to upload a WAR

TODO see http://tomcat.apache.org/maven-plugin-2.2/

TODO: add tomcat7-maven-plugin; see scratch application; see http://tomcat.apache.org/maven-plugin-2.0/tomcat7-maven-plugin/

TODO describe setting password in external configuration file; stress importance of credentials being outside the POM

TODO describe mvn tomcat7:deploy/redeploy

Review

Summary

Gotchas

In the Real World

Think About It

Self Evaluation

Task

A dedicated virtual CentOS server has been created for you, and you will be emailed the IP address and root credentials. Configure the server and deploy your Booker servlet on the server running on Tomcat.

  1. Log in as root.
  2. Immediately change the root password. TODO important: use a strong password.
  3. Create a separate user with sudo capabilities. TODO important: Log out and only log on as the other user under normal circumstances.
  4. Update the system with the latest packages.
  5. Install Git, Maven, and Tomcat.
  6. Deploy your Booker servlet on Tomcat
    1. Clone your booker-server repository somewhere in your individual use home directory.
    2. Package your booker-server project using Maven.
    3. Auto deploy the WAR file by copying it to the Tomcat webapps/ directory.

See Also

References

Resources

Acknowledgments