CSC315-DBS-and-CSC415-SE-with-Pulimood

CompSci DevOps Instructions

Getting Started

The server assigned to you (or your team) by your professor has the following course specific software already pre-installed: git, Apache httpd, GNU cc/gcc, GNU c++/g++, Ruby, Ruby on Rails, Postgresql client & server, Scala, Sun/Oracle Java JDK 1.8

The server is preconfigured with the following accounts: sysadmin (with a password provided by your professor), team, student1, student2, student3, student4, student5, student6 (all initially locked)

You (or someone from your team for team servers) should login via SSH to the sysadmin account using the password that your professor provided to you. You can then change that password to something else using the passwd command, e.g. passwd sysadmin

If this is a team/project server, you should set the team account password to something everyone on the team will remember. Since this account does not initially have a password, you will need to change it from the sysadmin account using the sudo command, e.g. sudo passwd team

If this is a team server, each team member should be assigned a student# account. The password can be set similar to the team account by using logging into the sysadmin account and using sudo, e.g. sudo passwd student1

For individual student servers, you should use student1 as your primary course account and use the team and other student accounts for testing, etc.

After all of your accounts are setup, you should update your system with the latest patches. Again, do this from the sysadmin account.

sudo yum -y update

Some patches won’t take effect until after a reboot. The best practice is to just reboot the server after the patches complete.

sudo init 6

NOTE: If your server ever becomes in accessible via SSH for whatever reason (crash, accidental shutdown, etc.), you will need to contact Shawn Sivy (ssivy@tcnj.edu) to request assistance in restarting the server since you do not have direct console access.  

Working with Yum

Yum is the package manager you will use to install new software and update existing packages. Below are some common commands you can use to update your system, find packages and install them. These should all be run from the sysadmin account.

Update the system with the latest patches. sudo yum -y update

Reboot the system. Always a good idea after installing patches, especially if it included a kernel patch. sudo init 6

Get a list of packages available. sudo yum list

List just the packages currently installed. sudo yum list installed

Install a specific package. It will add any dependencies to the installation too. sudo yum install wireshark

Remove a specific package. It will also remove any packages that depend on the one you are removing. A confirmation request will be displayed before anything is removed. sudo yum remove wireshark

List the packages that contain the file you specified. Always precede the file name with */ and enclose the entire string in double-quotes. This is good when you don’t know what package a particular file, binary, library or header file resides in. sudo yum whatprovides "*/wireshark"

Starting Services Manually and at Boot-up

Start PostgreSQL now. Replace start with stop to shutdown PostgreSQL now. sudo systemctl start postgresql

Set PostgreSQL to automatically start at boot-up. Use disable to reverse this feature. sudo systemctl enable postgresql

Start Apache httpd now. Replace start with stop to shutdown Apache httpd now. sudo systemctl start httpd

Set Apache httpd to automatically start at boot-up. Use disable to reverse this feature. sudo systemctl enable httpd

NOTE: PostgreSQL database and Apache httpd web server are already configured to start automatically at boot-up. The above instructions on enabling these services only apply if you change the default configuration.

Configuration Files

Apache httpd: Under directory /etc/httpd

PostgreSQL: In directory /var/lib/pgsql/data

PHP: In file /etc/php.ini

CakePHP: Installed under /usr/local/CakePHP (directory already included in /etc/php.ini)

Additional configuration files can be found under /etc and /etc/sysconfig

Any changes to the Apache httpd or PHP configuration files will most likely require a restart of the web server by running the following command as the sysadmin user: sudo systemctl restart httpd

Any changes to the PostgreSQL database configuration files will most likely require a restart of the database server by running the following command as the sysadmin user: sudo systemctl restart postgresql

Versions

To see the version of GNU C, C++, git and Ruby, use these commands:

Get the version of GNU C cc -v *OR* gcc -v

Get the version of GNU C++ c++ -v *OR* g++ -v

Get the version of Ruby ruby -v

Get the version of Apache httpd httpd -v

Get the version of PostgreSQL psql --version

Get the version of Git git --version

Creating a Web Pages and Web Programs

Web pages (.html, .php, etc.) should be created under /var/www/html. Any CGI programs go under /var/www/cgi-bin. You should avoid putting code or files that contain logins and passwords, such as a database login, in these directories. They should go in a directory outside of “web space” so that they are not accidentally served up by the web server.

The sysadmin, team, student1, student2, student3, student4 and student5 accounts have permissions to read and write files in the /var/www/html and /var/www/cgi-bin directories. Make sure the files you create are world readable since the web server runs as the apache user by default, e.g. chmod o+r myfile  

Python

There are two version of Python installed with your virtual server. The system provided one is version 2.7 and the custom installed one is 3.8. The 3.8 version is located in /usr/local/bin and should be found first when running python at the command-line. If you need python 2.7 specificy, specify the fully path on the command line, e.g. /usr/bin/python.

$ /usr/bin/python -V

Python 2.7.5

$ /usr/local/bin/python -V

Python 3.8.7

Installing the Ruby on Rails Demo

While a version of Ruby is installed with the operating system, it is probably too old for current Rails development. A custom copy of Ruby and Rails has been installed in the sysadmin account. If you need to update to a newer version or need to install Ruby and Rails in another account, copy the installruby.sh script (if necessary) from the sysadmin account and run it. Note that running this script may trigger this Rails demo bug. Follow any instructions that the script displays. If Ruby and Rails were previously installed, the script will instruct you to remove the existing installation before re-running the script.

Verify that the following is at the end of your .bash_profile login script (if necessary -- again, this is already configured in the sysadmin account). If changes are made, logout and back in again for them to take effect.

.bash_profile – add after the “###” line.

export PATH="$HOME/.rbenv/bin:$PATH"
eval "$(rbenv init -)"
export PATH="$HOME/.rbenv/plugins/ruby-build/bin:$PATH"

Before continuing, verify that you are using the correct location of Ruby.

[sysadmin@csc415-team01 ~]$ which ruby

~/.rbenv/shims/ruby = correct

/usr/bin/ruby = incorrect

Prior to creating your own Rails application in a team or student account, create the demo in the sysadmin account. It will download any missing or updated Rubygems as part of the Rails demo creation. The system may ask for the sysadmin password to authorized the updates. Only the sysadmin account and root are authorized to update the Ruby files.

cd
rails new demo
cd demo
rails server --binding=0.0.0.0 

Open a web browser and go to http://your_server_name:3000/ (e.g. http://csc415-server25.hpc.tcnj.edu:3000/) You should see an image like the one below (version numbers may be different).

When you are done admiring your Rails server demo page, you can press CTRL-C to exit. To remove the demo, use these commands.

cd ..
rm -r demo

If you ever get a file permissions error trying to create a new Rails server in the team or student accounts, go back to the sysadmin account and recreated the Rails demo. This will allow you to refresh any Rubygems that have been updated.

NAT Quirkiness

Because your virtual servers existing on the HPC Cluster, network address translation (NAT) is needed to talk to the campus network and the Internet. Because of this requirement, your server is known by two different IP addresses based on where you are trying to access it from.

When you are on the campus network proper (e.g. in a lab or in your dorm), your server is known by its name (e.g. csc415-server01.hpc.tcnj.edu) and a corresponding 159.91.x.x address (e.g. 159.91.51.1). Note that the name is csc415-server01.hpc.tcnj.edu, not csc415-server00.tcnj.edu. The 159.91.51.1 is this server’s NAT IP address. Inside the cluster its “real” IP address is 10.18.6.1.

When a packet is sent to 159.91.51.1, the HPC firewall intercepts it and maps the address to 10.18.6.1 on the way in (and way out). The interesting part is that if you want to talk to another server within the cluster (or even want to talk to yourself), you need to use the 10.18.x.x address and not the 159.91.x.x one. So if you need to access your neighbor’s server from within the cluster, you would use, for example, 10.18.6.2, not 159.91.51.2.

You don’t need to fully understand how NAT works, but be aware that if you see prompts for the “HPC-Firewall Password:” you are probably trying to access the wrong IP address.