Restart Process

The following procedure can help you backup the state of a Linux server running software, etc. The idea is to save all the

Restart netstat command

information that will allow you to go back to the same state after a restart. This can be helpful if you are a beginner or experienced sysadmin and need to restart a production server fast. At the same time make sure you can go back to initial state if stuff does not work automatically.

Backup your servers files and services

1. 1. Run any special backup system that can recover the system to the orginal state. I mean something like snapshot, other files or special backups you have for like DB, etc. If you do not have a backup system. I worked with a good friend on the following backup python script nc-backup-py . We have tested for Debian, Ubuntu, Redhat, CentOS and AMI Linux. The script is compatible with versions 2.6 and 27 of Python. It can do many types of backup and is quite flexible. Anyways I am pretty biased on this, so you can ignore what I am saying.

   2. Backup state.

      Save uptime so you can have information about running time, load date, etc:

      # uptime > /opt/abel_backup/uptime-`date +%Y%m%d`
      
      

   3. Make a copy the firewall state. While at the same time make sure it does not start when you reboot. If you are in the cloud and lock your self out, you might to through a lot of trouble trying to get out of this. Same goes to other environments.
      Rpm distros to disable firewall:

      # chkconfig iptables off

      Deb distros (Ubuntu):

      # ufw disable

      If there is no UFW, then you might be on some other Debian “Flat” distro. Make sure iptables accepts connections:

      # iptables -P INPUT ACCEPT
      # iptables -P OUTPUT ACCEPT
      # iptables -P FORWARD ACCEPT

      Clean iptables rules.

      # iptables -F

   4. Check the firewall. Most distros just use “iptables generators”. If that is the case.
      See if there is any firewall:

      # iptables -nvL

      There might be some other stuff. Netfilter is supposed to replace iptables soon. Anyhow for now the previous will work on more than 95% of the Linux machines. Then save the currently running iptables:

      # iptables-save > /opt/iptables-save-`date +%Y%m%d`

   5. Compare /etc/fstab which is what you want to have mounted with. Then /etc/mtab in other words what is currently mounted. Ensure they are the same outout:

      # egrep "^/dev/|^UUID|^LABEL" /etc/fstab | grep -v swap | awk '{print $2}' | sort | tee /opt/abel_backup/fstab-`date +%Y%m%d`
      /
      # egrep "^/dev/|^UUID" /etc/mtab | grep -v swap | awk '{print $2}' | sort | tee /opt/mtab-`date +%Y%m%d`
      /
      #

      These two outputs are supposed to be identical. If they are not you might have stuff mounted that is not in fstab. In which case you might want to check that and vise versa.

   6. Now, look for listening ports and save it so you can compare once done:

       # netstat -nltp > /opt/abel_backup/netstat-`date +%Y%m%d`

      If the server does not have netstat it might have ss. Ss can basically use the same parameters. Otherwise you can install ether of them.

   7. Save routes:

      route -n > /opt/abel_backup/route-`date +%Y%m%d`

   8. Save the list of running processes. This will allow you to check after restart and compare if required:

      ps wwwaux > /opt/abel_backup/ps-`date +%Y%m%d`
      
      

   9. Stop services

      Happily you are almost ready to restart. Stop the running services that you might have, like Mysql, webservers, etc. Depending on the distro this might be done with systemctl, service or init.d.
      CentoOS7, Ubuntu 16, Debian 9 and other new distros generally use systemd, but not all of them. For those with systemd, you can use the following:

      # systemctl stop <xxx.service>

      For other/older you might need to use init.d command:

      # /etc/init.d/xxx stop
      
      Service works for most distros and setups:

      # service xxx stop

      Notice: xxx is the service you want to stop, like mysql or nginx.
      If you installed some non standard software you might need to stop it by killing it (kill command), but be careful and inform yourself before doing it.

   10. Ensure all the important services are not running:

       # ps aux | grep

   11. You might need to do some other stuff specific to you platform. In most distributions simple procedure will help you go back to the same state after you perform the restart.
       So this step is the restart it self:

       # shutdown -rf now

Conclusions

This procedure different steps can help you restart a production server and once it comes back online check that all is working the same way as before. In another post I will explain how to check those things, but with not much knowledge you might already know what this is all about.

Thanks for reading.