wiki:SystemAdmin

Version 92 (modified by Richard Bramley, 10 years ago) ( diff )

--

System admin for the BRICCS infrastructure

Error: Macro BackLinks() failed
'Environment' object has no attribute 'get_db_cnx'

Error: Macro BackLinks('Servers') failed
'Environment' object has no attribute 'get_db_cnx'

​LAMP Wiki

A list of the servers and systems we have available, and what software they are running.

Hospital systems

On the hospital side, the servers are Virtual Machines created within the UHL data centre facility managed by IM&T. We currently have nine VMs, visible only from within the hospital environment. One of them (uhlbriccsex01) has an external facing address for client access.

In addition we have two laptops with server software - running Ubuntu on VMWare Player to allow for REDCap to be delivered in a stand-alone mode for use initially in the GRAPHIC 2 study. More information is on the study page.

  • Standard packages: mailutils build-essential clamav htop fail2ban logwatch nagios3 chkrootkit snort make zip unzip apt-file python-software-properties
  • Mail setting 'Internet with smarthost': System needs to be able to manage local and outgoing mail. 'smarthost' used for everything else: smtp.xuhl-tr.nhs.uk - the server IPs then need adding to the UHL smtp whitelist. This is arranged by UHL IM&T (Paul Tracey).
  • ClamAV: Put the following in cron.weekly and chmod 755: 
#!/bin/bash
clamscan -ri --exclude-dir=^/sys\|^/proc\|^/dev / | mail -s "ClamAV Scan Results for `date +%D`" email@email.domain
  • All UHL machines need to be able to access the internet for updates, etc. For this they need to be 'allowed by rule' through the BlueCoat firewall. This is arranged by UHL IM&T (Geoff Harrison for VMs in the data centre, Bradley Wright for the laptops / desktop machines).

uhlbriccsapp01

  • Hostname : uhlbriccsapp01.xuhl-tr.nhs.uk
  • Alias on UHL: briccs.xuhl-tr.nhs.uk
  • Location : UHL 10.156.254.207
  • Subnet - 255.255.255.128, Gateway - 10.156.254.129
  • DNS - 10.147.126.165 10.147.126.166 10.156.249.165 10.156.249.166 10.160.15.165 10.160.15.166
  • Hardware : Virtual machine
  • Software : Ubuntu 9.10 - upgraded to Ubuntu 12.04.1 LTS on 09/11/2012
    • Packages: sun-java6-bin sun-java6-jdk maven2 perl openssl apache2 php webmin.deb php5-mcrypt mcrypt
  • Services
    • Onyx and dependencies, including the pmi lookup
    • MRBS for room bookings
    • REDCap for production studies and surveys
    • webmin to port 10000 (HTTPS)
    • byobu for multiple terminals if desired
    • Printing: cups on app01. Configure using web interface, but when in normal mode comment out the Listen 10.156.254.207 line from the /etc/cups/cupsd.conf - currently only user in lpadmin group is nick. Printer configured as 'TMF-HP' for use in Onyx.

briccsdb

  • Hostname : briccsdb.xuhl-tr.nhs.uk
  • Location : UHL 10.156.254.206
  • Subnet - 255.255.255.128, Gateway - 10.156.254.129
  • DNS - 10.147.126.165 10.147.126.166 10.156.249.165 10.156.249.166 10.160.15.165 10.160.15.166
  • Hardware : Virtual machine
  • Software : Ubuntu 9.10 - upgraded to Ubuntu 12.04.1 LTS on 09/11/2012
    • Packages: phpmyadmin mysql perl openssl webmin.deb
  • Services
    • MySQL database for Onyx, REDCap and the BRICCS ID label system.
    • phpmyadmin at /phpmyadmin/
    • webmin to port 10000 (HTTPS)
    • byobu for multiple terminals if desired

briccsdev

  • Hostname : uhlbriccsdev.xuhl-tr.nhs.uk
  • Location : UHL 10.156.254.218
  • OLD LOCATION: 10.147.126.57
  • Subnet - 255.255.255.128, Gateway - 10.156.254.129
  • DNS - 10.147.126.165 10.147.126.166 10.156.249.165 10.156.249.166 10.160.15.165 10.160.15.166
  • Hardware : Virtual machine
  • Software : Ubuntu 10.04 - upgraded to Ubuntu 12.04.1 LTS on 15/10/2012
    • Packages: phpmyadmin mysql perl openssl webmin.deb php5-mcrypt mcrypt
  • Services
    • OnyxTestSystem and dependencies, including the test pmi lookup
    • REDCap for testing

briccsdbdev

  • Hostname : uhlbriccsdbdev.xuhl-tr.nhs.uk
  • Location : UHL 10.156.254.217
  • OLD LOCATION: 10.147.126.56
  • Subnet - 255.255.255.128, Gateway - 10.156.254.129
  • DNS - 10.147.126.165 10.147.126.166 10.156.249.165 10.156.249.166 10.160.15.165 10.160.15.166
  • Hardware : Virtual machine
  • Software : Ubuntu 10.04 - upgraded to Ubuntu 12.04.1 LTS on 15/10/2012
    • Packages: phpmyadmin mysql perl openssl webmin.deb

uhlbriccsapp02

  • Hostname : uhlbriccsapp02.xuhl-tr.nhs.uk
  • Location : UHL 10.156.254.252
  • Subnet - 255.255.255.128, Gateway - 10.156.254.129
  • DNS - 10.147.126.165 10.147.126.166 10.156.249.165 10.156.249.166 10.160.15.165 10.160.15.166
  • Hardware : Virtual machine
  • Software : Ubuntu 10.04 - upgraded to Ubuntu 12.04.1 LTS on 09/11/2012
    • Packages: phpmyadmin mysql perl openssl webmin.deb
  • Services
    • Testing version of i2b2

uhlbriccsapp03

  • Hostname : uhlbriccsapp03.xuhl-tr.nhs.uk
  • Location : UHL 10.156.254.254
  • Subnet - 255.255.255.128, Gateway - 10.156.254.129
  • DNS - 10.147.126.165 10.147.126.166 10.156.249.165 10.156.249.166 10.160.15.165 10.160.15.166
  • Hardware : Virtual machine
  • Software : Ubuntu 10.04 - upgraded to Ubuntu 12.04.1 LTS on 09/11/2012
    • Packages: phpmyadmin mysql perl openssl webmin.deb
  • Services
    • Live version of i2b2

uhlbriccsapp04

  • Hostname : uhlbriccsapp04.xuhl-tr.nhs.uk
  • Location : UHL 10.156.253.176
  • Subnet - 255.255.255.128, Gateway - 10.156.253.129
  • DNS - 10.147.126.165 10.147.126.166 10.156.249.165 10.156.249.166 10.160.15.165 10.160.15.166
  • Hardware : Virtual machine
  • Software : Ubuntu 12.04.1 LTS
    • Packages: pwgen php5 php5-mysql php5-ldap php5-sybase php5-gd php5-curl php5-gmp php-pear php5-dev subversion mysql-client-core-5.5 git
  • Services
    • Testing version of CiviCRM

uhlbriccsapp05

  • Hostname : uhlbriccsapp05.xuhl-tr.nhs.uk
  • Location : UHL 10.156.253.177
  • Subnet - 255.255.255.128, Gateway - 10.156.253.129
  • DNS - 10.147.126.165 10.147.126.166 10.156.249.165 10.156.249.166 10.160.15.165 10.160.15.166
  • Hardware : Virtual machine
  • Software : Ubuntu 12.04.1 LTS
    • Packages: pwgen php5 php5-mysql php5-ldap php5-sybase php5-gd php5-curl php5-gmp php-pear php5-dev subversion mysql-client-core-5.5 git
  • Services
    • Live version of CiviCRM

uhlbriccsEXT01

  • Hostname : uhlbriccsext01.xuhl-tr.nhs.uk
  • Location : UHL 10.156.253.175
  • Subnet - 255.255.255.128, Gateway - 10.156.254.129
  • DNS - 10.147.126.165 10.147.126.166 10.156.249.165 10.156.249.166 10.160.15.165 10.160.15.166
  • Hardware : Virtual machine
  • Software : Ubuntu 12.04.1 LTS
    • Packages: sun-java6-bin sun-java6-jdk maven2 perl openssl apache2 php webmin.deb
  • Services
    • Onyx and dependencies, NOT including the pmi lookup, to service external recruitment. Deployed on tomcat over SSL.
    • apache2 and REDCap for external use. Apache is configured to use https (port 443 not port 80), and uses a UHL wildcard certificate for *.uhl-tr.nhs.uk provided by Comodo. The certificate is due to expire in March 2016.

JLMVLX1

  • Hostname : JLMVLX1.xuhl-tr.nhs.uk
  • Location : UHL 10.161.54.246
  • Subnet - 255.255.255.128, Gateway - 10.161.54.129
  • DNS - Managed by the host laptop
  • Hardware : Laptop running VMWare Player
  • OS : Ubuntu 12.04.1 LTS
    • Packages: phpmyadmin mysql perl openssl webmin.deb open-vm-tools open-vm-toolbox
  • Services
    • REDCap for out of hospital use

4MMVLX1

  • Hostname : 4MMVLX1.xuhl-tr.nhs.uk
  • Location : UHL 10.161.54.247
  • Subnet - 255.255.255.128, Gateway - 10.161.54.129
  • DNS - Managed by the host laptop
  • Hardware : Laptop running VMWare Player
  • OS : Ubuntu 12.04.1 LTS
    • Packages: phpmyadmin mysql perl openssl webmin.deb open-vm-tools open-vm-toolbox
  • Services
    • REDCap for out of hospital use

Upgrade to Ubuntu 14.4.1 LTS

On Monday 29 December 2014, we began the process of upgrading servers to Ubuntu 14.4.1 LTS - using 'do-release-upgrade' from the initial 12.04.1 LTS.

  • uhlbriccsdbdev - ssh didn't come back up on restart. Used the ISO of boot-repair-disk to boot VM and re-install Grub.
  • uhlbriccsdev - also needed boot-repair-disk to fix Grub.
  • uhlbriccsapp02 - worked first time after upgrade.
  • uhlbriccsapp03 - worked first time after upgrade.
  • uhlbriccsext01 - seemed to hang during upgrade, but worked fine after reboot.
  • uhlbriccsapp04 - worked first time after upgrade.
  • uhlbriccsapp05 - worked first time after upgrade.
  • uhlbriccsapp01 - also needed boot-repair-disk to fix Grub. Different error though. "Failed to boot default entries" and "too many titles for menuentry". Worked on reboot.
  • uhlbriccsdb01 - also needed boot-repair-disk to fix Grub. Different error though. "Failed to boot default entries" and "too many titles for menuentry". Worked on reboot.

The upgrade seemed to cause a problem with the /etc/alias files on some of the servers, which failed to send the logwatch daily emails to an external email address because root was no longer aliased. Was it removed as a security measure?

New config options: Disable SSH for root? Yes

University systems: LAMP Infrastructure

For support issues email rcs.support@….

All systems on the University side should eventually run on LAMP insfrastructure. The systems are VM's provided by RCS.

In the case of the three CaTissue VMs, where the 'alias' is accessed over https, it is actually an alias for the LAMP Load Balancer run by RCS. The load balancer passes the requests to apache on the app server on port 80, and apache uses mod_proxy and mod_proxy_http or mod_proxy_ajp to pass the request to JBoss on 8080. JBoss works natively.

See ​https://wiki.lamp.le.ac.uk/lampdoc/index.php/Managing_Services for instructions on setting apache and mysql to autostart as services on LAMP machines. This should be done for them all.

CaTissue DEV / LAMP 15

The CaTissue DEV server is a LAMP 2 instance.

  • Control interface: 143.210.56.57 login15.lamp.le.ac.uk
  • Public interface: 143.210.56.58 www15.lamp.le.ac.uk
  • Alias: catissue-dev.lcbru.le.ac.uk
  • 2GB RAM
  • 8GB in /local
  • 5GB in /db
  • Integral MySQL server, apache, JBoss, CaTissue and Sampler app

CaTissue LIVE / LAMP 50

The CaTissue LIVE server is a LAMP 2 instance.

  • Control interface: 143.210.56.77 login50.lamp.le.ac.uk
  • Public interface: 143.210.56.78 www50.lamp.le.ac.uk
  • Alias: catissue-live.lcbru.le.ac.uk
  • 4GB RAM
  • 8GB in /local
  • 8GB in /db
  • Integral MySQL server, apache, JBoss, CaTissue and Sampler app

CaTissue TRAINING / LAMP 16

The CaTissue TRAINING server is a LAMP 2 instance.

  • Control interface: 143.210.56.55 login16.lamp.le.ac.uk
  • Public interface: 143.210.56.56 www16.lamp.le.ac.uk
  • Alias: catissue-trn.lcbru.le.ac.uk
  • 2GB RAM
  • 8GB in /local
  • 5GB in /db
  • Integral MySQL server, apache, JBoss, CaTissue and Sampler app

LCBRU Trac

New, LAMP 2 instance (upgraded 24 February 2014):

  • Control interface: 143.210.56.207 login51.lamp.le.ac.uk
  • Public interface: 143.210.56.208 www51.lamp.le.ac.uk
  • Alias: lcbru-trac.rcs.le.ac.uk
  • Alias: trac.lcbru.le.ac.uk (/local/maven/repo)
  • Integral PostgreSQL server, apache
  • 512MB RAM
  • 8GB in /local

LCBRU maven and data repositories

New, LAMP 2 instance (upgraded 24 February 2014):

  • Control interface: 143.210.56.209 login52.lamp.le.ac.uk
  • Public interface: 143.210.56.210 www52.lamp.le.ac.uk
  • Alias: maven.lcbru.le.ac.uk (/local/maven/repo)
  • Alias: data.lcbru.le.ac.uk (/local/data/repo)
  • Alias: lcbru-maven.rcs.le.ac.uk (/local/maven/repo)
  • Alias: lcbru-data.rcs.le.ac.uk (/local/data/repo)
  • Integral MySQL server, apache
  • 512MB RAM
  • 8GB in /local

SCAD project web server

  • 512MB RAM
  • 8GB in /local
  • Local MySQL instance - 1GB in /db

REDCap public-facing survey server

  • 512MB RAM
  • 8GB in /local
  • Local MySQL instance - 1GB in /db

DataShield Control Panel

  • 512MB RAM
  • 8GB in /local
  • Local MySQL instance - 1GB in /db

DataShield Opal Data Source 1

  • 512MB RAM
  • 8GB in /local
  • Local MySQL instance - 1GB in /db

DataShield Opal Data Source 2

  • 512MB RAM
  • 8GB in /local
  • Local MySQL instance - 1GB in /db

University systems: non-LAMP

We have a storage server installed in the CRC which is provisioned by the UOL IT Services, but with us having exclusive use of it.

Mark Penny (IT Services) says:


fsp-glenfield1.uol.le.ac.uk\shared-data

The folder structure is as follows to allow flexibility if the storage needs to be expanded in the future.

1: The UNC path to the server share.

2: The root folders below the share. These are Read Only and cannot be changed. No additional files or folders can be added at this level.

If you try to add a file/folder or change the existing folders you will receive this message. This is perfectly normal and is because an action that is not permitted has been attempted.

3: The Data folder at the next level

4: Files and folders can be created, modified and deleted here.

This design of folder and associated ACLs allows further storage to be added in the future without having to apply new ACLS to the entire file system. Any new storage will go in as a new “Data(/n/) folder, e.g. Data3 and so on.

I have added your username to the group to allow access. Please log a call with the help desk specifying the user name and requesting that it be added to this AD group:

RAM-FSP-Glenfield-Data1.

Amir Hakimi will also have full access to the share. To begin with, Don Jones and Amir will use 'Data1' and do some test transfers of data so that we can evaluate the service.

Pre-LAMP University systems : OBSOLETE

The following were systems on the University side which were subsequently moved to the LAMP infrastructure. The systems were provided by RCS.

briccs-1

  • Hostname : briccs-1.rcs.le.ac.uk
  • Location : RCS
  • Hardware : ?
  • Software : OpenSuse
  • VMs provisioned:

briccs-4.rcs.le.ac.uk 02:01:8F:D2:AA:8C 143.210.170.140
briccs-5.rcs.le.ac.uk 02:01:8F:D2:AA:8D 143.210.170.141
briccs-6.rcs.le.ac.uk 02:01:8F:D2:AA:8E 143.210.170.142
briccs-7.rcs.le.ac.uk 02:01:8F:D2:AA:8F 143.210.170.143

Settings

IP mask : 255.255.255.0 Gateway : 143.210.170.1 DNS one : 143.210.12.152 DNS two : 143.210.12.154

VBoxManage man page: ​http://www.virtualbox.org/manual/ch08.html

Note: when rebuilding a VM, the ssh key will change. On local machines use "ssh-keygen -R hostname" and "ssh-keygen -R IP" to remove them and then reconnect.

briccs-2

  • Hostname : briccs-2.rcs.le.ac.uk
  • Location : RCS
  • Hardware : VirtualBox virtual machine
  • Software : OpenSuse
  • Services
    • NONE
    • Was: Subversion (svn) repository - but this has now been transferred to the University RCS SVN service.

briccs-3

  • Hostname : briccs-3.rcs.le.ac.uk
  • Location : RCS
  • Hardware : VirtualBox virtual machine
  • Software : OpenSuse

BRICCS Server Security

The following should be considered the standard specification for BRICCS Server Security (applies to the Hospital servers, where we have root access):

ClamAV: Put the following in cron.weekly and chmod 755: 

#!\bin\bash
clamscan -ri --exclude-dir=^/sys\|^/proc\|^/dev / | mail -s "ClamAV Scan Results for `date +%D`" email@email.domain

fail2ban

logwatch

  • Note: By default logwatch mails daily reports to root@localhost - this probably isn't what we want. Configure /usr/share/logwatch/default.conf/logwatch.conf and alter the target address to someone who gives a damn.

NagiOS

chkrootkit

Snort

What else?

Note: See TracWiki for help on using the wiki.