HowTo: Services & Director HA config for Linux (OpenSuse)

Post any question you may have in regards to GoAnywhere Services and let our talented support staff and other users assist you.
2 posts Page 1 of 1

bothunbr

Posts: 12
Joined: Sun Mar 31, 2013 12:13 pm

Post by bothunbr » Wed Jan 08, 2014 11:57 pm
I'm going to attempt to document my HA configuration for Services and Director as I build, test and change it.
GAGluster.png
GA Cluster
GAGluster.png (34.79 KiB) Viewed 7506 times
Background: I'm going to use all GPL software to create this GoAnywhere HA system, including the Hypervisor. I'm also going to install both GA Services and GA Director into a chroot jail to protect the OS (mostly from director users/developers). The GA products with be contained to a single directory structure and will have no access to the rest of the operation system. I will use glusterfs (http://www.gluster.org/) as the clustered filesystem each gluster brick will run on the same GA server that is accessing it.

Equipment: I'm going to use two new Dell R620 servers with 128G of memory and disk allocated from a Complellent SAN. I'll also use a clustered mysql database.

You should be able to follow this document with some very basic hardware. For just following along a single VM host server with two VMs for the GoAnywhere cluster and 1 VM for a database should do the trick. We will need to create 2 separate linux installations with 2G of memory each, 40G of disk for OS, software and user data, two network cards with access to a public network and a private network. We will also need a third VM to act as our database server. You may use whatever database/OS combo is supported by goanywhere.

What I'm not going to cover: Hypervisor setup and configuration, database server setup, or networking. I may make reference to XenServer and MySQL during in this document but I will attempt to keep it to a minimum. Neither of these products are a requirement for following along. You could use VMware and DB2 or HyperV and M$ SQL Server. If you have enough bare metal hardware virtualization isn't even a requirement for this HA setup.

Software - Hypervisor: Opensource XenServer (http://www.xenserver.org/) Operating System: OpenSuse 13.1 64bit (http://software.opensuse.org/131/en)

OS installation
1. Download the opensuse 13.1 DVD from http://software.opensuse.org/131/en
2. Boot the CD and install the Operating System with the following options
Installation Mode Screen (http://activedoc.opensuse.org/sites/def ... mode_6.png) Uncheck "Automatic Configuration"
Desktop Selection Screen (http://activedoc.opensuse.org/sites/def ... type_6.png) Select "Other", "Minimal Server Selection (Text Mode)"
Suggested Partitioning Screen (http://activedoc.opensuse.org/sites/def ... ning_6.png) Select Create Partition Setup.. You will need three disk partitions
1. Swap 2G to 4G depending on how much memory is on your system. Format as swap
2. a root partition / Make this at least 10G. Format as ext4
3. a data partition This is for all your GA user data for both Services and Director so size appropriately. Format with XFS and mount in /export/gadata
Create New User Screen (http://activedoc.opensuse.org/sites/def ... user_6.png) Uncheck all the boxes and leave the fields blank, then click next. There will be some warning about not creating a user, just click OK and continue.
Installation Settings Screen: (http://activedoc.opensuse.org/sites/def ... mary_6.png) Disable the firewall and open ssh.

Note: at some point in the installation we drop out of the gui to a text based interface. This is because we chose not to install a windowing system. Don't get scared, just follow the prompts. The networking setup gets a bit weird with opensuse so make sure you check the top menus for DNS & routing setup.

Networking:
Setup eth0 so it has access to the internet, make your default route go out this interface as we will nee to download some packages from the OpenSuse repos.
Setup eth1 in a private network. This network should only have access to the other goanywhere server.
Note: You can follow this HowTo with only 1 interface. but I recommend separating the cluster traffic from the network.

Repeat the above configuration for server 2.

Bulk command insert with comments below. I'll make it pretty later...

# You can just copy and paste the below list of commands in your terminal if you are in a hurry.

zypper rm patterns-openSUSE-minimal_base-conflicts
zypper ar http://download.opensuse.org/repositori ... SUSE_13.1/ Security
zypper in findutils-locate jailkit glusterfs yast2-runlevel yast2-update yast2-online-update
mkdir -p /srv/jail/linoma/srv/linoma/services
mkdir -p /srv/jail/linoma/srv/linoma/director
jk_init -v -j /srv/jail/linoma basicshell editors extendedshell netutils
useradd -m gauser
chown -R gauser:users /srv/jail/linoma/srv/linoma
mkdir -p /srv/jail/linoma/usr/sbin
cp /usr/sbin/jk_lsh /srv/jail/linoma/usr/sbin/
jk_jailuser -m -j /srv/jail/linoma gauser
mkdir -p /usr/local/src/java
cd /usr/local/src/java
mv jdk1.6.0_45 /srv/jail/linoma/srv/linoma/
mkdir -p /srv/jail/linoma/usr/local/src/linoma/services
mkdir -p /srv/jail/linoma/usr/local/src/linoma/director
cp /usr/bin/basename /srv/jail/linoma/usr/bin/basename
cp /usr/bin/dirname /srv/jail/linoma/usr/bin/dirname
cp /usr/bin/uname /srv/jail/linoma/usr/bin/uname
cp /usr/bin/tty /srv/jail/linoma/usr/bin/tty
chmod 04755 /usr/sbin/jk_chrootsh
mkdir -p /srv/jail/linoma/proc
mount --bind /proc /srv/jail/linoma/proc
mkdir /srv/jail/linoma/tmp
chmod 777 /srv/jail/linoma/tmp
## Fix Timezone for jail (Pick your timezone. Mine is central)
cp /usr/share/zoneinfo/US/Central /srv/jail/linoma/etc/localtime

### Manual Processing starts here #####

# Download java from http://www.oracle.com/technetwork/java/ ... index.html
# I used "Java SE Development Kit 6u45" from the "Previous Releases" tab.
# Important!! Make sure you do not download the -rpm.bin file. The rpm installer will fail or even if you get it to work it will come back to haunt you (Trust me).
./jdk-6u45-linux-x64.bin
# Move the extracted java folder to your jail
mv jdk1.6.0_45 /srv/jail/linoma/srv/linoma

# Update the findutils DB
updatedb

Change /etc/passwd to:
gauser:x:1000:100::/srv/jail/linoma/./srv/linoma:/usr/sbin/jk_chrootsh

Change: /srv/jail/linoma/etc/passwd
gauser:x:1000:100::/srv/linoma:/bin/bash


#Copy installer to /srv/jail/linoma/usr/local/src/linoma/services
cp ?? /srv/jail/linoma/usr/local/src/linoma/services
#Make Executeable
chmod 755 /srv/jail/linoma/usr/local/src/linoma/services/*


## Enter Jail
su gauser
#Install Go Anywhere Services
cd /usr/local/src/linoma/services/
./gaservices3_4_0_linux_x64.sh

#Add This to: /srv/jail/linoma/srv/linoma/services/gaservices.sh
export JAVA_HOME=/srv/linoma/jdk1.6.0_45
export PATH=$PATH:$JAVA_HOME/bin
export LD_LIBRARY_PATH=/srv/linoma/jdk1.6.0_45/jre/lib/amd64/jli

Change /etc/passwd to:
gauser:x:1000:100::/srv/jail/linoma/./srv/linoma:/usr/sbin/jk_chrootsh

Change: /srv/jail/linoma/etc/passwd
gauser:x:1000:100::/srv/linoma:/usr/sbin/jk_lsh

######## Gluster Config ############
#attach the disk you want to use for userdata & clusterdata to the system
mkdir -p /export/gadata

## Format the partition with XFS and mount it in: /export/gadata ###

### Add static hostnames to the /etc/hosts file on each system for DNS on the second interface#
## This is the interface we want replication traffic to run on ##
10.36.136.116 ga-test-01.halldata.com ga-test-01
10.36.142.116 ga-test-01mgmt.halldata.com ga-test-01mgmt
10.36.142.117 ga-test-02mgmt.halldata.com ga-test-02mgmt
## And on the other server ####
10.36.136.117 ga-test-02.halldata.com ga-test-02
10.36.142.117 ga-test-02mgmt.halldata.com ga-test-02mgmt
10.36.142.116 ga-test-01mgmt.halldata.com ga-test-01mgmt

#### Use yast runlevel editor to start glusterd on both systems #####
yast
## System->System Services (Runlevel)
# Enable glusterd

## Test gluster, make sure it can connect from both systems ##
##From ga-test-01
gluster peer probe ga-test-02mgmt.halldata.com
##From ga-test-02
gluster peer probe ga-test-01mgmt.halldata.com

### From ga-test-01 create gluster volume ####
gluster volume create gadata replica 2 ga-test-01mgmt.halldata.com:/export/gadata ga-test-02mgmt.halldata.com:/export/gadata
gluster volume start gadata

mkdir -p /srv/jail/linoma/import/gadata

# Server 1
mount -t glusterfs -o log-level=WARNING,log-file=/var/log/gluster.log ga-test-01mgmt.halldata.com:gadata /srv/jail/linoma/import/gadata

# Server 2
mount -t glusterfs -o log-level=WARNING,log-file=/var/log/gluster.log ga-test-02mgmt.halldata.com:gadata /srv/jail/linoma/import/gadata

cd /srv/jail/linoma/srv/linoma/services/userdata
mkdir -p /srv/jail/linoma/import/gadata/services

## Server 1 only
mv documents /srv/jail/linoma/import/gadata/services/documents
mv emailtemplates /srv/jail/linoma/import/gadataservices/emailtemplates
mv keys /srv/jail/linoma/import/gadata/services/keys
mv packages /srv/jail/linoma/import/gadata/services/packages
mv webdocs /srv/jail/linoma/import/gadata/services/webdocs
mkdir /srv/jail/linoma/import/gadata/services/logs

### Server 1 & 2
cd /srv/jail/linoma/srv/linoma/services/userdata
rm -r documents
rm -r emailtemplates
rm -r keys
rm -r packages
rm -r webdocs


ln -s ../../../../import/gadata/services/documents documents
ln -s ../../../../import/gadata/services/emailtemplates emailtemplates
ln -s ../../../../import/gadata/services/keys keys
ln -s ../../../../import/gadata/services/packages packages
ln -s ../../../../import/gadata/services/webdocs webdocs
ln -s ../../../../import/gadata/services/logs logs

chown -R gadata:users /srv/jail/linoma/import/gadata/services

### Create MySQL Database & user. Use the GA migration wizard to move the database to an outside server
#
##To setup a cluster edit /srv/jail/linoma/srv/linoma/services/config/gaservices.xml on each system read the GA install guide for details





#### Director Cluster
# Server 1
cd /srv/jail/linoma/srv/linoma/director/

cd /srv/jail/linoma/srv/linoma/director/userdata
mkdir -p /srv/jail/linoma/import/gadata/director/logs
mv projects ../../../../import/gadata/director/
mv workspace ../../../../import/gadata/director/
mv documents ../../../../import/gadata/director/
mv monitors ../../../../import/gadata/director/
mv keys ../../../../import/gadata/director/
mv emailtemplates ../../../../import/gadata/director/

chown -R gadata:users /srv/jail/linoma/import/gadata/director

#### Server 1 & Server 2
cd /srv/jail/linoma/srv/linoma/director/userdata
rm -r projects
rm -r workspace
rm -r documents
rm -r monitors
rm -r keys
rm -r emailtemplates

ln -s ../../../../import/gadata/director/projects projects
ln -s ../../../../import/gadata/director/workspace workspace
ln -s ../../../../import/gadata/director/documents documents
ln -s ../../../../import/gadata/director/monitors monitors
ln -s ../../../../import/gadata/director/keys keys
ln -s ../../../../import/gadata/director/emailtemplates emailtemplates

#Start Director on server 1 !! After server 1 is complete start here again for Server 2
su gauser -c "director/goanywhere.sh start"

## Change the database from Derby to whatever DB system you use. See the director install guide ###

#Restart director after database change

su gauser -c "director/goanywhere.sh stop"
su gauser -c "director/goanywhere.sh start"

## Change the logs directory in the director web interface
# Logs > Log Manager
# /import/gadata/director/logs

# Repeat the steps to move the logs directory on server 2

bothunbr

Posts: 12
Joined: Sun Mar 31, 2013 12:13 pm

Post by bothunbr » Fri Jan 24, 2014 12:37 am
Save attached startup scripts in /etc/init.d on both systems in the cluster

#make them executeable
cd /etc/init.d
chmod 755 services
chmod 755 services

#start service
/etc/init.d/services start

#stop services
/etc/init.d/services stop

#start director
/etc/init.d/director start

#stop director
/etc/init.d/director stop
2 posts Page 1 of 1