User:Daveh/uespdeploy

The uespdeploy is a relatively simple script which is used to help deploy applications on the various UESP servers. For more detailed and up to date information look at the following files directly on the Google Code repository:

uespdeploy - The uespdeploy script itself.
example.deploy - Contains a detailed description of the deploy file.

Why UespDeploy?Edit

At one point in the not too distant past the UESP was small enough to be hosted on one server. Installing and updating the small amount of custom source code on it was simple and straight forward with only one server administrator. Fast forward to the present and the UESP is now hosted on multiple servers with much more complex software architecture with several server admins. Manually having to deploy source changes to multiple servers is possible but awkward and prone to error.

The uespdeploy script was meant as a relatively simple way of simply installing (mostly just copying) source files onto the required servers along with some minimal setup.

The BasicsEdit

The uespdeploy script lives, along with a bunch of other scripts, in the /home/uesp/scripts/ path on all UESP servers. This should be automatically in your PATH but if it isn't you can edit your ~/.bash_profile to add this path yourself. When run the script looks for the uesp.deploy file in the current directory. This file contains the information needed to deploy the given application to one or more servers. Each application will have its own uesp.deploy file specific to its installation needs.

For example the typical deployment should just look like:

     cd /path/someapp
     uespdeploy

There are multiple command line options which uespdeploy accepts but are not generally required. For reference purposes all deployments are logged in a MySQL database which can be viewed using the uespdeploylog script.

The deployment itself is merely copying files from the source to one or more destinations. Files and directories that already exist on the destination will be overwritten which permits deploying over an existing installation. Files in the destination that don't exist in the source will be left alone (you can delete specific old files on the destination).

The uespdeploy ScriptEdit

Some typical command line arguments used by the uespdeploy script are shown below. Most of the time, however, you will just run the script with no arguments at all:

    uespdeploy

Note that any options specified on the command line will override the same option found in the deploy file. Basic script help can be shown via:

    uespdeploy -h

To show more deployment details you can increase the verbosity:

    uespdeploy -v
    uespdeploy -vv
    uespdeploy -vvv

You can limit the deployment to a single server by using the -l (localhost) or -s arguments which is useful during testing and debugging:

    uespdeploy -l
    uespdeploy -s content3
    uespdeploy -s content3.uesp.net

Note that this only works if the given server has a deployment destination in the deploy file.

You can override the destination(s) found in the deploy file with the -d option:

    uespdeploy -d /home/uesp/www/w/extensions/

You can similarly override the deployment's source path which is the current working directory by default (./):

    uespdeploy -s /home/dave/uesp.maps/

Likewise, the deployment file itself can be manually specified:

    uespdeploy -f /path/to/something.deploy

Backups of the existing destination files can be enabled (disabled by default) and their storage location specified (/tmp// by default):

    uespdeploy -b
    uespdeploy -b -B /home/backup/deploy/

The uesp.deploy FileEdit

The format of the deploy file is best described in the example.deploy file but a few basic examples can be shown.

The deploy file for the UESP scripts is basically just:

    #
    # Deploy file for the uesp.scripts
    #
    name = uesp.scripts
    repository = https://code.google.com/p/uesp.scripts/
    backup = false
    verbosity = 0
    
    ignore = *.deploy
    ignore = .hg*
    
    dest = content1.uesp.net:/home/uesp/scripts/
    dest = content2.uesp.net:/home/uesp/scripts/
    dest = content3.uesp.net:/home/uesp/scripts/
    dest = files1.uesp.net:/home/uesp/scripts/
    dest = db1.uesp.net:/home/uesp/scripts/
    dest = db2.uesp.net:/home/uesp/scripts/
    dest = squid1.uesp.net:/home/uesp/scripts/
    dest = squid2.uesp.net:/home/uesp/scripts/
    
    delete_dest = uespdeploy.py
    delete_dest = uespdeploylog.py

The only required variables are name and at least one dest. This deploy file simply translates to:

Copy all files (except *.deploy and .hg*) to all servers in the /home/uesp/scripts/ folder.
Delete the old files uespdeploy.py and uespdeploylog.py from each destination folder.

For other examples of deploy files see the various UESP applications:

Setup RequirementsEdit

There are a few things you need t

Have a shell access to the UESP servers with uespadmin access.
If write access is desired to the remote code repository you'll need a GMail e-mail address to be setup by the repository admin. Proper setup of hg will also be needed.
Setup ssh private/public keys to permit password-less access between UESP servers. If you don't do this you'll soon get very tired of having to enter your password multiple times during a deployment. Eventually there will be a script to help automate this but for now you can manually setup it up using the following:

Create a public/private key pair in the default locations on the UESP server (typically on content3) with no passphrase:

       ssh-keygen -t dsa -b 1024

For each server run the following commands (you will need your password to login):

       ssh server.uesp.net mkdir -p .ssh; chmod 700 .ssh
       cat .ssh/id_dsa.pub | ssh server.uesp.net 'cat >> .ssh/authorized_keys'
       ssh server.uesp.net chmod 640 .ssh/authorized_keys

Test logging into to a UESP server from another server. If it works you won't be prompted for a password:

       ssh server.uesp.net

Putting it All TogetherEdit

Combined with using the UESP's Code Repository we can now very easily deploy apps from scratch with just a few commands. For example, assuming your dev environment is setup correctly you can do:

    ssh user@content3.uesp.net
    cd ~
    hg clone https://code.google.com/p/uesp.wikimap/       # Create a copy of the map extension in your directory
    cd uesp.wikimap
    vi UespMap.php                                         # Make edits to the php file
    uespdeploy -l                                          # Deploy the extension to just the localhost (content3 in this case) for testing                           
    hg commit -m "fixed bug 123"                           # Commit the changes and update the remote repository
    hg push
    hg pull -u
    uespdeploy                                             # Deploy the code to all servers as given in the uesp.deploy file

If you don't have write access to the code repository you can still deploy code like:

    hg clone https://code.google.com/p/uesp.wikimap/
    cd uesp.wikimap
    uespdeploy

UESPWiki β