
Installing the UCSC Genome browser

This file is from:
  http://genome-source.soe.ucsc.edu/gitlist/kent.git/blob/master/src/product/README.install

Scripts to perform all of the functions below can be found in
the directory here: ./scripts/
Please see the ./scripts/README file for more information.

1.  Confirm the following:
	a. Apache WEB server is installed and working
		http://localhost/
	    Provides the Apache default home page from your machine
	    NOTE: The browser static html WEB pages require the Apache
	    XBitHack option to be enabled to allow:
	      <!--#include ... --> statements to function.
            Add 'Options +Includes' for your html directory, your
            httpd.conf file entry looks like:
XBitHack on
<Directory /usr/local/apache/htdocs>
   Options +Includes
</Directory>

	    You can test your Apache cgi-bin/ directory with the script:
		scripts/printEnv.pl

	b. MySQL database is installed and working
		$ mysql -u browser -pgenome -e 'show tables;' mysql
	    MySQL can be run from the command line, and
	    the tables from the database mysql can be displayed.
	    See also: README.mysql.setup
        c. MySQL development package is installed (mysql-devel)
	    The directory: /usr/include/mysql/ has the mysql .h files
	    And the library: /usr/lib/mysql/libmysqlclient.a exists
	    (your exact pathnames may vary depending upon your installation)

2.  Set MySQL database access permissions.  The examples mentioned
	in the README.mysql.setup instructions will allow this
	setup to function as described here.

	To setup the example user accounts as mentioned in these
	instructions, run the script:

	ex.MySQLUserPerms.sh

3.  Find the location of your Apache WEB server DocumentRoot
	and cgi-bin directory.
	Typical locations are:  /var/www and /usr/local/apache
		/var/www/html
		/var/www/cgi-bin
	The directory where these are located is referred to as
	WEBROOT in this documentation:
		WEBROOT=/var/www
		export WEBROOT
	The browser WEB pages and cgi-bin binaries expect these
	two directories to be next to each other in ${WEBROOT}
	since referrals in html are often: "../cgi-bin"

	The browser should function even if WEBROOT is in a different
	directory from the primary Apache web root.  In this case,
	the three directories: html cgi-bin and trash should be
	at the same level in this other WEBROOT.  For example:
		/some/other/directory/path/html/
		/some/other/directory/path/cgi-bin/
		/some/other/directory/path/trash/
	Symlinks to the trash directory should exist from the html
	directory.  As so:
		/some/other/directory/path/html/trash -> ../trash
	The actual trash directory can be somewhere else.  If it is
	not in your Apache /var/www/trash/ directory, then create
	that symlink as well as the html/trash symlink.  For example
	/var/www/trash -> /some/other/directory/trash
	/var/www/html/trash -> /some/other/directory/trash

4.  Create html, cgi-bin and trash directories:
	mkdir ${WEBROOT}/html
	mkdir ${WEBROOT}/cgi-bin
	chmod 755 ${WEBROOT}/cgi-bin
	(this chmod 755 will prevent suexec failures that are indicated
	    by "Premature end of script headers" errors in the Apache
	    error_log.  Your cgi binaries should also be 755 permissions.)

	mkdir ${WEBROOT}/trash
	chmod 777 ${WEBROOT}/trash
	ln -s ${WEBROOT}/trash  ${WEBROOT}/html/trash

	The browser creates .png (and other) files in the trash directory.
	The 'chmod 777' allows the Apache WEB server to write into
	that directory.

	A cron job should be set to periodically clean the files in trash.
	See also, the two scripts here:
		scripts/trashCleanMonitor.csh
		scripts/trashCleaner.csh

5.  Download static WEB page content:
	See also: scripts/updateHtml.sh

6.  Copy CGI binaries:
	rsync -avP rsync://hgdownload.soe.ucsc.edu/cgi-bin/ ${WEBROOT}/cgi-bin/
	This set of binaries are for x86_64 types of Linux machines.
	If you need to instead build binaries for your platform,
	follow the instructions in: README.building.source
	See also: scripts/kentSrcUpdate.sh

7.  Create hgcentral database and tables.  This is the primary gateway
	database that allows the browser to find specific organism
	databases.  See also: scripts/fetchHgCentral.sh to fetch
        a current copy of hgcentral.sql

	mysql -u browser -pgenome -e "create database hgcentral;"
	mysql -u browser -pgenome hgcentral < hgcentral.sql

	Please note, it is possible to create alternative hgcentral
	databases.  For example, for test purposes.  In this
	case use a unique name for the hgcentral database, such
	as "hgcentraltest", and it can be specified in the hg.conf
	file as mentioned in the next step.  To create a second copy
	of the hgcentral database:

	mysql -u browser -pgenome -e "create database hgcentraltest;"
	mysql -u browser -pgenome hgcentraltest < hgcentral.sql

8.  Create the hg.conf file in ${WEBROOT}/cgi-bin/hg.conf
	to allow the CGI binaries to find the hgcentral database

	Use the file here: ex.hg.conf
	as the beginning template for your system:

	Copy the sample hg.conf:
	cp ex.hg.conf ${WEBROOT}/cgi-bin/hg.conf

	Please edit this hg.conf file and set any parameters required
	for your installation.  Use the comments in that file as your guide.

	Browser developers will want a copy of this file in
	their home directory with mode 600 and named:
		~/.hg.conf
	These copies may have different db.user specification
	to allow developers write access to the database.
	See also: README.mysql.setup

10.  Load databases of interest.  See also: README.QuickStart
	scripts/activeDbList.sh
	scripts/minimal.db.list.txt
	scripts/loadDb.sh
	And discussion in scripts/README about whether you can use directly
	the MySQL binary database files, or if you need to download goldenPath
	database text dumps and load them into the database.

	An alternative to loading the database tables from text files,
	is to directly rsync the MySQL tables themselves and place them
	in your MySQL /var/ directory.  These tables are much larger
	than the text files due to the sizes of indexes created during a
	table load, but it can save a lot of time since the data loading
	step is quite compute intensive.  A typical rsync command for an
	entire database (e.g. ce4) would be something like:
	rsync -avP --delete --max-delete=20 \
		rsync://hgdownload.soe.ucsc.edu/mysql/ce4/ \
		/var/lib/mysql/ce4/

11.  Download extra databases to work with a full genome assembly
     such as human/hg38: hgFixed go140213 proteins140122 sp140122
     Construct symlinks in your MySQL data directory to use database
     names: go proteome uniProt for these database directories:

$ ls -og proteome go uniProt
lrwxrwxrwx 1  8 Feb 26 11:39 go -> go140213
lrwxrwxrwx 1 14 Mar 27 12:01 proteome -> proteins140122
lrwxrwxrwx 1  8 Mar 27 12:01 uniProt -> sp140122

$ ls -ld go140213 proteins140122 sp140122
drwx------ 2 mysql mysql 4096 Feb 26 10:57 go140213
drwx------ 2 mysql mysql 4096 Aug 19 08:08 proteins140122
drwx------ 2 mysql mysql 4096 Mar 26 13:01 sp140122

     These file names are data stamped YYMMDD to indicate changes
     over time as they are updated with new builds of the UCSC gene track.
     When a new UCSC gene track is released, fetch new databases and
     change the symlink.

12.  Copy the gbdb data to /gbdb - See also:
		scripts/fetchFullGbdb.sh
		scripts/fetchMinimalGbdb.sh

13.  The browser should now appear at the URL:
	http://localhost/

	Check your Apache error_log file for hints to solving problems.

14.  blat server setup
	The blatServers table in the database hgcentral needs to
	have a fully qualified host name specified in the 'host' column.
	Educational and non-profit institutions are allowed to use
	blat free of charge.  Commercial installations of the browser
	require a license for blat.  See also: http://kentinformatics.com/index.html
	and: http://genome.ucsc.edu/license/
	In the source tree: src/gfServer/README.blat

15.  Useful links:

	http://genomewiki.ucsc.edu/index.php/Category:Mirror_Site_FAQ

	There are numerous README files in the source tree on
	a variety of specific subjects, e.g.:
		./src/README
		./src/product/README.*
		./src/hg/makeDb/trackDB/README
		./src/hg/makeDb/doc/make*.txt

16. Apache configuration:

    To lock down your trash directory from scanning via "indexes"
    enter the following in your httpd.conf:

<Directory "/var/www/html/trash">
    Options MultiViews
    AllowOverride None
    Order allow,deny
    Allow from all
</Directory>

    The specified directory name is your apache: DocumentRoot/trash
    e.g. /usr/local/apache/htdocs/trash

====================================================================
Information last updated: 19 August 2014
====================================================================
