You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 37 Next »

Work in progress

This guide is written for Ubuntu 10.04.

For neo4j-embedded:

sudo apt-get install nginx git-core python-virtualenv postfix openjdk-6-jdk postgresql build-essential

For neo4j-rest-client:

sudo apt-get install apache2 libapache2-mod-wsgi git-core python-virtualenv postfix openjdk-6-jre-headless postgresql python-psycopg2 libpq-dev python-dev

For both neo4j versions:
Make a virtual python environment.

cd /opt/python_environments/
sudo virtualenv --no-site-packages norduni

Install needed python modules.

sudo su
source python_environments/norduni/bin/activate
pip install django ipaddr argparse pexpect markdown psycopg2 lucene-querybuilder gunicorn

Add the following python modules if you are running the Neo4j stand alone server:

pip install httplib2

Get the NORDUni files:
For neo4j-embedded:

cd /var/opt/
git clone git://git.nordu.net/norduni.git

For neo4j-rest-client:

cd /var/opt/
git clone git://git.nordu.net/norduni.git
git checkout operation

For both neo4j versions:
Get the NERDS files:

mkdir /opt/norduni/tools/
cd /opt/norduni/tools/
git clone https://github.com/fredrikt/nerds.git

Symlink admin media and the virtual python environment to niweb.

cd /var/opt/norduni/scr/niweb/
ln -s /var/opt/python_environments/norduni/ env
cd static
ln -s ../env/lib/python2.6/site-packages/django/contrib/admin/media/ admin

Set password for database user and create a new database

sudo -u postgres psql postgres
\password postgres
Write password
Write password again
Ctrl+D
sudo -u postgres createdb norduni

Change the django settings.

# Django settings for niweb project
cd /opt/norduni/scr/niweb/
cp generic_settings.py settings.py
vi settings.py

Change at least the following settings.

NIWEB_ROOT = '/opt/norduni/src/niweb/'
NIWEB_MEDIA_URL = '/static/'
# Database settings
DATABASES = {
    'default': {
        'ENGINE': 'django.db.backends.postgresql_psycopg2',
        'NAME': 'norduni',
        'USER': 'postgres',
        'PASSWORD': 'secret',
        'HOST': 'localhost'
    }
}
# Neo4j settings
NEO4J_RESOURCE_URI = '/opt/norduni/dependencies/neo4jdb/'

Comment out the static media url in /opt/norduni/src/niweb/urls.py.

urls.py
# Static serve
    #(r'^site_media/(?P<path>.*)$', 'django.views.static.serve',
    #    {'document_root': settings.STATIC_DEV_MEDIA}),

Create a gunicorn start file.

/opt/norduni/src/start_noclook.sh
#!/bin/bash
set -e
export JAVA_HOME=/usr/lib/jvm/java-6-openjdk/
LOGFILE=/var/log/ni/noclook.log
LOGDIR=$(dirname $LOGFILE)
NUM_WORKERS=1
# user/group to run as
USER=user
GROUP=group
cd /opt/norduni/src/niweb
source env/bin/activate
test -d $LOGDIR || mkdir -p $LOGDIR
exec env/bin/gunicorn_django -w $NUM_WORKERS \
 --user=$USER --group=$GROUP --log-level=debug \
 --log-file=$LOGFILE 2>>$LOGFILE

Configure nginx.

/etc/nginx/sites-available/default
server {
    listen 80;
    root /opt/norduni/src/niweb;
    server_name ni.example.net;
    access_log /var/log/ni/noclook-access.log;
    error_log /var/log/ni/noclook-error.log;

    location /static/ {
        root   /opt/norduni/src/niweb/;
        autoindex on;
        access_log   off;
        expires      30d; 
    }

    location / {
        proxy_pass_header Server;
        proxy_set_header Host $http_host;
        proxy_redirect off;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Scheme $scheme;
        proxy_connect_timeout 10;
        proxy_read_timeout 10;
        proxy_pass http://localhost:8000/;
    }   
}

Install supervisord and set up the following start script.

easy_install supervisor
echo_supervisord_conf > /etc/supervisord.conf
/etc/init/supervisord.confdescription "supervisord"
start on runlevel [2345]stop on runlevel [!2345]
respawn
exec /usr/local/bin/supervisord --nodaemon --configuration /etc/supervisord.conf {code

Add the noclook start script to the supervisor configuration.

[program:noclook]
directory = /opt/norduni/src/niweb/
user = user
command = /opt/norduni/src/start_noclook.sh
stdout_logfile = /var/log/ni/supervisor_logfile.log
stderr_logfile = /var/log/ni/supervisor_err_logfile.log


Set up a GIT repository for the JSON files produced by the NERDS producers and make a clone in /var/opt/. We have a GIT repository called nistore
and we have made a clone found at /var/opt/nistore/.

Produce JSON files with the producers you want.

Consume the recently produces JSON files with the NORDUni consume scripts.

h2. Install Neo4j graph database

h3. Neo4j 1.5 embedded with Python bindings (recommended)
Install JPype and Neo4j-embedded.
Download jpype. (http://sourceforge.net/projects/jpype/files/)

pip install neo4j-embedded
export JAVA_HOME=/usr/lib/jvm/java-6-openjdk/jre/
pip install /path/to/jpype-version.zip


h3. Neo4j 1.3 REST stand alone server
Download neo4j-1.3 from http://neo4j.org.
Extract the file to, for example, /var/opt/norduni/dependencies/.

cd /var/opt/norduni/dependencies/
tar xfz neo4j-1.3.tar.gz


Copy the startup script from /var/opt/norduni/scr/scripts/neo4j/ to /etc/init.d/ and then make neo4j start on boot.

update-rc.d /et/init.d/neo4j defaults


You need to shut down port 7474 from anything except localhost.

iptables -A INPUT -p tcp --dport 7474 -s localhost -j ACCEPT
iptables -A INPUT -p tcp --dport 7474 -j DROP


Save the rules and add the following line to your interface configuration so you will restore the rules on a reboot.

bash -c "iptables-save > /etc/iptables.rules"

vi /etc/network/interfaces

iface eth0 inet static
pre-up iptables-restore < /etc/iptables.rules



h3. Old Apache stuff
Configure the web server to play nice with neo4j-embedded. Install apache2-mpm-prefork libapache2-mod-wsgi.

Create a file name niweb in /etc/apache2/conf.d/.

WSGIPythonPath "/var/opt/norduni/scr:/var/opt/norduni/scr/niweb"
WSGIPythonHome "/opt/python_environments/norduni"
WSGIDaemonProcess processes=1 threads=25 # For neo4j-embedded
WSGIApplicationGroup %

Unknown macro: {GLOBAL}

# For neo4j-embedded
WSGIRestrictStdin Off
WSGIRestrictStdout Off
WSGIScriptAlias / /var/opt/norduni/niweb.wsgi


Add niweb path to niweb.wsgi

sys.path.append('/opt/norduni/src/niweb')


/etc/apache2.conf

<IfModule mpm_prefork_module>
StartServers 1
ServerLimit 1
MinSpareServers 5
MaxSpareServers 10
MaxClients 1
MaxRequestsPerChild 0
</IfModule>


Add the following lines to /etc/apache2/sites-available/default-ssl

<LocationMatch "Shibboleth.sso">
SetHandler None
</LocationMatch>

<Location "/accounts/login-federated/">
AuthType shibboleth
ShibRequireSession On
require valid-user
RequestHeader set X_REMOTE_USER %

Unknown macro: {eppn}

e
RequestHeader set EPPN %

e
RequestHeader set GIVENNAME %

Unknown macro: {givenName}

e
RequestHeader set SN %

Unknown macro: {sn}

e
RequestHeader set MAIL %

Unknown macro: {mail}

e
RequestHeader set AFFILIATION %

Unknown macro: {affiliation}

e
RequestHeader set ENTITLEMENT %

Unknown macro: {entitlement}

e
</Location>

Alias /admin_media /var/opt/norduni/scr/niweb/static/admin/
Alias /site_media /var/opt/norduni/scr/niweb/static/
<LocationMatch "^/(admin_media|site_media)">
Options FollowSymLinks
Order allow,deny
allow from all
SetHandler None
</LocationMatch>


  • No labels