wiki:ProductieStappen

Version 7 (modified by mschenk, 10 years ago) (diff)

These are the systems involved in the website:

dex-db1

This physical machine holds the master database-server

MySQL master

The master database is installed directly on dex-db1. Configuration is done by the ICT department's database support.

dex-app1

This physical machine runs the front-end webserver, one application server, and a slave-database server. All these component are run from docker.

Full docker-configurations (using fig) can be found here: services/trunk/production/dex-app1

MySQL-Slave

MySQL-Slave provides a read-only database on each application server. This database is only present for performance reasons.

Slave mysql-servers are run as docker instances.

Slaves need to be configured for first time use using the script services/trunk/production/dex-app1/mysql/etc/initialcopy.sh. This script is available from within the docker instance through /etc/mysql/initialcopy.sh

Input for this script is the hostname for the master-server, and passwords for an admin user to retrieve the full database from the master and a replication user to replicate the database.

Error: Macro IncludeSource(services/trunk/production/dex-app1/mysql/etc/initialcopy.sh,start=3,end=8) failed
No node services/trunk/production/dex-app1/mysql/etc/initialcopy.sh at revision 1879

The script services/trunk/production/dex-app1/mysql/etc/renewcopy.sh does almost the same as initialcopy, it only uses authentication in a different fashion. This script can be used when a slave-database has lost synchronization with the master for whatever reason.

NGINX

Nginx provides a web-server that is used to serve static content and forward requests to the uwsgi servers running the actual website application

Full nginx configuration can be found in services/trunk/production/dex-app1/nginx

UWSGI

This container runs the actual website code. The current website code needs to be installed in /var/o2eappserver/fig/website. This directory is available as /code inside the docker image. (Further paths in this section will be specific to the docker instance)

The website as checked out from svn will not work properly due to missing settings. In order to create a working setup a settings_local.py file needs to be installed in /code/website/conf. The file website/trunk/website/conf/production/settings_live.py provides a template for this file

Migration

When migration's are run (using Django south) the system will mess up in a master-slave configuration. This is due to bugs in the south multi-db implementation. In order to circumvent these problems a number of temporary changes need to made in the /code/website/conf/settings_local.py. Specifically, the following entries need to be commented out:

Error: Macro IncludeSource(website/trunk/website/conf/production/settings_live.py,start=10,end=10) failed
No node website/trunk/website/conf/production/settings_live.py at revision 1879
Error: Macro IncludeSource(website/trunk/website/conf/production/settings_live.py,start=25,end=35) failed
No node website/trunk/website/conf/production/settings_live.py at revision 1879
Error: Macro IncludeSource(website/trunk/website/conf/production/settings_live.py,start=38,end=40) failed
No node website/trunk/website/conf/production/settings_live.py at revision 1879

Initial start

Static content must be created first time the application is deployed. This is done in the docker instance using

% /code/manage.py collectstatic

Media server

Media (pictures etc.) are not served from the application servers. Instead this content is served from a media server. The application is configured to write all uploaded media (uploaded through the admin interface that is) to the media-server. When rendering pages that contain these media files the url is changed to point to the public part of the media server.

The django plugin that is used to create this behaviour is called django-resto (see https://django-resto.readthedocs.org/en/latest/)

In the config file the following settings are important:

Error: Macro IncludeSource(website/trunk/website/conf/production/settings_live.py,start=65,end=67) failed
No node website/trunk/website/conf/production/settings_live.py at revision 1879

The RESTO_MEDIA_HOSTS is an array of media servers that need to hold the data.

the MEDIA_URL setting is prepended to the rendered locations of media-files. This needs to be a publicly available address.

SAML Single sign on

Signing in on the website happens through a separate sign-on-server. This server needs to implement the SAML2 protocol. The website uses the django-plugin "djangosaml2" which uses pysaml2 internally.

There is a number of important settings to be made for SAML to work properly.

SAML_CONFIG

Error: Macro IncludeSource(website/trunk/website/conf/production/settings_live.py,start=95,end=171) failed
No node website/trunk/website/conf/production/settings_live.py at revision 1879

In this structure there is a number of important parameters:

  • xmlsec_binary: the location of the xmlsec program that is used internally. This should usually be /usr/bin/xmlssec1

Error: Macro IncludeSource(website/trunk/website/conf/production/settings_live.py,start=69,end=181) failed
No node website/trunk/website/conf/production/settings_live.py at revision 1879

dex-content1

NGINX

The media-serving capabilities are implemented entirely in the built-in webdav functionality of nginx. The only necessary configuration that is necessary is registration of the IP-addresses of the application servers. These IP-addresses are allowed to do write operations on the media-server. A template for the configuration file is given in services/trunk/production/dex-content1/config/conf.d/webdav.conf. The editable part is the access-list:

Error: Macro IncludeSource(services/trunk/production/dex-content1/config/conf.d/webdav.conf,start=21,end=26) failed
No node services/trunk/production/dex-content1/config/conf.d/webdav.conf at revision 1879

Attachments (1)

Download all attachments as: .zip