heckyel/mediagoblin
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Review and update the deploment docs for Debian 10 and CentOS 8 [#5593].

Browse Source 
Mostly a thorough test and cleanup. The significant change is moving the systemd
config into the deployment docs an the registration/security considerations to
production considerations docs.
This commit is contained in:
Ben Sturmfels 2020-04-29 02:19:16 +10:00
parent 16fbe85247
commit 62a565e13b
No known key found for this signature in database
GPG Key ID: 023C05E2C9C068F0
5 changed files with 317 additions and 332 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
Dockerfile-debian-python2-sqlite
View File

@ -54,7 +54,7 @@ RUN groupadd --system mediagoblin --gid 1024 && adduser www-data mediagoblin
USER www-data USER www-data
RUN git clone --depth=1 git://git.savannah.gnu.org/mediagoblin.git -b master . RUN git clone --depth=1 git://git.savannah.gnu.org/mediagoblin.git --branch master .
RUN ./bootstrap.sh RUN ./bootstrap.sh
RUN VIRTUALENV_FLAGS='--system-site-packages' ./configure --without-python3 RUN VIRTUALENV_FLAGS='--system-site-packages' ./configure --without-python3

2
Dockerfile-debian-python3-sqlite
View File

@ -138,7 +138,7 @@ USER www-data
# version, so that local changes are immediately available to the running # version, so that local changes are immediately available to the running
# container. Not as easy as it sounds. We have this working with docker-compose, # container. Not as easy as it sounds. We have this working with docker-compose,
# but still uses upstream MediaGoblin for the build. # but still uses upstream MediaGoblin for the build.
RUN git clone --depth=1 git://git.savannah.gnu.org/mediagoblin.git -b master . RUN git clone --depth=1 git://git.savannah.gnu.org/mediagoblin.git --branch master .
RUN ./bootstrap.sh RUN ./bootstrap.sh
RUN VIRTUALENV_FLAGS='--system-site-packages' ./configure RUN VIRTUALENV_FLAGS='--system-site-packages' ./configure

9
Dockerfile-fedora-python3-sqlite
View File

@ -4,18 +4,17 @@
FROM fedora:31 FROM fedora:31
RUN yum -y update RUN dnf -y install \
RUN yum -y install \
automake \ automake \
gcc \ gcc \
git \ git-core \
make \ make \
nodejs \ nodejs \
npm \ npm \
python3-devel \ python3-devel \
virtualenv virtualenv
RUN yum -y install \ RUN dnf -y install \
findutils \ findutils \
python3-alembic \ python3-alembic \
python3-celery \ python3-celery \
@ -70,7 +69,7 @@ RUN chown root:www-data /var/www
USER www-data USER www-data
RUN git clone --depth=1 git://git.savannah.gnu.org/mediagoblin.git -b master . RUN git clone --depth=1 git://git.savannah.gnu.org/mediagoblin.git --branch master .
RUN ./bootstrap.sh RUN ./bootstrap.sh
RUN VIRTUALENV_FLAGS='--system-site-packages' ./configure RUN VIRTUALENV_FLAGS='--system-site-packages' ./configure

469
docs/source/siteadmin/deploying.rst
View File

@ -1,6 +1,6 @@
.. MediaGoblin Documentation .. MediaGoblin Documentation
Written in 2011, 2012, 2013 by MediaGoblin contributors Written in 2011, 2012, 2013, 2020 by MediaGoblin contributors
To the extent possible under law, the author(s) have dedicated all To the extent possible under law, the author(s) have dedicated all
copyright and related and neighboring rights to this software to copyright and related and neighboring rights to this software to
@ -17,20 +17,20 @@
Deploying MediaGoblin Deploying MediaGoblin
===================== =====================
GNU MediaGoblin is fairly new, and so at the time of writing there aren't Following this deployment guide will take you step-by-step through
easy package-manager-friendly methods to install it. However, doing a basic setting up your own instance of MediaGoblin.
install isn't too complex in and of itself. Following this deployment guide
will take you step-by-step through setting up your own instance of MediaGoblin.
Of course, when it comes to setting up web applications like MediaGoblin, MediaGoblin most likely isn't yet available from your operating
there's an almost infinite way to deploy things, so for now, we'll keep it system's package manage, however, a basic install isn't too complex in
simple with some assumptions. We recommend a setup that combines MediaGoblin + and of itself. We recommend a setup that combines
virtualenv + Waitress + Nginx on a .deb- or .rpm-based GNU/Linux distro. MediaGoblin, virtualenv, Waitress and Nginx on a .deb or .rpm-based
GNU/Linux distribution.
Other deployment options (e.g., deploying on FreeBSD, Arch Linux, using Experts may of course choose other deployment options, including
Apache, etc.) are possible, though! If you'd prefer a different deployment Apache. See our `Deployment wiki page
approach, see our <http://wiki.mediagoblin.org/Deployment>`_ for for more details.
`Deployment wiki page <http://wiki.mediagoblin.org/Deployment>`_. Please note that we are not able to provide support for these
alternative deployment options.
.. note:: .. note::
@ -57,59 +57,73 @@ Dependencies
MediaGoblin has the following core dependencies: MediaGoblin has the following core dependencies:
- Python 2.7 or Python 3.4+ - Python 3.4+ (Python 2.7 is supported, but not recommended)
- `python3-lxml <http://lxml.de/>`_ - `python3-lxml <http://lxml.de/>`_
- `git <http://git-scm.com/>`_ - `git <http://git-scm.com/>`_
- `SQLite <http://www.sqlite.org/>`_/`PostgreSQL <http://www.postgresql.org/>`_ - `SQLite <http://www.sqlite.org/>`_/`PostgreSQL <http://www.postgresql.org/>`_
- `Python Imaging Library <http://www.pythonware.com/products/pil/>`_ (PIL) - `Python Imaging Library <http://www.pythonware.com/products/pil/>`_ (PIL)
- `virtualenv <http://www.virtualenv.org/>`_ - `virtualenv <http://www.virtualenv.org/>`_
- `nodejs <https://nodejs.org>`_ - `Node.js <https://nodejs.org>`_
On a DEB-based system (e.g Debian, gNewSense, Trisquel, Ubuntu, and On a DEB-based system (e.g Debian, Trisquel, Ubuntu and
derivatives) issue the following command:: derivatives) issue the following commands::
sudo apt-get install git-core python python3-dev python3-lxml \ # Debian, Trisquel, Ubuntu and derivatives (Hereafter Debian and co.)
python3-imaging python3-virtualenv npm nodejs-legacy automake \ sudo apt update
nginx rabbitmq-server sudo apt install automake git nodejs npm python3-dev python3-gi \
python3-gst-1.0 python3-lxml python3-pil virtualenv
On a RPM-based system (e.g. Fedora, RedHat, and derivatives) issue the # Fedora, CentOS, RHEL and derivatives (Hereafter Fedora and co.)
following command:: dnf install automake gcc git-core make nodejs npm python3-devel \
python3-lxml python3-pillow virtualenv
sudo yum install python3-paste-deploy python3-paste-script \ .. note::
git-core python python3-devel python3-lxml python3-imaging \
python3-virtualenv npm automake nginx rabbitmq-server
(Note: MediaGoblin now uses Python 3 by default. To use Python 2, you may instead MediaGoblin now uses Python 3 by default. To use Python 2, you may
substitute from "python3" to "python" for most package names in the instead substitute from "python3" to "python" for most package
Debian instructions and this should cover dependency installation. names in the Debian instructions and this should cover dependency
These instructions have not yet been tested on Fedora.) installation. Python 2 installation has not been tested on Fedora
and co.
(Note: you might have to include additional repositories to a RPM- For a production deployment, you'll also need Nginx as frontend web
based system, because rabbitmq-server might be not included in server and RabbitMQ to store the media processing queue::
official repositories. As an alternative, you can try installing
redis-server and configure it as celery broker) # Debian and co.
sudo apt install nginx-light rabbitmq-server
# Fedora and co.
dnf install nginx rabbitmq
.. note::
You might have to enable additional repositories under Fedora and
co. because rabbitmq-server might be not included in official
repositories. That looks like this for CentOS::
dnf install centos-release-rabbitmq-38.noarc
As an alternative, you can try installing redis-server and
configure it as celery broker.
Configure PostgreSQL Configure PostgreSQL
~~~~~~~~~~~~~~~~~~~~ ~~~~~~~~~~~~~~~~~~~~
.. note:: .. note::
MediaGoblin currently supports PostgreSQL and SQLite. The default is a MediaGoblin currently supports PostgreSQL and SQLite. The default
local SQLite database. This will "just work" for small deployments. is a local SQLite database. This will "just work" for small
deployments. For medium to large deployments we recommend
For medium to large deployments we recommend PostgreSQL. PostgreSQL. If you don't want/need PostgreSQL, skip this section.
If you don't want/need PostgreSQL, skip this section.
These are the packages needed for Debian Jessie (stable):: These are the packages needed for Debian Jessie (stable)::
sudo apt-get install postgresql postgresql-client python3-psycopg2 sudo apt install postgresql python3-psycopg2
These are the packages needed for an RPM-based system:: These are the packages needed for an RPM-based system::
sudo yum install postgresql postgresql-server python3-psycopg2 sudo dnf install postgresql postgresql-server python3-psycopg2
An rpm-based system also requires that you initialize and start the Fedora and co. also requires that you initialize and start the
PostgreSQL database with a few commands. The following commands are PostgreSQL database with a few commands. The following commands are
not needed on a Debian-based platform, however:: not needed on a Debian-based platform, however::
@ -123,29 +137,13 @@ new database user with restricted privileges and a new database owned by our
restricted database user for our MediaGoblin instance. restricted database user for our MediaGoblin instance.
In this example, the database user will be ``mediagoblin`` and the database In this example, the database user will be ``mediagoblin`` and the database
name will be ``mediagoblin`` too. name will be ``mediagoblin`` too. We'll first at the user::
We'll add these entities by first switching to the *postgres* account:: sudo --login --user=postgres createuser --no-createdb mediagoblin
sudo su - postgres
This will change your prompt to a shell prompt, such as *-bash-4.2$*. Enter
the following *createuser* and *createdb* commands at that prompt. We'll
create the *mediagoblin* database user first::
# this command and the one that follows are run as the ``postgres`` user:
createuser -A -D mediagoblin
Then we'll create the database where all of our MediaGoblin data will be stored:: Then we'll create the database where all of our MediaGoblin data will be stored::
createdb -E UNICODE -O mediagoblin mediagoblin sudo --login --user=postgres createdb --encoding=UTF8 --owner=mediagoblin mediagoblin
where the first ``mediagoblin`` is the database owner and the second
``mediagoblin`` is the database name.
Type ``exit`` to exit from the 'postgres' user account.::
exit
.. caution:: Where is the password? .. caution:: Where is the password?
@ -168,34 +166,35 @@ create a dedicated, unprivileged system user for the sole purpose of running
MediaGoblin. Running MediaGoblin processes under an unprivileged system user MediaGoblin. Running MediaGoblin processes under an unprivileged system user
helps to keep it more secure. helps to keep it more secure.
The following command (entered as root or with sudo) will create a The following command will create a system account with a username of
system account with a username of ``mediagoblin``. You may choose a different ``mediagoblin``.
username if you wish.
If you are using a Debian-based system, enter this command:: If you are using a Debian-based system, enter this command::
sudo useradd -c "GNU MediaGoblin system account" -d /var/lib/mediagoblin -m -r -g www-data mediagoblin # Debian and co.
sudo useradd --system --create-home --home-dir /var/lib/mediagoblin --group www-data --comment 'GNU MediaGoblin system account' mediagoblin
If you are using an RPM-based system, enter this command:: # Fedora and co.
sudo useradd --system --create-home --home-dir /var/lib/mediagoblin --group nginx --comment 'GNU MediaGoblin system account' mediagoblin
sudo useradd -c "GNU MediaGoblin system account" -d /var/lib/mediagoblin -m -r -g nginx mediagoblin
This will create a ``mediagoblin`` user and assign it to a group that is This will create a ``mediagoblin`` user and assign it to a group that is
associated with the web server. This will ensure that the web server can associated with the web server. This will ensure that the web server can
read the media files (images, videos, etc.) that users upload. read the media files that users upload (images, videos, etc.)
We will also create a ``mediagoblin`` group and associate the mediagoblin Many operating systems will automatically create a group
user with that group, as well:: ``mediagoblin`` to go with the new user ``mediagoblin``, but just to
be sure::
sudo groupadd mediagoblin && sudo usermod --append -G mediagoblin mediagoblin sudo groupadd --force mediagoblin
sudo usermod --append --groups mediagoblin mediagoblin
No password will be assigned to this account, and you will not be able No password will be assigned to this account, and you will not be able
to log in as this user. To switch to this account, enter:: to log in as this user. To switch to this account, enter::
sudo su mediagoblin -s /bin/bash sudo su mediagoblin --shell=/bin/bash
To return to your regular user account after using the system account, type To return to your regular user account after using the system account, type
``exit``. ``exit`` or ``Ctrl-d``.
.. _create-mediagoblin-directory: .. _create-mediagoblin-directory:
@ -211,19 +210,16 @@ Setting up the working directory requires that we first create the directory
with elevated privileges, and then assign ownership of the directory with elevated privileges, and then assign ownership of the directory
to the unprivileged system account. to the unprivileged system account.
To do this, enter the following command, changing the defaults to suit your To do this, enter the following commands, changing the defaults to suit your
particular requirements. On a Debian-based platform you will enter this:: particular requirements::
sudo mkdir -p /srv/mediagoblin.example.org && sudo chown -hR mediagoblin:www-data /srv/mediagoblin.example.org # Debian and co.
sudo mkdir --parents /srv/mediagoblin.example.org
sudo chown --no-dereference --recursive mediagoblin:www-data /srv/mediagoblin.example.org
On an RPM-based distribution, enter this command:: # Fedora and co.
sudo mkdir --parents /srv/mediagoblin.example.org
sudo mkdir -p /srv/mediagoblin.example.org && sudo chown -hR mediagoblin:nginx /srv/mediagoblin.example.org sudo chown --no-dereference --recursive mediagoblin:nginx /srv/mediagoblin.example.org
.. note::
Unless otherwise noted, the remainder of this document assumes that all
operations are performed using this unprivileged account.
Install MediaGoblin and Virtualenv Install MediaGoblin and Virtualenv
@ -233,41 +229,46 @@ We will now switch to our 'mediagoblin' system account, and then set up
our MediaGoblin source code repository and its necessary services. our MediaGoblin source code repository and its necessary services.
You should modify these commands to suit your own environment. You should modify these commands to suit your own environment.
Change to the MediaGoblin directory that you just created:: Switch to the ``mediagoblin`` unprivileged user and change to the
MediaGoblin directory that you just created::
sudo su mediagoblin -s /bin/bash # to change to the 'mediagoblin' account sudo su mediagoblin --shell=/bin/bash
$ cd /srv/mediagoblin.example.org $ cd /srv/mediagoblin.example.org
Clone the MediaGoblin repository and set up the git submodules::
$ git clone https://git.savannah.gnu.org/git/mediagoblin.git -b stable
$ cd mediagoblin
$ git submodule init && git submodule update
.. note:: .. note::
The MediaGoblin repository used to be on gitorious.org, but since Unless otherwise noted, the remainder of this document assumes that all
gitorious.org shut down, we had to move. We are presently on operations are performed using the unprivileged ``mediagoblin``
Savannah. You may need to update your git repository location:: account, indicated by the ``$`` prefix.
$ git remote set-url origin https://git.savannah.gnu.org/git/mediagoblin.git Clone the MediaGoblin repository and set up the git submodules::
Set up the hacking environment:: $ git clone --depth=1 https://git.savannah.gnu.org/git/mediagoblin.git --branch stable
$ cd mediagoblin
$ git submodule init && git submodule update
$ ./bootstrap.sh && ./configure && make Set up the environment::
(Note that if you'd prefer to run MediaGoblin with Python 2, pass in $ ./bootstrap.sh && VIRTUALENV_FLAGS='--system-site-packages' \
``--without-python3`` to the ``./configure`` command.) ./configure && make
.. note::
If you'd prefer to run MediaGoblin with Python 2, pass in
``--without-python3`` to the ``./configure`` command.
Create and set the proper permissions on the ``user_dev`` directory. Create and set the proper permissions on the ``user_dev`` directory.
This directory will be used to store uploaded media files:: This directory will be used to store uploaded media files::
$ mkdir user_dev && chmod 750 user_dev $ mkdir --mode='u=rwx,g=rx,o=' user_dev
$ chmod g+s user_dev
This concludes the initial configuration of the MediaGoblin This concludes the initial configuration of the MediaGoblin
environment. In the future, when you update your environment. In the future, when you update your
codebase, you should also run:: codebase, you should also run::
sudo su mediagoblin --shell=/bin/bash
$ cd /srv/mediagoblin.example.org
$ git submodule update && ./bin/python setup.py develop --upgrade && ./bin/gmg dbupdate $ git submodule update && ./bin/python setup.py develop --upgrade && ./bin/gmg dbupdate
.. note:: .. note::
@ -284,34 +285,30 @@ Deploy MediaGoblin Services
Edit site configuration Edit site configuration
~~~~~~~~~~~~~~~~~~~~~~~ ~~~~~~~~~~~~~~~~~~~~~~~
A few basic properties must be set before MediaGoblin will work. First Edit ``mediagoblin.ini`` and update ``email_sender_address`` to the
make a copy of ``paste.ini`` for editing so the original address you wish to be used as the sender for system-generated emails.
config files aren't lost (you likely won't need to edit the paste configuration,
but we'll make a local copy of it just in case)::
$ cp -av paste.ini paste_local.ini .. note::
``mediagoblin.ini`` does not need to be copied, because original config is If you're changing the MediaGoblin directories or URL prefix, you
stored in ``mediagoblin.example.ini``. may need to edit ``direct_remote_path``, ``base_dir``, and
``base_url``.
Then edit ``mediagoblin.ini``: .. note::
- Set ``email_sender_address`` to the address you wish to be used as
the sender for system-generated emails
- Edit ``direct_remote_path``, ``base_dir``, and ``base_url`` if
your mediagoblin directory is not the root directory of your
site.
The default config is stored in ``mediagoblin.example.ini`` in case
you ever need it.
Configure MediaGoblin to use the PostgreSQL database Configure MediaGoblin to use the PostgreSQL database
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
If you are using PostgreSQL, edit the ``[mediagoblin]`` section in your If you are using PostgreSQL, edit the ``[mediagoblin]`` section in your
``mediagoblin.ini`` and put in:: ``mediagoblin.ini`` and remove the `# ` prefix on the line containing::
sql_engine = postgresql:///mediagoblin sql_engine = postgresql:///mediagoblin
if you are running the MediaGoblin application as the same 'user' as the This assumes you are running the MediaGoblin application under the
database owner. same system account and database account; both ``mediagoblin``.
Update database data structures Update database data structures
@ -324,6 +321,16 @@ Before you start using the database, you need to run::
to populate the database with the MediaGoblin data structures. to populate the database with the MediaGoblin data structures.
Create an admin account
~~~~~~~~~~~~~~~~~~~~~~~
Create a MediaGoblin account with full administration access. Provide
your own email address and enter a secure password when prompted::
$ ./bin/gmg adduser --username you --email you@example.com
$ ./bin/gmg makeadmin you
Test the Server Test the Server
~~~~~~~~~~~~~~~ ~~~~~~~~~~~~~~~
@ -333,45 +340,45 @@ test the deployment with the following command::
$ ./lazyserver.sh --server-name=broadcast $ ./lazyserver.sh --server-name=broadcast
You should be able to connect to the machine on port 6543 in your You should be able to connect to the machine on port 6543 in your
browser to confirm that the service is operable. browser to confirm that the service is operable. You should also be
able to log in with the admin username and password.
The next series of commands will need to be run as a privileged user. Type Type ``Ctrl-c`` to exit the above server test.
exit to return to the root/sudo account.::
exit The next series of commands will need to be run as a privileged user.
To return to your regular user account after using the system account,
type ``exit`` or ``Ctrl-d``.
.. _webserver-config: .. _webserver-config:
Waitress and nginx Waitress and Nginx
~~~~~~~~~~~~~~~~~~ ~~~~~~~~~~~~~~~~~~
This configuration example will use Nginx, however, you may This configuration example will use Nginx, however, you may use any
use any webserver of your choice. If you do not already have webserver of your choice. If you do not already have a web server,
a web server, consider Nginx, as the configuration files may consider Nginx, as the configuration files may be more clear than the
be more clear than the alternatives. alternatives.
Create a configuration file at Create a configuration file at
``/srv/mediagoblin.example.org/nginx.conf`` and create a symbolic link ``/srv/mediagoblin.example.org/nginx.conf`` and create a symbolic link
into a directory that will be included in your ``nginx`` configuration into a directory that will be included in your ``nginx`` configuration
(e.g. "``/etc/nginx/sites-enabled`` or ``/etc/nginx/conf.d``) with (e.g. "``/etc/nginx/sites-enabled`` or ``/etc/nginx/conf.d``) with the
one of the following commands.
On a DEB-based system (e.g Debian, Trisquel, Ubuntu and derivatives)
issue the following commands::
sudo ln -s /srv/mediagoblin.example.org/nginx.conf /etc/nginx/sites-enabled/
sudo systemctl enable nginx
On a RPM-based system (e.g. Fedora, RedHat, and derivatives) issue the
following commands:: following commands::
sudo ln -s /srv/mediagoblin.example.org/nginx.conf /etc/nginx/conf.d/ # Debian and co.
sudo ln --symbolic /srv/mediagoblin.example.org/nginx.conf /etc/nginx/sites-enabled/mediagoblin.conf
sudo rm --force /etc/nginx/sites-enabled/default
sudo systemctl enable nginx sudo systemctl enable nginx
You can modify these commands and locations depending on your preferences and # Fedora and co.
the existing configuration of your Nginx instance. The contents of sudo ln -s /srv/mediagoblin.example.org/nginx.conf /etc/nginx/conf.d/mediagoblin.conf
this ``nginx.conf`` file should be modeled on the following:: sudo systemctl enable nginx
You can modify these commands and locations depending on your
preferences and the existing configuration of your Nginx instance. The
contents of this ``/srv/mediagoblin.example.org/nginx.conf`` file
should be modeled on the following::
server { server {
################################################# #################################################
@ -446,25 +453,137 @@ process. This approach is faster and requires less memory.
Nginx is now configured to serve the MediaGoblin application. Perform a quick Nginx is now configured to serve the MediaGoblin application. Perform a quick
test to ensure that this configuration works:: test to ensure that this configuration works::
nginx -t sudo nginx -t
If you encounter any errors, review your Nginx configuration files, and try to If you encounter any errors, review your Nginx configuration files, and try to
resolve them. If you do not encounter any errors, you can start your Nginx resolve them. If you do not encounter any errors, you can start your Nginx
server with one of the following commands (depending on your environment):: server (may vary depending on your operating system)::
sudo /etc/init.d/nginx restart
sudo /etc/rc.d/nginx restart
sudo systemctl restart nginx sudo systemctl restart nginx
Now start MediaGoblin. Use the following command sequence as an Now start MediaGoblin to test your Nginx configuration::
example::
cd /srv/mediagoblin.example.org/mediagoblin/ sudo su mediagoblin --shell=/bin/bash
su mediagoblin -s /bin/bash $ cd /srv/mediagoblin.example.org/mediagoblin/
./lazyserver.sh --server-name=main $ ./lazyserver.sh --server-name=main
Visit the site you've set up in your browser by visiting You should be able to connect to the machine on port 80 in your
<http://mediagoblin.example.org>. You should see MediaGoblin! browser to confirm that the service is operable. If this is the
machine in front of you, visit <http://localhost/> or if it is a
remote server visit the URL or IP address provided to you by your
hosting provider. You should see MediaGoblin; this time via Nginx!
.. note::
If you see an Nginx placeholder page, you may need to remove the
Nginx default configuration, or explictly set a ``server_name``
directive in the Nginx config.
Type ``Ctrl-c`` to exit the above server test and ``exit`` or
``Ctrl-d`` to exit the mediagoblin shell.
.. _create-log-file-dir:
Create the directory for your log file:
---------------------------------------
Production logs for the MediaGoblin application are kept in the
``/var/log/mediagoblin`` directory. Create the directory and give it the
proper permissions::
sudo mkdir --parents /var/log/mediagoblin
sudo chown --no-dereference --recursive mediagoblin:mediagoblin /var/log/mediagoblin
.. _systemd-service-files:
Run MediaGoblin as a system service
-----------------------------------
To ensure MediaGoblin is automatically started and restarted in case of
problems, we need to run it as a system service. If your operating system uses
Systemd, you can use Systemd ``service files`` to manage both the Celery and
Paste processes.
Place the following service files in the ``/etc/systemd/system/``
directory. The first file should be named
``mediagoblin-celeryd.service``. Be sure to modify it to suit your
environment's setup:
.. code-block:: bash
# Set the WorkingDirectory and Environment values to match your environment.
[Unit]
Description=MediaGoblin Celeryd
[Service]
User=mediagoblin
Group=mediagoblin
Type=simple
WorkingDirectory=/srv/mediagoblin.example.org/mediagoblin
Environment=MEDIAGOBLIN_CONFIG=/srv/mediagoblin.example.org/mediagoblin/mediagoblin.ini \
CELERY_CONFIG_MODULE=mediagoblin.init.celery.from_celery
ExecStart=/srv/mediagoblin.example.org/mediagoblin/bin/celery worker \
--logfile=/var/log/mediagoblin/celery.log \
--loglevel=INFO
[Install]
WantedBy=multi-user.target
The second file should be named ``mediagoblin-paster.service``:
.. code-block:: bash
# Set the WorkingDirectory and Environment values to match your environment.
[Unit]
Description=Mediagoblin
[Service]
Type=simple
User=mediagoblin
Group=mediagoblin
Environment=CELERY_ALWAYS_EAGER=false
WorkingDirectory=/srv/mediagoblin.example.org/mediagoblin
ExecStart=/srv/mediagoblin.example.org/mediagoblin/bin/paster serve \
/srv/mediagoblin.example.org/mediagoblin/paste.ini \
--log-file=/var/log/mediagoblin/mediagoblin.log \
--server-name=main
[Install]
WantedBy=multi-user.target
Enable these processes to start at boot by entering::
sudo systemctl enable mediagoblin-celeryd.service && sudo systemctl enable mediagoblin-paster.service
Start the processes for the current session with::
sudo systemctl start mediagoblin-celeryd.service
sudo systemctl start mediagoblin-paster.service
If either command above gives you an error, you can investigate the cause of
the error by entering either of::
sudo systemctl status mediagoblin-celeryd.service
sudo systemctl status mediagoblin-paster.service
The above ``systemctl status`` command is also useful if you ever want to
confirm that a process is still running. If you make any changes to the service
files, you can reload the service files by entering::
sudo systemctl daemon-reload
After entering that command, you can attempt to start the Celery or Paste
processes again using ``restart`` instead of ``start``.
Assuming the above was successful, you should now have a MediaGoblin
server that will continue to operate, even after being restarted.
Great job!
.. note:: .. note::
@ -473,49 +592,11 @@ Visit the site you've set up in your browser by visiting
with larger processing requirements, see the with larger processing requirements, see the
":doc:`production-deployments`" documentation. ":doc:`production-deployments`" documentation.
.. note::
Apache This configuration supports upload of images only, but MediaGoblin
~~~~~~ also supports other types of media, such as audio, video, PDFs and
3D models. For details, see ":doc:`media-types-chapter`".
Instructions and scripts for running MediaGoblin on an Apache server
can be found on the `MediaGoblin wiki <http://wiki.mediagoblin.org/Deployment>`_.
Should I Keep Open Registration Enabled?
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Unfortunately, in this current release of MediaGoblin we are suffering
from spammers registering to public instances en masse. As such, you
may want to either:
a) Disable registration on your instance and just make
accounts for people you know and trust (eg via the `gmg adduser`
command). You can disable registration in your mediagoblin.ini
like so::
[mediagoblin]
allow_registration = false
b) Enable a CAPTCHA plugin. But unfortunately, though some CAPTCHA
plugins exist, for various reasons we do not have any general
recommendations we can make at this point.
We hope to have a better solution to this situation shortly. We
apologize for the inconvenience in the meanwhile.
Security Considerations
~~~~~~~~~~~~~~~~~~~~~~~
.. warning::
The directory ``user_dev/crypto/`` contains some very
sensitive files.
Especially the ``itsdangeroussecret.bin`` is very important
for session security. Make sure not to leak its contents anywhere.
If the contents gets leaked nevertheless, delete your file
and restart the server, so that it creates a new secret key.
All previous sessions will be invalidated.
.. ..
Local variables: Local variables:

167
docs/source/siteadmin/production-deployments.rst
View File

@ -19,142 +19,48 @@ This document contains a number of suggestions for deploying
MediaGoblin in actual production environments. Consider MediaGoblin in actual production environments. Consider
":doc:`deploying`" for a basic overview of how to deploy MediaGoblin. ":doc:`deploying`" for a basic overview of how to deploy MediaGoblin.
Deploy with paste
-----------------
The MediaGoblin WSGI application instance you get with ``./lazyserver.sh`` is Should I Keep Open Registration Enabled?
not ideal for a production MediaGoblin deployment. Ideally, you should be able ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
to use a Systemd service file or an init script to launch and restart the
MediaGoblin process.
We will explore setting up MediaGoblin Systemd service files and init scripts, Unfortunately, in this current release of MediaGoblin we are suffering
but first we need to create the directory that will store the MediaGoblin logs. from spammers registering to public instances en masse. As such, you
may want to either:
a) Disable registration on your instance and just make
accounts for people you know and trust (eg via the `gmg adduser`
command). You can disable registration in your mediagoblin.ini
like so::
[mediagoblin]
allow_registration = false
b) Enable a CAPTCHA plugin. But unfortunately, though some CAPTCHA
plugins exist, for various reasons we do not have any general
recommendations we can make at this point.
We hope to have a better solution to this situation shortly. We
apologize for the inconvenience in the meanwhile.
.. _create-log-file-dir: Security Considerations
~~~~~~~~~~~~~~~~~~~~~~~
Create the directory for your log file: .. warning::
---------------------------------------
Production logs for the MediaGoblin application are kept in the The directory ``user_dev/crypto/`` contains some very
``/var/log/mediagoblin`` directory. Create the directory and give it the sensitive files.
proper permissions:: Especially the ``itsdangeroussecret.bin`` is very important
for session security. Make sure not to leak its contents anywhere.
If the contents gets leaked nevertheless, delete your file
and restart the server, so that it creates a new secret key.
All previous sessions will be invalidated.
sudo mkdir -p /var/log/mediagoblin && sudo chown -hR mediagoblin:mediagoblin /var/log/mediagoblin
.. _systemd-service-files:
Use Systemd service files
-------------------------
If your operating system uses Systemd, you can use Systemd ``service files``
to manage both the Celery and Paste processes. Place the following service
files in the ``/etc/systemd/system/`` directory.
The first file should be named ``mediagoblin-celeryd.service``. Be sure to
modify it to suit your environment's setup:
.. code-block:: bash
# Set the WorkingDirectory, Environment and ExecStart values to match your environment.
# If using Debian/Ubuntu, mkdir and chown are located in /bin/mkdir and /bin/chown, respectively.
# If using Fedora/CentOS/Red Hat, mkdir and chown are located in /usr/bin/mkdir and /usr/bin/chown, respectively.
[Unit]
Description=MediaGoblin Celeryd
[Service]
User=mediagoblin
Group=mediagoblin
Type=simple
WorkingDirectory=/srv/mediagoblin.example.org/mediagoblin
# Start mg-celeryd process as root, then switch to mediagoblin user/group
# (This is needed to run the ExecStartPre commands)
PermissionsStartOnly=true
# Create directory for PID (if needed) and set ownership
ExecStartPre=/bin/mkdir -p /run/mediagoblin
ExecStartPre=/bin/chown -hR mediagoblin:mediagoblin /run/mediagoblin
# Celery process will run as the `mediagoblin` user after start.
Environment=MEDIAGOBLIN_CONFIG=/srv/mediagoblin.example.org/mediagoblin/mediagoblin.ini \
CELERY_CONFIG_MODULE=mediagoblin.init.celery.from_celery
ExecStart=/srv/mediagoblin.example.org/mediagoblin/bin/celery worker \
--logfile=/var/log/mediagoblin/celery.log \
--loglevel=INFO
PIDFile=/run/mediagoblin/mediagoblin-celeryd.pid
[Install]
WantedBy=multi-user.target
The second file should be named ``mediagoblin-paster.service``:
.. code-block:: bash
# Set the WorkingDirectory, Environment and ExecStart values to match your environment.
# If using Debian/*buntu, mkdir and chown are located in /bin/mkdir and /bin/chown, respectively.
# If using Fedora/CentOS/Red Hat, mkdir and chown are located in /usr/bin/mkdir and /usr/bin/chown, respectively.
[Unit]
Description=Mediagoblin
[Service]
Type=forking
User=mediagoblin
Group=mediagoblin
Environment=CELERY_ALWAYS_EAGER=false
WorkingDirectory=/srv/mediagoblin.example.org/mediagoblin
# Start mg-paster process as root, then switch to mediagoblin user/group
PermissionsStartOnly=true
ExecStartPre=-/bin/mkdir -p /run/mediagoblin
ExecStartPre=/bin/chown -hR mediagoblin:mediagoblin /run/mediagoblin
ExecStart=/srv/mediagoblin.example.org/mediagoblin/bin/paster serve \
/srv/mediagoblin.example.org/mediagoblin/paste_local.ini \
--pid-file=/var/run/mediagoblin/mediagoblin.pid \
--log-file=/var/log/mediagoblin/mediagoblin.log \
--daemon \
--server-name=main
ExecStop=/srv/mediagoblin.example.org/mediagoblin/bin/paster serve \
--pid-file=/var/run/mediagoblin/mediagoblin.pid \
/srv/mediagoblin.example.org/mediagoblin/paste_local.ini stop
PIDFile=/var/run/mediagoblin/mediagoblin.pid
[Install]
WantedBy=multi-user.target
Enable these processes to start at boot by entering::
sudo systemctl enable mediagoblin-celeryd.service && sudo systemctl enable mediagoblin-paster.service
Start the processes for the current session with::
sudo systemctl start mediagoblin-paster.service
sudo systemctl start mediagoblin-celeryd.service
If either command above gives you an error, you can investigate the cause of
the error by entering::
sudo systemctl status mediagoblin-celeryd.service or
sudo systemctl status mediagoblin-paster.service
The above ``systemctl status`` command is also useful if you ever want to
confirm that a process is still running. If you make any changes to the service
files, you can reload the service files by entering::
sudo systemctl daemon-reload
After entering that command, you can attempt to start the Celery or Paste
processes again.
.. _init-script: .. _init-script:
Use an init script Alternative init scripts
------------------ ------------------------
If your system does not use Systemd, you can use the following command as the If your system does not use Systemd, you can use the following command as the
basis for an init script: basis for an init script:
@ -187,11 +93,6 @@ as the basis for your script:
Members of the MediaGoblin community have provided init scripts for the Members of the MediaGoblin community have provided init scripts for the
following GNU/Linux distributions: following GNU/Linux distributions:
Debian
* `GNU MediaGoblin init scripts
<https://github.com/joar/mediagoblin-init-scripts>`_
by `Joar Wandborg <http://wandborg.se>`_
Arch Linux Arch Linux
* `MediaGoblin - ArchLinux rc.d scripts * `MediaGoblin - ArchLinux rc.d scripts
<http://whird.jpope.org/2012/04/14/mediagoblin-archlinux-rcd-scripts>`_ <http://whird.jpope.org/2012/04/14/mediagoblin-archlinux-rcd-scripts>`_
@ -209,6 +110,9 @@ one distribution to the next.
Separate celery Separate celery
--------------- ---------------
":doc:`deploying`" describes a configuration with a separate Celery process, but
the following section covers this in more detail.
MediaGoblin uses `Celery`_ to handle heavy and long-running tasks. Celery can MediaGoblin uses `Celery`_ to handle heavy and long-running tasks. Celery can
be launched in two ways: be launched in two ways:
@ -251,6 +155,7 @@ background.
.. _sentry: .. _sentry:
Set up sentry to monitor exceptions Set up sentry to monitor exceptions
----------------------------------- -----------------------------------