heckyel/mediagoblin
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

401. Plugin infrastructure

Browse Source 
* implements installing, loading and setup for plugins
* codifies configuration
* has a sample plugin
* docs
* tests
This commit is contained in:
Will Kahn-Greene 2012-03-12 21:17:08 -04:00
parent f10c3bb8e5
commit 29b6f91740
11 changed files with 541 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

105
docs/source/plugins.rst Normal file
View File

@ -0,0 +1,105 @@
=========
Plugins
=========
GNU MediaGoblin supports plugins that, when installed, allow you to
augment MediaGoblin's behavior.
This chapter covers discovering, installing, configuring and removing
plugins.
Discovering plugins
===================
MediaGoblin comes with core plugins. Core plugins are located in the
``mediagoblin.plugins`` module of the MediaGoblin code. Because they
come with MediaGoblin, you don't have to install them, but you do have
to add them to your config file if you're interested in using them.
You can also write your own plugins and additionally find plugins
elsewhere on the Internet. Since these plugins don't come with
MediaGoblin, you must first install them, then add them to your
configuration.
Installing plugins
==================
MediaGoblin core plugins don't need to be installed. For core plugins,
you can skip installation!
If the plugin is not a core plugin and is packaged and available on
the Python Package Index, then you can install the plugin with pip::
pip install <plugin-name>
For example, if we wanted to install the plugin named
"mediagoblin-restrictfive", we would do::
pip install mediagoblin-restrictfive
.. Note::
If you're using a virtual environment, make sure to activate the
virtual environment before installing with pip. Otherwise the
plugin may get installed in a different environment.
Once you've installed the plugin software, you need to tell
MediaGoblin that this is a plugin you want MediaGoblin to use. To do
that, you edit the ``mediagoblin.ini`` file and add the plugin as a
subsection of the plugin section.
For example, say the "mediagoblin-restrictfive" plugin had the Python
package path ``restrictfive``, then you would add ``restrictfive`` to
the ``plugins`` section as a subsection::
[plugins]
[[restrictfive]]
Configuring plugins
===================
Generally, configuration goes in the ``.ini`` file. Configuration for
a specific plugin, goes in a subsection of the ``plugins`` section.
Example 1: Core MediaGoblin plugin
If you wanted to use the core MediaGoblin flatpages plugin, the module
for that is ``mediagoblin.plugins.flatpages`` and you would add that
to your ``.ini`` file like this::
[plugins]
[[mediagoblin.plugins.flatpages]]
# configuration for flatpages plugin here!
Example 2: Plugin that is not a core MediaGoblin plugin
If you installed a hypothetical restrictfive plugin which is in the
module ``restrictfive``, your ``.ini`` file might look like this (with
comments making the bits clearer)::
[plugins]
[[restrictfive]]
# configuration for restrictfive here!
Check the plugin's documentation for what configuration options are
available.
Removing plugins
================
To remove a plugin, use ``pip uninstall``. For example::
pip uninstall mediagoblin-restrictfive
.. Note::
If you're using a virtual environment, make sure to activate the
virtual environment before uninstalling with pip. Otherwise the
plugin may get installed in a different environment.

5
mediagoblin.ini
View File

@ -30,3 +30,8 @@ base_url = /mgoblin_media/
[celery] [celery]
# Put celery stuff here # Put celery stuff here
# place plugins here---each in their own subsection of [plugins]. see
# documentation for details.
#[plugins]

6
mediagoblin/app.py
View File

@ -27,6 +27,7 @@ from mediagoblin.tools.response import render_404
from mediagoblin.tools import request as mg_request from mediagoblin.tools import request as mg_request
from mediagoblin.mg_globals import setup_globals from mediagoblin.mg_globals import setup_globals
from mediagoblin.init.celery import setup_celery_from_config from mediagoblin.init.celery import setup_celery_from_config
from mediagoblin.init.plugins import setup_plugins
from mediagoblin.init import (get_jinja_loader, get_staticdirector, from mediagoblin.init import (get_jinja_loader, get_staticdirector,
setup_global_and_app_config, setup_workbench, setup_database, setup_global_and_app_config, setup_workbench, setup_database,
setup_storage, setup_beaker_cache) setup_storage, setup_beaker_cache)
@ -64,6 +65,11 @@ class MediaGoblinApp(object):
# Setup other connections / useful objects # Setup other connections / useful objects
########################################## ##########################################
# Set up plugins -- need to do this early so that plugins can
# affect startup.
_log.info("Setting up plugins.")
setup_plugins()
# Set up the database # Set up the database
self.connection, self.db = setup_database() self.connection, self.db = setup_database()

59
mediagoblin/init/plugins/__init__.py Normal file
View File

@ -0,0 +1,59 @@
# GNU MediaGoblin -- federated, autonomous media hosting
# Copyright (C) 2011, 2012 MediaGoblin contributors. See AUTHORS.
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU Affero General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU Affero General Public License for more details.
#
# You should have received a copy of the GNU Affero General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
import logging
from mediagoblin import mg_globals
from mediagoblin.tools import pluginapi
_log = logging.getLogger(__name__)
def setup_plugins():
"""This loads, configures and registers plugins
See plugin documentation for more details.
"""
global_config = mg_globals.global_config
plugin_section = global_config.get('plugins', {})
if not plugin_section:
_log.info("No plugins to load")
return
pcache = pluginapi.PluginCache()
# Go through and import all the modules that are subsections of
# the [plugins] section.
for plugin_module, config in plugin_section.items():
_log.info("Importing plugin module: %s" % plugin_module)
# If this throws errors, that's ok--it'll halt mediagoblin
# startup.
__import__(plugin_module)
# Note: One side-effect of importing things is that anything that
# subclassed pluginapi.Plugin is registered.
# Go through all the plugin classes, instantiate them, and call
# setup_plugin so they can figure things out.
for plugin_class in pcache.plugin_classes:
name = plugin_class.__module__ + "." + plugin_class.__name__
_log.info("Loading plugin: %s" % name)
plugin_obj = plugin_class()
plugin_obj.setup_plugin()
pcache.register_plugin_object(plugin_obj)

6
mediagoblin/plugins/README Normal file
View File

@ -0,0 +1,6 @@
========
README
========
This directory holds the MediaGoblin core plugins. These plugins are not
enabled by default. See documentation for enabling plugins.

16
mediagoblin/plugins/__init__.py Normal file
View File

@ -0,0 +1,16 @@
# GNU MediaGoblin -- federated, autonomous media hosting
# Copyright (C) 2011, 2012 MediaGoblin contributors. See AUTHORS.
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU Affero General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU Affero General Public License for more details.
#
# You should have received a copy of the GNU Affero General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.

6
mediagoblin/plugins/sampleplugin/README Normal file
View File

@ -0,0 +1,6 @@
========
README
========
This is a sample plugin. It does nothing interesting other than show
one way to structure a MediaGoblin plugin.

20
mediagoblin/plugins/sampleplugin/__init__.py Normal file
View File

@ -0,0 +1,20 @@
# GNU MediaGoblin -- federated, autonomous media hosting
# Copyright (C) 2011, 2012 MediaGoblin contributors. See AUTHORS.
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU Affero General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU Affero General Public License for more details.
#
# You should have received a copy of the GNU Affero General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
# This imports the module that has the Plugin subclass in it which
# causes that module to get imported and that class to get registered.
import mediagoblin.plugins.sampleplugin.main

42
mediagoblin/plugins/sampleplugin/main.py Normal file
View File

@ -0,0 +1,42 @@
# GNU MediaGoblin -- federated, autonomous media hosting
# Copyright (C) 2011, 2012 MediaGoblin contributors. See AUTHORS.
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU Affero General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU Affero General Public License for more details.
#
# You should have received a copy of the GNU Affero General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
import logging
from mediagoblin.tools.pluginapi import Plugin, get_config
_log = logging.getLogger(__name__)
class SamplePlugin(Plugin):
"""
This is a sample plugin class. It automatically registers itself
with mediagoblin when this module is imported.
The setup_plugin method prints configuration for this plugin if
it exists.
"""
def __init__(self):
self._setup_plugin_called = 0
def setup_plugin(self):
_log.info('Sample plugin set up!')
config = get_config('mediagoblin.plugins.sampleplugin')
if config:
_log.info('%r' % config)
else:
_log.info('There is no configuration set.')
self._setup_plugin_called += 1

158
mediagoblin/tests/test_pluginapi.py Normal file
View File

@ -0,0 +1,158 @@
# GNU MediaGoblin -- federated, autonomous media hosting
# Copyright (C) 2011, 2012 MediaGoblin contributors. See AUTHORS.
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU Affero General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU Affero General Public License for more details.
#
# You should have received a copy of the GNU Affero General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
import sys
from configobj import ConfigObj
from mediagoblin import mg_globals
from mediagoblin.init.plugins import setup_plugins
from mediagoblin.tools import pluginapi
from nose.tools import eq_
def with_cleanup(*modules_to_delete):
def _with_cleanup(fun):
"""Wrapper that saves and restores mg_globals"""
def _with_cleanup_inner(*args, **kwargs):
old_app_config = mg_globals.app_config
old_global_config = mg_globals.global_config
# Need to delete icky modules before and after so as to make
# sure things work correctly.
for module in modules_to_delete:
try:
del sys.modules[module]
except KeyError:
pass
# The plugin cache gets populated as a side-effect of
# importing, so it's best to clear it before and after a test.
pcache = pluginapi.PluginCache()
pcache.clear()
try:
return fun(*args, **kwargs)
finally:
mg_globals.app_config = old_app_config
mg_globals.global_config = old_global_config
# Need to delete icky modules before and after so as to make
# sure things work correctly.
for module in modules_to_delete:
try:
del sys.modules[module]
except KeyError:
pass
pcache.clear()
_with_cleanup_inner.__name__ = fun.__name__
return _with_cleanup_inner
return _with_cleanup
def build_config(sections):
"""Builds a ConfigObj object with specified data
:arg sections: list of ``(section_name, section_data,
subsection_list)`` tuples where section_data is a dict and
subsection_list is a list of ``(section_name, section_data,
subsection_list)``, ...
For example:
>>> build_config([
... ('mediagoblin', {'key1': 'val1'}, []),
... ('section2', {}, [
... ('subsection1', {}, [])
... ])
... ])
"""
cfg = ConfigObj()
cfg.filename = 'foo'
def _iter_section(cfg, section_list):
for section_name, data, subsection_list in section_list:
cfg[section_name] = data
_iter_section(cfg[section_name], subsection_list)
_iter_section(cfg, sections)
return cfg
@with_cleanup()
def test_no_plugins():
"""Run setup_plugins with no plugins in config"""
cfg = build_config([('mediagoblin', {}, [])])
mg_globals.app_config = cfg['mediagoblin']
mg_globals.global_config = cfg
pcache = pluginapi.PluginCache()
setup_plugins()
# Make sure we didn't load anything.
eq_(len(pcache.plugin_classes), 0)
eq_(len(pcache.plugin_objects), 0)
@with_cleanup('mediagoblin.plugins.sampleplugin',
'mediagoblin.plugins.sampleplugin.main')
def test_one_plugin():
"""Run setup_plugins with a single working plugin"""
cfg = build_config([
('mediagoblin', {}, []),
('plugins', {}, [
('mediagoblin.plugins.sampleplugin', {}, [])
])
])
mg_globals.app_config = cfg['mediagoblin']
mg_globals.global_config = cfg
pcache = pluginapi.PluginCache()
setup_plugins()
# Make sure we only found one plugin class
eq_(len(pcache.plugin_classes), 1)
# Make sure the class is the one we think it is.
eq_(pcache.plugin_classes[0].__name__, 'SamplePlugin')
# Make sure there was one plugin created
eq_(len(pcache.plugin_objects), 1)
# Make sure we called setup_plugin on SamplePlugin
eq_(pcache.plugin_objects[0]._setup_plugin_called, 1)
@with_cleanup('mediagoblin.plugins.sampleplugin',
'mediagoblin.plugins.sampleplugin.main')
def test_same_plugin_twice():
"""Run setup_plugins with a single working plugin twice"""
cfg = build_config([
('mediagoblin', {}, []),
('plugins', {}, [
('mediagoblin.plugins.sampleplugin', {}, []),
('mediagoblin.plugins.sampleplugin', {}, []),
])
])
mg_globals.app_config = cfg['mediagoblin']
mg_globals.global_config = cfg
pcache = pluginapi.PluginCache()
setup_plugins()
# Make sure we only found one plugin class
eq_(len(pcache.plugin_classes), 1)
# Make sure the class is the one we think it is.
eq_(pcache.plugin_classes[0].__name__, 'SamplePlugin')
# Make sure there was one plugin created
eq_(len(pcache.plugin_objects), 1)
# Make sure we called setup_plugin on SamplePlugin
eq_(pcache.plugin_objects[0]._setup_plugin_called, 1)

118
mediagoblin/tools/pluginapi.py Normal file
View File

@ -0,0 +1,118 @@
# GNU MediaGoblin -- federated, autonomous media hosting
# Copyright (C) 2011, 2012 MediaGoblin contributors. See AUTHORS.
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU Affero General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU Affero General Public License for more details.
#
# You should have received a copy of the GNU Affero General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
"""
This module implements the plugin api bits and provides the plugin
base.
Two things about things in this module:
1. they should be excessively well documented because we should pull
from this file for the docs
2. they should be well tested
How do plugins work?
====================
You create a Python package. In that package, you define a high-level
``__init__.py`` that either defines or imports modules that define
classes that inherit from the ``Plugin`` class.
Lifecycle
=========
1. All the modules listed as subsections of the ``plugins`` section in
the config file are imported and any ``Plugin`` subclasses are
loaded causing it to be registered with the ``PluginCache``.
2. After all plugin modules are imported, registered plugins are
instantiated and ``setup_plugin`` is called with the configuration.
How to build a plugin
=====================
See the documentation on building plugins.
"""
import logging
from mediagoblin import mg_globals
_log = logging.getLogger(__name__)
class PluginCache(object):
"""Cache of plugin things"""
__state = {
# list of plugin classes
"plugin_classes": [],
# list of plugin objects
"plugin_objects": []
}
def clear(self):
"""This is only useful for testing."""
del self.plugin_classes[:]
del self.plugin_objects[:]
def __init__(self):
self.__dict__ = self.__state
def register_plugin_class(self, plugin_class):
"""Registers a plugin class"""
self.plugin_classes.append(plugin_class)
def register_plugin_object(self, plugin_obj):
"""Registers a plugin object"""
self.plugin_objects.append(plugin_obj)
class MetaPluginClass(type):
"""Metaclass for PluginBase derivatives"""
def __new__(cls, name, bases, attrs):
new_class = super(MetaPluginClass, cls).__new__(cls, name, bases, attrs)
parents = [b for b in bases if isinstance(b, MetaPluginClass)]
if not parents:
return new_class
PluginCache().register_plugin_class(new_class)
return new_class
class Plugin(object):
__metaclass__ = MetaPluginClass
def setup_plugin(self):
pass
def get_config(key):
"""Retrieves the configuration for a specified plugin by key
Example:
>>> get_config('mediagoblin.plugins.sampleplugin')
{'foo': 'bar'}
"""
global_config = mg_globals.global_config
plugin_section = global_config.get('plugins', {})
return plugin_section.get(key, {})