heckyel/mediagoblin
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Merge branch 'master' of git://gitorious.org/mediagoblin/mediagoblin

Browse Source
This commit is contained in:
xray7224 2013-07-14 15:27:52 +01:00
commit e49263564b
48 changed files with 395 additions and 261 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

1
docs/source/index.rst
View File

@ -74,6 +74,7 @@ This guide covers writing new GNU MediaGoblin plugins.
pluginwriter/database
pluginwriter/api
pluginwriter/tests
pluginwriter/media_type_hooks
Part 4: Developer's Zone

38
docs/source/pluginwriter/media_type_hooks.rst Normal file
View File

@ -0,0 +1,38 @@
==================
Media Type hooks
==================
This documents the hooks that are currently available for ``media_type`` plugins.
What hooks are available?
=========================
'sniff_handler'
---------------
This hook is used by ``sniff_media`` in ``mediagoblin.media_types.__init__``.
Your media type should return its ``sniff_media`` method when this hook is
called.
.. Note::
Your ``sniff_media`` method should return either the ``media_type`` or
``None``.
'get_media_type_and_manager'
----------------------------
This hook is used by ``get_media_type_and_manager`` in
``mediagoblin.media_types.__init__``. When this hook is called, your media type
plugin should check if it can handle the given extension. If so, your media
type plugin should return the media type and media manager.
('media_manager', MEDIA_TYPE)
-----------------------------
If you already know the string representing the media type of a type
of media, you can pull down the manager specifically. Note that this
hook is not a string but a tuple of two strings, the latter being the
name of the media type.
This is used by media entries to pull down their media managers, and
so on.

37
docs/source/siteadmin/media-types.rst
View File

@ -18,16 +18,18 @@ Media Types
====================
In the future, there will be all sorts of media types you can enable,
but in the meanwhile there are three additional media types: video, audio
and ascii art.
but in the meanwhile there are five additional media types: video, audio,
ascii art, STL/3d models, PDF and Document.
First, you should probably read ":doc:`configuration`" to make sure
you know how to modify the mediagoblin config file.
Enabling Media Types
====================
.. note::
Media types are now plugins
Media types are enabled in your mediagoblin configuration file, typically it is
created by copying ``mediagoblin.ini`` to ``mediagoblin_local.ini`` and then
applying your changes to ``mediagoblin_local.ini``. If you don't already have a
@ -37,11 +39,13 @@ Most media types have additional dependencies that you will have to install.
You will find descriptions on how to satisfy the requirements of each media type
on this page.
To enable a media type, edit the ``media_types`` list in your
``mediagoblin_local.ini``. For example, if your system supported image and
video media types, then the list would look like this::
To enable a media type, add the the media type under the ``[plugins]`` section
in you ``mediagoblin_local.ini``. For example, if your system supported image
and video media types, then it would look like this::
media_types = mediagoblin.media_types.image, mediagoblin.media_types.video
[plugins]
[[mediagoblin.media_types.image]]
[[mediagoblin.media_types.video]]
Note that after enabling new media types, you must run dbupdate like so::
@ -83,8 +87,8 @@ good/bad/ugly). On Debianoid systems
gstreamer0.10-ffmpeg
Add ``mediagoblin.media_types.video`` to the ``media_types`` list in your
``mediagoblin_local.ini`` and restart MediaGoblin.
Add ``[[mediagoblin.media_types.video]]`` under the ``[plugins]`` section in
your ``mediagoblin_local.ini`` and restart MediaGoblin.
Run
@ -133,7 +137,7 @@ Then install ``scikits.audiolab`` for the spectrograms::
./bin/pip install scikits.audiolab
Add ``mediagoblin.media_types.audio`` to the ``media_types`` list in your
Add ``[[mediagoblin.media_types.audio]]`` under the ``[plugins]`` section in your
``mediagoblin_local.ini`` and restart MediaGoblin.
Run
@ -158,13 +162,8 @@ library, which is necessary for creating thumbnails of ascii art
Next, modify (and possibly copy over from ``mediagoblin.ini``) your
``mediagoblin_local.ini``. In the ``[mediagoblin]`` section, add
``mediagoblin.media_types.ascii`` to the ``media_types`` list.
For example, if your system supported image and ascii art media types, then
the list would look like this::
media_types = mediagoblin.media_types.image, mediagoblin.media_types.ascii
``mediagoblin_local.ini``. In the ``[plugins]`` section, add
``[[mediagoblin.media_types.ascii]]``.
Run
@ -184,7 +183,7 @@ your execution path. This feature has been tested with Blender 2.63.
It may work on some earlier versions, but that is not guaranteed (and
is surely not to work prior to Blender 2.5X).
Add ``mediagoblin.media_types.stl`` to the ``media_types`` list in your
Add ``[[mediagoblin.media_types.stl]]`` under the ``[plugins]`` section in your
``mediagoblin_local.ini`` and restart MediaGoblin.
Run
@ -233,7 +232,7 @@ This feature has been tested on Fedora with:
It may work on some earlier versions, but that is not guaranteed.
Add ``mediagoblin.media_types.pdf`` to the ``media_types`` list in your
Add ``[[mediagoblin.media_types.pdf]]`` under the ``[plugins]`` section in your
``mediagoblin_local.ini`` and restart MediaGoblin.
Run

13
mediagoblin.ini
View File

@ -11,19 +11,15 @@ email_sender_address = "notice@mediagoblin.example.org"
## Uncomment and change to your DB's appropiate setting.
## Default is a local sqlite db "mediagoblin.db".
## Don't forget to run `./bin/gmg dbupdate` after having changed it.
# sql_engine = postgresql:///mediagoblin
# set to false to enable sending notices
# Set to false to enable sending notices
email_debug_mode = true
# Set to false to disable registrations
allow_registration = true
## Uncomment this to turn on video or enable other media types
## You may have to install dependencies, and will have to run ./bin/gmg dbupdate
## See http://docs.mediagoblin.org/siteadmin/media-types.html for details.
# media_types = mediagoblin.media_types.image, mediagoblin.media_types.video
## Uncomment this to put some user-overriding templates here
# local_templates = %(here)s/user_dev/templates/
@ -43,8 +39,9 @@ base_url = /mgoblin_media/
[celery]
# Put celery stuff here
# place plugins here---each in their own subsection of [plugins]. see
# documentation for details.
# Place plugins here, each in their own subsection of [plugins].
# See http://docs.mediagoblin.org/siteadmin/plugins.html for details.
[plugins]
[[mediagoblin.plugins.geolocation]]
[[mediagoblin.plugins.basic_auth]]
[[mediagoblin.media_types.image]]

3
mediagoblin/app.py
View File

@ -29,6 +29,7 @@ from mediagoblin.tools import common, session, translate, template
from mediagoblin.tools.response import render_http_exception
from mediagoblin.tools.theme import register_themes
from mediagoblin.tools import request as mg_request
from mediagoblin.media_types.tools import media_type_warning
from mediagoblin.mg_globals import setup_globals
from mediagoblin.init.celery import setup_celery_from_config
from mediagoblin.init.plugins import setup_plugins
@ -69,6 +70,8 @@ class MediaGoblinApp(object):
# Open and setup the config
global_config, app_config = setup_global_and_app_config(config_path)
media_type_warning()
setup_crypto()
##########################################

2
mediagoblin/auth/views.py
View File

@ -88,6 +88,8 @@ def login(request):
if user:
# set up login in session
if login_form.stay_logged_in.data:
request.session['stay_logged_in'] = True
request.session['user_id'] = unicode(user.id)
request.session.save()

3
mediagoblin/config_spec.ini
View File

@ -5,9 +5,6 @@ html_title = string(default="GNU MediaGoblin")
# link to source for this MediaGoblin site
source_link = string(default="https://gitorious.org/mediagoblin/mediagoblin")
# Enabled media types
media_types = string_list(default=list("mediagoblin.media_types.image"))
# database stuff
sql_engine = string(default="sqlite:///%(here)s/mediagoblin.db")

12
mediagoblin/db/base.py
View File

@ -24,18 +24,6 @@ Session = scoped_session(sessionmaker())
class GMGTableBase(object):
query = Session.query_property()
@classmethod
def find(cls, query_dict):
return cls.query.filter_by(**query_dict)
@classmethod
def find_one(cls, query_dict):
return cls.query.filter_by(**query_dict).first()
@classmethod
def one(cls, query_dict):
return cls.find(query_dict).one()
def get(self, key):
return getattr(self, key)

3
mediagoblin/db/migration_tools.py
View File

@ -175,8 +175,7 @@ class MigrationManager(object):
if self.name == u'__main__':
return u"main mediagoblin tables"
else:
# TODO: Use the friendlier media manager "human readable" name
return u'media type "%s"' % self.name
return u'plugin "%s"' % self.name
def init_or_migrate(self):
"""

17
mediagoblin/db/mixin.py
View File

@ -29,15 +29,14 @@ real objects.
import uuid
import re
import datetime
from datetime import datetime
from werkzeug.utils import cached_property
from mediagoblin import mg_globals
from mediagoblin.media_types import get_media_managers, FileTypeNotSupported
from mediagoblin.media_types import FileTypeNotSupported
from mediagoblin.tools import common, licenses
from mediagoblin.tools.pluginapi import hook_handle
from mediagoblin.tools.text import cleaned_markdown_conversion
from mediagoblin.tools.url import slugify
@ -204,14 +203,14 @@ class MediaEntryMixin(GenerateSlugMixin):
Raises FileTypeNotSupported in case no such manager is enabled
"""
# TODO, we should be able to make this a simple lookup rather
# than iterating through all media managers.
for media_type, manager in get_media_managers():
if media_type == self.media_type:
return manager(self)
manager = hook_handle(('media_manager', self.media_type))
if manager:
return manager(self)
# Not found? Then raise an error
raise FileTypeNotSupported(
"MediaManager not in enabled types. Check media_types in config?")
"MediaManager not in enabled types. Check media_type plugins are"
" enabled in config?")
def get_fail_exception(self):
"""

4
mediagoblin/db/open.py
View File

@ -52,10 +52,6 @@ class DatabaseMaster(object):
def load_models(app_config):
import mediagoblin.db.models
for media_type in app_config['media_types']:
_log.debug("Loading %s.models", media_type)
__import__(media_type + ".models")
for plugin in mg_globals.global_config.get('plugins', {}).keys():
_log.debug("Loading %s.models", plugin)
try:

2
mediagoblin/db/util.py
View File

@ -24,7 +24,7 @@ from mediagoblin.db.models import MediaEntry, Tag, MediaTag, Collection
def atomic_update(table, query_dict, update_values):
table.find(query_dict).update(update_values,
table.query.filter_by(**query_dict).update(update_values,
synchronize_session=False)
Session.commit()

22
mediagoblin/decorators.py
View File

@ -90,8 +90,8 @@ def user_may_alter_collection(controller):
"""
@wraps(controller)
def wrapper(request, *args, **kwargs):
creator_id = request.db.User.find_one(
{'username': request.matchdict['user']}).id
creator_id = request.db.User.query.filter_by(
username=request.matchdict['user']).first().id
if not (request.user.is_admin or
request.user.id == creator_id):
raise Forbidden()
@ -165,15 +165,15 @@ def get_user_collection(controller):
"""
@wraps(controller)
def wrapper(request, *args, **kwargs):
user = request.db.User.find_one(
{'username': request.matchdict['user']})
user = request.db.User.query.filter_by(
username=request.matchdict['user']).first()
if not user:
return render_404(request)
collection = request.db.Collection.find_one(
{'slug': request.matchdict['collection'],
'creator': user.id})
collection = request.db.Collection.query.filter_by(
slug=request.matchdict['collection'],
creator=user.id).first()
# Still no collection? Okay, 404.
if not collection:
@ -190,14 +190,14 @@ def get_user_collection_item(controller):
"""
@wraps(controller)
def wrapper(request, *args, **kwargs):
user = request.db.User.find_one(
{'username': request.matchdict['user']})
user = request.db.User.query.filter_by(
username=request.matchdict['user']).first()
if not user:
return render_404(request)
collection_item = request.db.CollectionItem.find_one(
{'id': request.matchdict['collection_item'] })
collection_item = request.db.CollectionItem.query.filter_by(
id=request.matchdict['collection_item']).first()
# Still no collection item? Okay, 404.
if not collection_item:

1
mediagoblin/edit/forms.py
View File

@ -66,7 +66,6 @@ class EditAccountForm(wtforms.Form):
[wtforms.validators.Optional(),
normalize_user_or_email_field(allow_user=False)])
wants_comment_notification = wtforms.BooleanField(
label='',
description=_("Email me when others comment on my media"))
license_preference = wtforms.SelectField(
_('License preference'),

6
mediagoblin/edit/views.py
View File

@ -305,9 +305,9 @@ def edit_collection(request, collection):
form.slug.data, collection.id)
# Make sure there isn't already a Collection with this title
existing_collection = request.db.Collection.find_one({
'creator': request.user.id,
'title':form.title.data})
existing_collection = request.db.Collection.query.filter_by(
creator=request.user.id,
title=form.title.data).first()
if existing_collection and existing_collection.id != collection.id:
messages.add_message(

10
mediagoblin/gmg_commands/dbupdate.py
View File

@ -42,7 +42,7 @@ class DatabaseData(object):
self.name, self.models, self.migrations, session)
def gather_database_data(media_types, plugins):
def gather_database_data(plugins):
"""
Gather all database data relevant to the extensions we have
installed so we can do migrations and table initialization.
@ -59,13 +59,6 @@ def gather_database_data(media_types, plugins):
DatabaseData(
u'__main__', MAIN_MODELS, MAIN_MIGRATIONS))
# Then get all registered media managers (eventually, plugins)
for media_type in media_types:
models = import_component('%s.models:MODELS' % media_type)
migrations = import_component('%s.migrations:MIGRATIONS' % media_type)
managed_dbdata.append(
DatabaseData(media_type, models, migrations))
for plugin in plugins:
try:
models = import_component('{0}.models:MODELS'.format(plugin))
@ -127,7 +120,6 @@ def run_all_migrations(db, app_config, global_config):
"""
# Gather information from all media managers / projects
dbdatas = gather_database_data(
app_config['media_types'],
global_config.get('plugins', {}).keys())
Session = sessionmaker(bind=db.engine)

4
mediagoblin/gmg_commands/import_export.py
View File

@ -63,7 +63,7 @@ def _import_media(db, args):
# TODO: Add import of queue files
queue_cache = BasicFileStorage(args._cache_path['queue'])
for entry in db.MediaEntry.find():
for entry in db.MediaEntry.query.filter_by():
for name, path in entry.media_files.items():
_log.info('Importing: {0} - {1}'.format(
entry.title.encode('ascii', 'replace'),
@ -204,7 +204,7 @@ def _export_media(db, args):
# TODO: Add export of queue files
queue_cache = BasicFileStorage(args._cache_path['queue'])
for entry in db.MediaEntry.find():
for entry in db.MediaEntry.query.filter_by():
for name, path in entry.media_files.items():
_log.info(u'Exporting {0} - {1}'.format(
entry.title,

12
mediagoblin/gmg_commands/users.py
View File

@ -40,9 +40,9 @@ def adduser(args):
db = mg_globals.database
users_with_username = \
db.User.find({
'username': args.username.lower(),
}).count()
db.User.query.filter_by(
username=args.username.lower()
).count()
if users_with_username:
print u'Sorry, a user with that name already exists.'
@ -71,7 +71,8 @@ def makeadmin(args):
db = mg_globals.database
user = db.User.one({'username': unicode(args.username.lower())})
user = db.User.query.filter_by(
username=unicode(args.username.lower())).one()
if user:
user.is_admin = True
user.save()
@ -94,7 +95,8 @@ def changepw(args):
db = mg_globals.database
user = db.User.one({'username': unicode(args.username.lower())})
user = db.User.query.filter_by(
username=unicode(args.username.lower())).one()
if user:
user.pw_hash = auth.gen_password_hash(args.password)
user.save()

17
mediagoblin/listings/views.py
View File

@ -14,6 +14,7 @@
# You should have received a copy of the GNU Affero General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
from mediagoblin import mg_globals
from mediagoblin.db.models import MediaEntry
from mediagoblin.db.util import media_entries_for_tag_slug
from mediagoblin.tools.pagination import Pagination
@ -80,6 +81,17 @@ def atom_feed(request):
link = request.urlgen('index', qualified=True)
feed_title += "for all recent items"
atomlinks = [
{'href': link,
'rel': 'alternate',
'type': 'text/html'}]
if mg_globals.app_config["push_urls"]:
for push_url in mg_globals.app_config["push_urls"]:
atomlinks.append({
'rel': 'hub',
'href': push_url})
cursor = cursor.order_by(MediaEntry.created.desc())
cursor = cursor.limit(ATOM_DEFAULT_NR_OF_UPDATED_ITEMS)
@ -87,9 +99,8 @@ def atom_feed(request):
feed_title,
feed_url=request.url,
id=link,
links=[{'href': link,
'rel': 'alternate',
'type': 'text/html'}])
links=atomlinks)
for entry in cursor:
feed.add(entry.get('title'),
entry.description_html,

77
mediagoblin/media_types/__init__.py
View File

@ -15,12 +15,10 @@
# along with this program. If not, see <http://www.gnu.org/licenses/>.
import os
import sys
import logging
import tempfile
from mediagoblin import mg_globals
from mediagoblin.tools.common import import_component
from mediagoblin.tools.pluginapi import hook_handle
from mediagoblin.tools.translate import lazy_pass_to_ugettext as _
_log = logging.getLogger(__name__)
@ -52,36 +50,6 @@ class MediaManagerBase(object):
return hasattr(self, i)
class CompatMediaManager(object):
def __init__(self, mm_dict, entry=None):
self.mm_dict = mm_dict
self.entry = entry
def __call__(self, entry):
"So this object can look like a class too, somehow"
assert self.entry is None
return self.__class__(self.mm_dict, entry)
def __getitem__(self, i):
return self.mm_dict[i]
def __contains__(self, i):
return (i in self.mm_dict)
@property
def media_fetch_order(self):
return self.mm_dict.get('media_fetch_order')
def sniff_handler(self, *args, **kwargs):
func = self.mm_dict.get("sniff_handler", None)
if func is not None:
return func(*args, **kwargs)
return False
def __getattr__(self, i):
return self.mm_dict[i]
def sniff_media(media):
'''
Iterate through the enabled media types and find those suited
@ -98,40 +66,18 @@ def sniff_media(media):
media_file.write(media.stream.read())
media.stream.seek(0)
for media_type, manager in get_media_managers():
_log.info('Sniffing {0}'.format(media_type))
if manager.sniff_handler(media_file, media=media):
_log.info('{0} accepts the file'.format(media_type))
return media_type, manager
else:
_log.debug('{0} did not accept the file'.format(media_type))
media_type = hook_handle('sniff_handler', media_file, media=media)
if media_type:
_log.info('{0} accepts the file'.format(media_type))
return media_type, hook_handle(('media_manager', media_type))
else:
_log.debug('{0} did not accept the file'.format(media_type))
raise FileTypeNotSupported(
# TODO: Provide information on which file types are supported
_(u'Sorry, I don\'t support that file type :('))
def get_media_types():
"""
Generator, yields the available media types
"""
for media_type in mg_globals.app_config['media_types']:
yield media_type
def get_media_managers():
'''
Generator, yields all enabled media managers
'''
for media_type in get_media_types():
mm = import_component(media_type + ":MEDIA_MANAGER")
if isinstance(mm, dict):
mm = CompatMediaManager(mm)
yield media_type, mm
def get_media_type_and_manager(filename):
'''
Try to find the media type based on the file name, extension
@ -142,11 +88,10 @@ def get_media_type_and_manager(filename):
# Get the file extension
ext = os.path.splitext(filename)[1].lower()
for media_type, manager in get_media_managers():
# Omit the dot from the extension and match it against
# the media manager
if ext[1:] in manager.accepted_extensions:
return media_type, manager
# Omit the dot from the extension and match it against
# the media manager
if hook_handle('get_media_type_and_manager', ext[1:]):
return hook_handle('get_media_type_and_manager', ext[1:])
else:
_log.info('File {0} has no file extension, let\'s hope the sniffers get it.'.format(
filename))

22
mediagoblin/media_types/ascii/__init__.py
View File

@ -17,15 +17,31 @@
from mediagoblin.media_types import MediaManagerBase
from mediagoblin.media_types.ascii.processing import process_ascii, \
sniff_handler
from mediagoblin.tools import pluginapi
ACCEPTED_EXTENSIONS = ["txt", "asc", "nfo"]
MEDIA_TYPE = 'mediagoblin.media_types.ascii'
def setup_plugin():
config = pluginapi.get_config(MEDIA_TYPE)
class ASCIIMediaManager(MediaManagerBase):
human_readable = "ASCII"
processor = staticmethod(process_ascii)
sniff_handler = staticmethod(sniff_handler)
display_template = "mediagoblin/media_displays/ascii.html"
default_thumb = "images/media_thumbs/ascii.jpg"
accepted_extensions = ["txt", "asc", "nfo"]
MEDIA_MANAGER = ASCIIMediaManager
def get_media_type_and_manager(ext):
if ext in ACCEPTED_EXTENSIONS:
return MEDIA_TYPE, ASCIIMediaManager
hooks = {
'setup': setup_plugin,
'get_media_type_and_manager': get_media_type_and_manager,
('media_manager', MEDIA_TYPE): lambda: ASCIIMediaManager,
'sniff_handler': sniff_handler,
}

6
mediagoblin/media_types/ascii/processing.py
View File

@ -28,17 +28,19 @@ from mediagoblin.media_types.ascii import asciitoimage
_log = logging.getLogger(__name__)
SUPPORTED_EXTENSIONS = ['txt', 'asc', 'nfo']
MEDIA_TYPE = 'mediagoblin.media_types.ascii'
def sniff_handler(media_file, **kw):
_log.info('Sniffing {0}'.format(MEDIA_TYPE))
if kw.get('media') is not None:
name, ext = os.path.splitext(kw['media'].filename)
clean_ext = ext[1:].lower()
if clean_ext in SUPPORTED_EXTENSIONS:
return True
return MEDIA_TYPE
return False
return None
def process_ascii(proc_state):

21
mediagoblin/media_types/audio/__init__.py
View File

@ -17,14 +17,29 @@
from mediagoblin.media_types import MediaManagerBase
from mediagoblin.media_types.audio.processing import process_audio, \
sniff_handler
from mediagoblin.tools import pluginapi
ACCEPTED_EXTENSIONS = ["mp3", "flac", "wav", "m4a"]
MEDIA_TYPE = 'mediagoblin.media_types.audio'
def setup_plugin():
config = pluginapi.get_config(MEDIA_TYPE)
class AudioMediaManager(MediaManagerBase):
human_readable = "Audio"
processor = staticmethod(process_audio)
sniff_handler = staticmethod(sniff_handler)
display_template = "mediagoblin/media_displays/audio.html"
accepted_extensions = ["mp3", "flac", "wav", "m4a"]
MEDIA_MANAGER = AudioMediaManager
def get_media_type_and_manager(ext):
if ext in ACCEPTED_EXTENSIONS:
return MEDIA_TYPE, AudioMediaManager
hooks = {
'setup': setup_plugin,
'get_media_type_and_manager': get_media_type_and_manager,
'sniff_handler': sniff_handler,
('media_manager', MEDIA_TYPE): lambda: AudioMediaManager,
}

9
mediagoblin/media_types/audio/processing.py
View File

@ -27,19 +27,22 @@ from mediagoblin.media_types.audio.transcoders import (AudioTranscoder,
_log = logging.getLogger(__name__)
MEDIA_TYPE = 'mediagoblin.media_types.audio'
def sniff_handler(media_file, **kw):
_log.info('Sniffing {0}'.format(MEDIA_TYPE))
try:
transcoder = AudioTranscoder()
data = transcoder.discover(media_file.name)
except BadMediaFail:
_log.debug('Audio discovery raised BadMediaFail')
return False
return None
if data.is_audio == True and data.is_video == False:
return True
return MEDIA_TYPE
return False
return None
def process_audio(proc_state):

25
mediagoblin/media_types/image/__init__.py
View File

@ -13,21 +13,28 @@
#
# You should have received a copy of the GNU Affero General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
import datetime
from mediagoblin.media_types import MediaManagerBase
from mediagoblin.media_types.image.processing import process_image, \
sniff_handler
from mediagoblin.tools import pluginapi
ACCEPTED_EXTENSIONS = ["jpg", "jpeg", "png", "gif", "tiff"]
MEDIA_TYPE = 'mediagoblin.media_types.image'
def setup_plugin():
config = pluginapi.get_config('mediagoblin.media_types.image')
class ImageMediaManager(MediaManagerBase):
human_readable = "Image"
processor = staticmethod(process_image)
sniff_handler = staticmethod(sniff_handler)
display_template = "mediagoblin/media_displays/image.html"
default_thumb = "images/media_thumbs/image.png"
accepted_extensions = ["jpg", "jpeg", "png", "gif", "tiff"]
media_fetch_order = [u'medium', u'original', u'thumb']
def get_original_date(self):
@ -52,4 +59,14 @@ class ImageMediaManager(MediaManagerBase):
return None
MEDIA_MANAGER = ImageMediaManager
def get_media_type_and_manager(ext):
if ext in ACCEPTED_EXTENSIONS:
return MEDIA_TYPE, ImageMediaManager
hooks = {
'setup': setup_plugin,
'get_media_type_and_manager': get_media_type_and_manager,
'sniff_handler': sniff_handler,
('media_manager', MEDIA_TYPE): lambda: ImageMediaManager,
}

9
mediagoblin/media_types/image/processing.py
View File

@ -35,6 +35,8 @@ PIL_FILTERS = {
'BICUBIC': Image.BICUBIC,
'ANTIALIAS': Image.ANTIALIAS}
MEDIA_TYPE = 'mediagoblin.media_types.image'
def resize_image(proc_state, resized, keyname, target_name, new_size,
exif_tags, workdir):
@ -95,17 +97,18 @@ def resize_tool(proc_state, force, keyname, target_name,
exif_tags, conversions_subdir)
SUPPORTED_FILETYPES = ['png', 'gif', 'jpg', 'jpeg']
SUPPORTED_FILETYPES = ['png', 'gif', 'jpg', 'jpeg', 'tiff']
def sniff_handler(media_file, **kw):
_log.info('Sniffing {0}'.format(MEDIA_TYPE))
if kw.get('media') is not None: # That's a double negative!
name, ext = os.path.splitext(kw['media'].filename)
clean_ext = ext[1:].lower() # Strip the . from ext and make lowercase
if clean_ext in SUPPORTED_FILETYPES:
_log.info('Found file extension in supported filetypes')
return True
return MEDIA_TYPE
else:
_log.debug('Media present, extension not found in {0}'.format(
SUPPORTED_FILETYPES))
@ -113,7 +116,7 @@ def sniff_handler(media_file, **kw):
_log.warning('Need additional information (keyword argument \'media\')'
' to be able to handle sniffing')
return False
return None
def process_image(proc_state):

22
mediagoblin/media_types/pdf/__init__.py
View File

@ -17,15 +17,31 @@
from mediagoblin.media_types import MediaManagerBase
from mediagoblin.media_types.pdf.processing import process_pdf, \
sniff_handler
from mediagoblin.tools import pluginapi
ACCEPTED_EXTENSIONS = ['pdf']
MEDIA_TYPE = 'mediagoblin.media_types.pdf'
def setup_plugin():
config = pluginapi.get_config(MEDIA_TYPE)
class PDFMediaManager(MediaManagerBase):
human_readable = "PDF"
processor = staticmethod(process_pdf)
sniff_handler = staticmethod(sniff_handler)
display_template = "mediagoblin/media_displays/pdf.html"
default_thumb = "images/media_thumbs/pdf.jpg"
accepted_extensions = ["pdf"]
MEDIA_MANAGER = PDFMediaManager
def get_media_type_and_manager(ext):
if ext in ACCEPTED_EXTENSIONS:
return MEDIA_TYPE, PDFMediaManager
hooks = {
'setup': setup_plugin,
'get_media_type_and_manager': get_media_type_and_manager,
'sniff_handler': sniff_handler,
('media_manager', MEDIA_TYPE): lambda: PDFMediaManager,
}

9
mediagoblin/media_types/pdf/processing.py
View File

@ -25,6 +25,8 @@ from mediagoblin.tools.translate import fake_ugettext_passthrough as _
_log = logging.getLogger(__name__)
MEDIA_TYPE = 'mediagoblin.media_types.pdf'
# TODO - cache (memoize) util
# This is a list created via uniconv --show and hand removing some types that
@ -163,16 +165,17 @@ def check_prerequisites():
return True
def sniff_handler(media_file, **kw):
_log.info('Sniffing {0}'.format(MEDIA_TYPE))
if not check_prerequisites():
return False
return None
if kw.get('media') is not None:
name, ext = os.path.splitext(kw['media'].filename)
clean_ext = ext[1:].lower()
if clean_ext in supported_extensions():
return True
return MEDIA_TYPE
return False
return None
def create_pdf_thumb(original, thumb_filename, width, height):
# Note: pdftocairo adds '.png', remove it

21
mediagoblin/media_types/stl/__init__.py
View File

@ -17,15 +17,30 @@
from mediagoblin.media_types import MediaManagerBase
from mediagoblin.media_types.stl.processing import process_stl, \
sniff_handler
from mediagoblin.tools import pluginapi
MEDIA_TYPE = 'mediagoblin.media_types.stl'
ACCEPTED_EXTENSIONS = ["obj", "stl"]
def setup_plugin():
config = pluginapi.get_config(MEDIA_TYPE)
class STLMediaManager(MediaManagerBase):
human_readable = "stereo lithographics"
processor = staticmethod(process_stl)
sniff_handler = staticmethod(sniff_handler)
display_template = "mediagoblin/media_displays/stl.html"
default_thumb = "images/media_thumbs/video.jpg"
accepted_extensions = ["obj", "stl"]
MEDIA_MANAGER = STLMediaManager
def get_media_type_and_manager(ext):
if ext in ACCEPTED_EXTENSIONS:
return MEDIA_TYPE, STLMediaManager
hooks = {
'setup': setup_plugin,
'get_media_type_and_manager': get_media_type_and_manager,
'sniff_handler': sniff_handler,
('media_manager', MEDIA_TYPE): lambda: STLMediaManager,
}

6
mediagoblin/media_types/stl/processing.py
View File

@ -29,6 +29,7 @@ from mediagoblin.media_types.stl import model_loader
_log = logging.getLogger(__name__)
SUPPORTED_FILETYPES = ['stl', 'obj']
MEDIA_TYPE = 'mediagoblin.media_types.stl'
BLEND_FILE = pkg_resources.resource_filename(
'mediagoblin.media_types.stl',
@ -43,13 +44,14 @@ BLEND_SCRIPT = pkg_resources.resource_filename(
def sniff_handler(media_file, **kw):
_log.info('Sniffing {0}'.format(MEDIA_TYPE))
if kw.get('media') is not None:
name, ext = os.path.splitext(kw['media'].filename)
clean_ext = ext[1:].lower()
if clean_ext in SUPPORTED_FILETYPES:
_log.info('Found file extension in supported filetypes')
return True
return MEDIA_TYPE
else:
_log.debug('Media present, extension not found in {0}'.format(
SUPPORTED_FILETYPES))
@ -57,7 +59,7 @@ def sniff_handler(media_file, **kw):
_log.warning('Need additional information (keyword argument \'media\')'
' to be able to handle sniffing')
return False
return None
def blender_render(config):

27
mediagoblin/media_types/tools.py Normal file
View File

@ -0,0 +1,27 @@
# GNU MediaGoblin -- federated, autonomous media hosting
# Copyright (C) 2011, 2012 MediaGoblin contributors. See AUTHORS.
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU Affero General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU Affero General Public License for more details.
#
# You should have received a copy of the GNU Affero General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
import logging
from mediagoblin import mg_globals
_log = logging.getLogger(__name__)
def media_type_warning():
if mg_globals.app_config.get('media_types'):
_log.warning('Media_types have been converted to plugins. Old'
' media_types will no longer work. Please convert them'
' to plugins to continue using them.')

23
mediagoblin/media_types/video/__init__.py
View File

@ -17,20 +17,35 @@
from mediagoblin.media_types import MediaManagerBase
from mediagoblin.media_types.video.processing import process_video, \
sniff_handler
from mediagoblin.tools import pluginapi
MEDIA_TYPE = 'mediagoblin.media_types.video'
ACCEPTED_EXTENSIONS = [
"mp4", "mov", "webm", "avi", "3gp", "3gpp", "mkv", "ogv", "m4v"]
def setup_plugin():
config = pluginapi.get_config(MEDIA_TYPE)
class VideoMediaManager(MediaManagerBase):
human_readable = "Video"
processor = staticmethod(process_video)
sniff_handler = staticmethod(sniff_handler)
display_template = "mediagoblin/media_displays/video.html"
default_thumb = "images/media_thumbs/video.jpg"
accepted_extensions = [
"mp4", "mov", "webm", "avi", "3gp", "3gpp", "mkv", "ogv", "m4v"]
# Used by the media_entry.get_display_media method
media_fetch_order = [u'webm_640', u'original']
default_webm_type = 'video/webm; codecs="vp8, vorbis"'
MEDIA_MANAGER = VideoMediaManager
def get_media_type_and_manager(ext):
if ext in ACCEPTED_EXTENSIONS:
return MEDIA_TYPE, VideoMediaManager
hooks = {
'setup': setup_plugin,
'get_media_type_and_manager': get_media_type_and_manager,
'sniff_handler': sniff_handler,
('media_manager', MEDIA_TYPE): lambda: VideoMediaManager,
}

9
mediagoblin/media_types/video/processing.py
View File

@ -29,6 +29,8 @@ from .util import skip_transcode
_log = logging.getLogger(__name__)
_log.setLevel(logging.DEBUG)
MEDIA_TYPE = 'mediagoblin.media_types.video'
class VideoTranscodingFail(BaseProcessingFail):
'''
@ -41,17 +43,18 @@ def sniff_handler(media_file, **kw):
transcoder = transcoders.VideoTranscoder()
data = transcoder.discover(media_file.name)
_log.info('Sniffing {0}'.format(MEDIA_TYPE))
_log.debug('Discovered: {0}'.format(data))
if not data:
_log.error('Could not discover {0}'.format(
kw.get('media')))
return False
return None
if data['is_video'] == True:
return True
return MEDIA_TYPE
return False
return None
def process_video(proc_state):

3
mediagoblin/plugins/basic_auth/forms.py
View File

@ -41,3 +41,6 @@ class LoginForm(wtforms.Form):
normalize_user_or_email_field()])
password = wtforms.PasswordField(
_('Password'))
stay_logged_in = wtforms.BooleanField(
label='',
description=_('Stay logged in'))

4
mediagoblin/static/css/base.css
View File

@ -334,6 +334,10 @@ text-align: center;
width: 20px;
}
.boolean {
margin-bottom: 8px;
}
textarea#description, textarea#bio {
resize: vertical;
height: 100px;

13
mediagoblin/submit/views.py
View File

@ -19,6 +19,7 @@ import mediagoblin.mg_globals as mg_globals
from os.path import splitext
import logging
import uuid
_log = logging.getLogger(__name__)
@ -53,6 +54,10 @@ def submit_start(request):
try:
filename = request.files['file'].filename
# If the filename contains non ascii generate a unique name
if not all(ord(c) < 128 for c in filename):
filename = unicode(uuid.uuid4()) + splitext(filename)[-1]
# Sniff the submitted media to determine which
# media plugin should handle processing
media_type, media_manager = sniff_media(
@ -63,7 +68,7 @@ def submit_start(request):
entry.media_type = unicode(media_type)
entry.title = (
unicode(submit_form.title.data)
or unicode(splitext(filename)[0]))
or unicode(splitext(request.files['file'].filename)[0]))
entry.description = unicode(submit_form.description.data)
@ -133,9 +138,9 @@ def add_collection(request, media=None):
collection.generate_slug()
# Make sure this user isn't duplicating an existing collection
existing_collection = request.db.Collection.find_one({
'creator': request.user.id,
'title':collection.title})
existing_collection = request.db.Collection.query.filter_by(
creator=request.user.id,
title=collection.title).first()
if existing_collection:
add_message(request, messages.ERROR,

2
mediagoblin/templates/mediagoblin/edit/change_pass.html
View File

@ -39,7 +39,7 @@
Changing {{ username }}'s password
{%- endtrans -%}
</h1>
{{ wtforms_util.render_divs(form) }}
{{ wtforms_util.render_divs(form, True) }}
{{ csrf_token }}
<div class="form_submit_buttons">
<input type="submit" value="{% trans %}Save{% endtrans %}"

16
mediagoblin/templates/mediagoblin/media_displays/stl.html
View File

@ -108,32 +108,26 @@ window.show_things = function () {
<div style="padding: 4px;">
<a class="button_action" onclick="show('perspective');"
title="{%- trans %}Toggle Rotate{% endtrans -%}">
<a class="button_action" onclick="show('perspective');">
{%- trans %}Perspective{% endtrans -%}
</a>
<a class="button_action" onclick="show('front_view');"
title="{%- trans %}Front{% endtrans -%}">
<a class="button_action" onclick="show('front_view');">
{%- trans %}Front{% endtrans -%}
</a>
<a class="button_action" onclick="show('top_view');"
title="{%- trans %}Top{% endtrans -%}">
<a class="button_action" onclick="show('top_view');">
{%- trans %}Top{% endtrans -%}
</a>
<a class="button_action" onclick="show('side_view');"
title="{%- trans %}Side{% endtrans -%}">
<a class="button_action" onclick="show('side_view');">
{%- trans %}Side{% endtrans -%}
</a>
{% if media.media_data.file_type == "stl" %}
<a id="webgl_button" class="button_action"
onclick="show_things();"
title="{%- trans %}WebGL{% endtrans -%}">
onclick="show_things();">
{%- trans %}WebGL{% endtrans -%}
</a>
{% endif %}
<a class="button_action" href="{{ model_download }}"
title="{%- trans %}Download{% endtrans -%}"
style="float:right;">
{%- trans %}Download model{% endtrans -%}
</a>

4
mediagoblin/templates/mediagoblin/utils/comment-subscription.html
View File

@ -21,13 +21,13 @@
{% if not subscription or not subscription.notify %}
<a type="submit" href="{{ request.urlgen('mediagoblin.notifications.subscribe_comments',
user=media.get_uploader.username,
media=media.slug)}}"
media=media.slug_or_id)}}"
class="button_action">Subscribe to comments
</a>
{% else %}
<a type="submit" href="{{ request.urlgen('mediagoblin.notifications.silence_comments',
user=media.get_uploader.username,
media=media.slug)}}"
media=media.slug_or_id)}}"
class="button_action">Silence comments
</a>
{% endif %}

52
mediagoblin/templates/mediagoblin/utils/wtforms.html
View File

@ -34,26 +34,26 @@
{# Generically render a field #}
{% macro render_field_div(field, autofocus_first=False) %}
{{- render_label_p(field) }}
<div class="form_field_input">
{% if autofocus_first %}
{{ field(autofocus=True) }}
{% else %}
{{ field }}
{% endif %}
{%- if field.errors -%}
{% for error in field.errors %}
<p class="form_field_error">{{ error }}</p>
{% endfor %}
{%- endif %}
{%- if field.description %}
{% if field.type == 'BooleanField' %}
<label for="{{ field.label.field_id }}">{{ field.description|safe }}</label>
{% if field.type == 'BooleanField' %}
{{ render_bool(field) }}
{% else %}
{{- render_label_p(field) }}
<div class="form_field_input">
{% if autofocus_first %}
{{ field(autofocus=True) }}
{% else %}
<p class="form_field_description">{{ field.description|safe }}</p>
{{ field }}
{% endif %}
{%- endif %}
</div>
{%- if field.errors -%}
{% for error in field.errors %}
<p class="form_field_error">{{ error }}</p>
{% endfor %}
{%- endif %}
{%- if field.description %}
<p class="form_field_description">{{ field.description|safe }}</p>
{%- endif %}
</div>
{% endif %}
{%- endmacro %}
{# Auto-render a form as a series of divs #}
@ -86,3 +86,19 @@
</tr>
{% endfor %}
{%- endmacro %}
{# Render a boolean field #}
{% macro render_bool(field) %}
<div class="boolean">
<label for="{{ field.label.field_id }}">
{{ field }}</input>
{{ field.description|safe }}
</label>
{%- if field.errors -%}
{% for error in field.errors %}
<p class="form_field_error">{{ error }}</p>
{% endfor %}
{% endif %}
</div>
{% endmacro %}

12
mediagoblin/tests/test_auth.py
View File

@ -93,8 +93,8 @@ def test_register_views(test_app):
assert 'mediagoblin/user_pages/user.html' in template.TEMPLATE_TEST_CONTEXT
## Make sure user is in place
new_user = mg_globals.database.User.find_one(
{'username': u'happygirl'})
new_user = mg_globals.database.User.query.filter_by(
username=u'happygirl').first()
assert new_user
assert new_user.status == u'needs_email_verification'
assert new_user.email_verified == False
@ -128,8 +128,8 @@ def test_register_views(test_app):
# assert context['verification_successful'] == True
# TODO: Would be good to test messages here when we can do so...
new_user = mg_globals.database.User.find_one(
{'username': u'happygirl'})
new_user = mg_globals.database.User.query.filter_by(
username=u'happygirl').first()
assert new_user
assert new_user.status == u'needs_email_verification'
assert new_user.email_verified == False
@ -142,8 +142,8 @@ def test_register_views(test_app):
'mediagoblin/user_pages/user.html']
# assert context['verification_successful'] == True
# TODO: Would be good to test messages here when we can do so...
new_user = mg_globals.database.User.find_one(
{'username': u'happygirl'})
new_user = mg_globals.database.User.query.filter_by(
username=u'happygirl').first()
assert new_user
assert new_user.status == u'active'
assert new_user.email_verified == True

4
mediagoblin/tests/test_edit.py
View File

@ -190,8 +190,8 @@ class TestUserEdit(object):
assert urlparse.urlsplit(res.location)[2] == '/'
# Email shouldn't be saved
email_in_db = mg_globals.database.User.find_one(
{'email': 'new@example.com'})
email_in_db = mg_globals.database.User.query.filter_by(
email='new@example.com').first()
email = User.query.filter_by(username='chris').first().email
assert email_in_db is None
assert email == 'chris@example.com'

4
mediagoblin/tests/test_mgoblin_app.ini
View File

@ -13,8 +13,6 @@ tags_max_length = 50
# So we can start to test attachments:
allow_attachments = True
media_types = mediagoblin.media_types.image, mediagoblin.media_types.pdf
[storage:publicstore]
base_dir = %(here)s/user_dev/media/public
base_url = /mgoblin_media/
@ -34,3 +32,5 @@ BROKER_HOST = "sqlite:///%(here)s/user_dev/kombu.db"
[[mediagoblin.plugins.piwigo]]
[[mediagoblin.plugins.basic_auth]]
[[mediagoblin.plugins.openid]]
[[mediagoblin.media_types.image]]
[[mediagoblin.media_types.pdf]]

12
mediagoblin/tests/test_openid.py
View File

@ -186,8 +186,8 @@ class TestOpenIDPlugin(object):
openid_plugin_app.get('/auth/logout')
# Get user and detach from session
test_user = mg_globals.database.User.find_one({
'username': u'chris'})
test_user = mg_globals.database.User.query.filter_by(
username=u'chris').first()
Session.expunge(test_user)
# Log back in
@ -314,8 +314,8 @@ class TestOpenIDPlugin(object):
assert 'mediagoblin/edit/edit_account.html' in template.TEMPLATE_TEST_CONTEXT
# OpenID Added?
new_openid = mg_globals.database.OpenIDUserURL.find_one(
{'openid_url': u'http://add.myopenid.com'})
new_openid = mg_globals.database.OpenIDUserURL.query.filter_by(
openid_url=u'http://add.myopenid.com').first()
assert new_openid
_test_add()
@ -365,8 +365,8 @@ class TestOpenIDPlugin(object):
assert 'mediagoblin/edit/edit_account.html' in template.TEMPLATE_TEST_CONTEXT
# OpenID deleted?
new_openid = mg_globals.database.OpenIDUserURL.find_one(
{'openid_url': u'http://add.myopenid.com'})
new_openid = mg_globals.database.OpenIDUserURL.query.filter_by(
openid_url=u'http://add.myopenid.com').first()
assert not new_openid
_test_delete(self, test_user)

14
mediagoblin/tests/test_submission.py
View File

@ -26,7 +26,7 @@ from mediagoblin.tests.tools import fixture_add_user
from mediagoblin import mg_globals
from mediagoblin.db.models import MediaEntry
from mediagoblin.tools import template
from mediagoblin.media_types.image import MEDIA_MANAGER as img_MEDIA_MANAGER
from mediagoblin.media_types.image import ImageMediaManager
from mediagoblin.media_types.pdf.processing import check_prerequisites as pdf_check_prerequisites
from .resources import GOOD_JPG, GOOD_PNG, EVIL_FILE, EVIL_JPG, EVIL_PNG, \
@ -77,7 +77,7 @@ class TestSubmission:
return {'upload_files': [('file', filename)]}
def check_comments(self, request, media_id, count):
comments = request.db.MediaComment.find({'media_entry': media_id})
comments = request.db.MediaComment.query.filter_by(media_entry=media_id)
assert count == len(list(comments))
def test_missing_fields(self):
@ -122,7 +122,7 @@ class TestSubmission:
assert 'mediagoblin/user_pages/user.html' in context
def check_media(self, request, find_data, count=None):
media = MediaEntry.find(find_data)
media = MediaEntry.query.filter_by(**find_data)
if count is not None:
assert media.count() == count
if count == 0:
@ -219,7 +219,7 @@ class TestSubmission:
media = self.check_media(request, {'title': u'Balanced Goblin'}, 1)
assert media.media_type == u'mediagoblin.media_types.image'
assert isinstance(media.media_manager, img_MEDIA_MANAGER)
assert isinstance(media.media_manager, ImageMediaManager)
assert media.media_manager.entry == media
@ -240,8 +240,8 @@ class TestSubmission:
request = context['request']
media = request.db.MediaEntry.find_one({
u'title': u'UNIQUE_TITLE_PLS_DONT_CREATE_OTHER_MEDIA_WITH_THIS_TITLE'})
media = request.db.MediaEntry.query.filter_by(
title=u'UNIQUE_TITLE_PLS_DONT_CREATE_OTHER_MEDIA_WITH_THIS_TITLE').first()
assert media.media_type == 'mediagoblin.media_types.image'
@ -252,7 +252,7 @@ class TestSubmission:
response, context = self.do_post({'title': title}, do_follow=True,
**self.upload_data(filename))
self.check_url(response, '/u/{0}/'.format(self.test_user.username))
entry = mg_globals.database.MediaEntry.find_one({'title': title})
entry = mg_globals.database.MediaEntry.query.filter_by(title=title).first()
assert entry.state == 'failed'
assert entry.fail_error == u'mediagoblin.processing:BadMediaFail'

2
mediagoblin/tests/tools.py
View File

@ -164,7 +164,7 @@ def assert_db_meets_expected(db, expected):
for collection_name, collection_data in expected.iteritems():
collection = db[collection_name]
for expected_document in collection_data:
document = collection.find_one({'id': expected_document['id']})
document = collection.query.filter_by(id=expected_document['id']).first()
assert document is not None # make sure it exists
assert document == expected_document # make sure it matches

9
mediagoblin/tools/session.py
View File

@ -21,6 +21,8 @@ import crypto
_log = logging.getLogger(__name__)
MAX_AGE = 30 * 24 * 60 * 60
class Session(dict):
def __init__(self, *args, **kwargs):
self.send_new_cookie = False
@ -64,5 +66,10 @@ class SessionManager(object):
elif not session:
response.delete_cookie(self.cookie_name)
else:
if session.get('stay_logged_in', False):
max_age = MAX_AGE
else:
max_age = None
response.set_cookie(self.cookie_name, self.signer.dumps(session),
httponly=True)
max_age=max_age, httponly=True)

2
mediagoblin/user_pages/views.py
View File

@ -142,7 +142,7 @@ def media_home(request, media, page, **kwargs):
comment_form = user_forms.MediaCommentForm(request.form)
media_template_name = media.media_manager['display_template']
media_template_name = media.media_manager.display_template
return render_to_response(
request,