Loading the thumb/medium sizes from the config, saving
things to the db, and loading the image is now all done by
resize_tool. It still calls resize_image for the actual
work.
To get us moving towards a MediaManager class, the first
idea is to create a class that wraps our current dict based
manager and makes all users happy.
Instead of leaving test early if they can not run, use the
pytest.mark.skipif marked to tell the test system not to
even run the test.
This also adds to the stats, because skipped tests are
counted differently. Thus making it obvious, that some
tests did not run, because of any reason.
The new media type supports pdf and a subset of media recognized by libreoffice via
unoconv.
Every document added goes through:
* conversion to pdf with unoconv if not already a pdf
* creation of thumbnail and medium sized image, and pdfinfo generates
some information (even for unoconv produces docs - should fix this)
Poppler (pdftocairo, pdfinfo) is used. http://poppler.freedesktop.org/
A working but uglified pdf.js integration exists, which is enabled by
setting pdf.pdf_js=true mediagoblin_local.ini (disabled in mediagoblin.ini)
Adds one test to the test_submission test suite, and another separate test_pdf suite.
The tests are only run if media_types.pdf.processing.check_prerequisites passes, so
the test suite will not require any extra package.
TODO: make test suite say 'skipped' in that case instead of just 'ok'
Signed-off-by: Alon Levy <alon@pobox.com>
By doing this, we can take advantage of py.test's ability to create
temporary directories that are then cleaned up later during testing.
This helps for sandboxing things.
This also involves a ton of changes:
- Changing the get_app stuff appropriately, getting rid of the
setup_fresh_app decorator
- Making said fixture
- Switching over a billion tests to use it
This test fails, because it tries to set a slug with an "="
in it. And tests that the "=" gets replaced by "-".
Well, it currently is not being replaced.
The next (cherry picked) commit by Sebastian Spaeth fixes
this test!
nosetests runs everything that even vaguely looks like a
test case... even our get_test_app. And as it is imported
everywhere... it is run everywhere as a test case. Renaming
it saves us about 10+ tests and a few seconds of time.
We have a bunch of URLs that are more for internal use. At
least they're definitely not intended to be posted
somewhere for long term useage.
When those things affect a media, it's much better to
reference the media by its id. This can't change, ever.
This is better for races.
Like someone posting a comment while the owner
corrects a typo in the slug.
These tests, don't need fresh databases, so don't discard and recreate
the tables. This reduces test suite runtime on my laptop from 130 to 96
seconds.
Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
We were refering to model._id in most of the code base as this is
what Mongo uses. However, each use of _id required a) fixup of queries:
e.g. what we did in our find() and find_one() functions moving all
'_id' to 'id'. It also required using AliasFields to make the ._id
attribute available. This all means lots of superfluous fixing and
transitioning in a SQL world.
It will also not work in the long run. Much newer code already refers
to the objects by model.id (e.g. in the oauth plugin), which will break
with Mongo. So let's be honest, rip out the _id mongoism and live with
.id as the one canonical way to address objects.
This commit modifies all users and providers of model._id to use
model.id instead. This patch works with or without Mongo removed first,
but will break Mongo usage (even more than before)
I have not bothered to fixup db.mongo.* and db.sql.convert
(which converts from Mongo to SQL)
Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
In all cases where get_media_manager(_media_type_as_string) was called in
our code base we ultimately passed in a "MediaEntry().media_type" to get
the matching MEDIA_MANAGER. It so makes sense to make this a function of
the MediaEntry rather than a global function in mediagoblin.media_types and
passing around media_entry.media_type as arguments all the time.
It saves a few import statements and arguments. I also made it so the
Media_manager property is cached for subsequent calls, although I am not too
sure that this is needed (there are other cases for which this would make
more sense)
Also add a get_media_manager test to the media submission tests. It submits
an image and checks that both media.media_type and media.media_manager
return the right thing. Not sure if these tests could not be merged with an
existing submission test, but it can't hurt to have things explicit.
TODO: Right now we iterate through all existing media_managers to find the
right one based on the string of its module name. This should be made a simple
dict lookup to avoid all the extra work.
Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
Attributes of SQLAlchemy objects get "lost". So "cache"
them locally in the code.
This is really the simple explanation for some scarry
sqlalchemy details.
This merge involved moving the new FilenameBuilder class to
processing/__init__.py, and putting the comment deletion tests back into
test_submission.py using the refactored functions.
- Sniffing now goes through the old extension-based filter before
doing it the bitsniffing way.
- Refractored get_media_type_and_manager(filename).
- Removed ogg extension from video accepted extensions, audio
will take care of that.
- Added custom audio player, still WIP,but working.
- Added test for sniffing. This only tests for the
mediagoblin.media_types.image type, as that is the only
one enabled from start.
This test helps verify that bug #261 is actually fixed.
In order to test that all the processed images are smaller, I needed to add
an image that's bigger than processing.MEDIUM_SIZE, hence bigblue.png.
All the data posts in these tests had a lot of common code. Putting all
that into a function makes it easier to write more tests (which I'll be
doing in a bit) and see what's really being tested.
The actual code is just a simple for loop; there might be a better
implementation but this is a fine start. I also extended test_delete to
check this too.
* Added extlib/freesound/audioprocessing.py
* config_spec
* Added create_spectrogram setting
* Added media:medium and media:thumb max_{width,height} settings
* Added sniffing logic to
- audio.processing:sniff_handler
- video.processing:sniff_handler
* Changed audio.processing:sniff_handler logic
* Added audio thumbnailing functionality to audio.processing
(works only with create_spectrogram enabled)
* Refractored contexts in audio.processing
* Added audio.transcoders:AudioThumbnailer
Used for creating spectrograms and spectrogram thumbnails -
Wadsworth's Constant, we meet again :)
* audio.transcoders:AudioTranscoder
- Added mux_string kwarg
- Delete self.pipeline on self.halt()
* Changed str.format formatting in image.processing:sniff_handler
Had {1} without an {0}, changed to {0}
* Refractored VideoTranscoder to use transcode() for transcoding instead
of __init__()
* Added discover() method to video.transcoders:VideoTranscoder
* Added spectrogram display to media_displays/audio.html
* Updated test_submission to reflect changes in media plugin delegation
attachments are an optional part. But it doesn't hurt to
enable them in the test suite at all.
Also (with enabled attachmemtns) the main media view fails,
if one isn't logged in (joar found it!). So add a simple
(currently failing) test for this.
* Added ASCII media processing
* Added ASCII media display
* Added ASCII media type
Rebased from Joar Wandborg's ascii art branch (squashed to remove the
commits borrowing code of dubious license)
Fixed a bug in file submission error handling:
- Moved file-extension condition out of loop (what did it do there?)
- Updated file submission tests
- Changed error handling in file submission, should now report more
than absolutely necessary.
Some unit tests need a user in the database, especially to
act as that user. Some routines did that on their own. So
factored this whole thing into a new function and use it
around.
- Added handling of InvalidFileType to submit.views
- Updated test_celery_setup and test_submission tests to reflect the changes to the
media procesing infrastructure
Note: Migrations can't use "Dot Notation"!
Migrations run on pymongo, not mongokit.
So they can't use the "Dot Notation".
This isn't really a big issue, as migrations are anyway
quite mongo specific.
