The copy_locally and copy_local_to_storage (very inconsistent terms BTW)
were simply slurping in everything in RAM and writing it out at once.
(the copy_locally was actually memory efficient if the remote system was local)
Use shutil.copyfileobj which does chunked reads/writes on file objects.
The default buffer size is 16kb, and as each chunk means a separate HTTP
request for e.g. cloudfiles, we use a chunksize of 4MB here (which has
just been arbitrarily set by me without tests).
This should help with the failure to upload large files issue #419.
The reason for this is to avoid defining this twice as we were
previously (once in the template, once in video/models.py)
This commit sponsored by Roland McIntosh. Thank you!
It's kind of awkward because it relies on there being a entry.media_data,
but that's not guaranteed... (see http://issues.mediagoblin.org/ticket/650)
so we use a dopey fallback in the template in that case (kind of
annoying info duplication).
This commit sponsored by Piotr Wieczorek. Thank you!
Of course, the version that appears here is not really dangerous
because it's for the "call the file individually" form of debugging,
but it isn't allowed anyway.
This commit sponsored by Michael Faryniarz. Thanks!
This comes in several parts:
- Store the metadata from gstreamer during processing
- Add a new JSONEncoded field to the VideoData table
- And, of course, add a migration for that field!
This commit sponsored by Julius Tuomisto. Thank you, Julius!
- Update get_display_media in several ways:
- now uses the media type's own declaration of the order of things
- returns both the media_size and the media_path, as per the docstring
- implicitly uses self.media_files as opposed to forcing you to pass it in
- update videos to use get_display_media
- update images to declare media_fetch_order in the media manager (videos also)
- update stl to use media.media_files['original'] instead of weird
use of get_display_media
- update sidebar to only conditionally show webm_640
TODO still: identify video type information *during* processing, show
that in the <video><source /></video> element.
This commit sponsored by Nathan Yergler. Thanks, nyergler!
Check for CELERY_CONFIG_MODULE before we import raven.contrib.celery. It
seems that the import otherwise sets up the celery client before we get
to pass it our mediagoblin-specific settings.
It seems that (our implementation of) cloudfiles.write() takes
all existing data and appends write(data) to it, sending the
full monty over the wire everytime. This would of course
absolutely kill chunked writes with some O(1^n) performance
and bandwidth usage. So, override this method and use the
Cloudfile's "send" interface instead.
Also make the Cloudfile file wrapper an iterator that allows us to
simply do "for data in cloudfile:" which will stream the data in a
memory-efficient way.
DO NOTE THAT THIS PATCH IS COMPLETELY UNTESTED DUE TO LACK OF SETUP
PLEASE REVIEW AND VERIFY.
Removed the translation marking and passed in empty strings to avoid
WTForms automagically creating the labels from the field names (i.e.
client_id => 'Client Id').
instead of rendering it by hand, use the normal field
rendering tools.
Old:
[choose box] License preference
New:
License preference
[choose box]
This will be your default license on upload forms.
There's no reason to copy it over to 'webm_640' in such a case,
clearly.
Added logic so we don't do it twice either.
Haven't tested this yet though ;)
This commit sponsored by Algot Runeman. Thank you!
We had switched mg_globals to have the default translations set to
None so that it would be set up by the mediagoblin app later.
However, this would mean that things like scripts would try to call
gettext and error out.
Thanks to Tumulte for catching this.
This commit sponsored by Aurimas Fišeras. Thank you!
Thanks to tchernobog for catching this (it was breaking on postgres)
and Elrond for the suggestion on how to fix it.
This commit sponsored by Caleb Cooper. Thanks Caleb!
Issues of racism seem to have been resolved and removed from upstream,
but make having this as a dependency somewhat uncomfortable:
https://github.com/aFarkas/html5shiv/issues/91
Regardless, at the time of writing the project doesn't comply with its
own license... it states to be dual licensed under MIT and GPLv2 but
distributes neither of these licenses with its source.
Okay, that's a totally confusing statement, but the docstring of this
migration summarizes it well:
Entries without slugs now display differently in the url like:
/u/cwebber/m/id=251/
... because of this, we should back-convert:
- entries without slugs should be converted to use the id, if possible, to
make old urls still work
- slugs with = (or also : which is now also not allowed) to have those
stripped out (small possibility of breakage here sadly)
This commit sponsored by John Sullivan. Thanks johnsu01! :)
