The copy_locally and copy_local_to_storage (very inconsistent terms BTW)
were simply slurping in everything in RAM and writing it out at once.
(the copy_locally was actually memory efficient if the remote system was local)
Use shutil.copyfileobj which does chunked reads/writes on file objects.
The default buffer size is 16kb, and as each chunk means a separate HTTP
request for e.g. cloudfiles, we use a chunksize of 4MB here (which has
just been arbitrarily set by me without tests).
This should help with the failure to upload large files issue #419.
Check for CELERY_CONFIG_MODULE before we import raven.contrib.celery. It
seems that the import otherwise sets up the celery client before we get
to pass it our mediagoblin-specific settings.
It seems that (our implementation of) cloudfiles.write() takes
all existing data and appends write(data) to it, sending the
full monty over the wire everytime. This would of course
absolutely kill chunked writes with some O(1^n) performance
and bandwidth usage. So, override this method and use the
Cloudfile's "send" interface instead.
Also make the Cloudfile file wrapper an iterator that allows us to
simply do "for data in cloudfile:" which will stream the data in a
memory-efficient way.
DO NOTE THAT THIS PATCH IS COMPLETELY UNTESTED DUE TO LACK OF SETUP
PLEASE REVIEW AND VERIFY.
Removed the translation marking and passed in empty strings to avoid
WTForms automagically creating the labels from the field names (i.e.
client_id => 'Client Id').
instead of rendering it by hand, use the normal field
rendering tools.
Old:
[choose box] License preference
New:
License preference
[choose box]
This will be your default license on upload forms.
We had switched mg_globals to have the default translations set to
None so that it would be set up by the mediagoblin app later.
However, this would mean that things like scripts would try to call
gettext and error out.
Thanks to Tumulte for catching this.
This commit sponsored by Aurimas Fišeras. Thank you!
Thanks to tchernobog for catching this (it was breaking on postgres)
and Elrond for the suggestion on how to fix it.
This commit sponsored by Caleb Cooper. Thanks Caleb!
Issues of racism seem to have been resolved and removed from upstream,
but make having this as a dependency somewhat uncomfortable:
https://github.com/aFarkas/html5shiv/issues/91
Regardless, at the time of writing the project doesn't comply with its
own license... it states to be dual licensed under MIT and GPLv2 but
distributes neither of these licenses with its source.
Okay, that's a totally confusing statement, but the docstring of this
migration summarizes it well:
Entries without slugs now display differently in the url like:
/u/cwebber/m/id=251/
... because of this, we should back-convert:
- entries without slugs should be converted to use the id, if possible, to
make old urls still work
- slugs with = (or also : which is now also not allowed) to have those
stripped out (small possibility of breakage here sadly)
This commit sponsored by John Sullivan. Thanks johnsu01! :)
