99a54c0095
The copy_locally and copy_local_to_storage (very inconsistent terms BTW) were simply slurping in everything in RAM and writing it out at once. (the copy_locally was actually memory efficient if the remote system was local) Use shutil.copyfileobj which does chunked reads/writes on file objects. The default buffer size is 16kb, and as each chunk means a separate HTTP request for e.g. cloudfiles, we use a chunksize of 4MB here (which has just been arbitrarily set by me without tests). This should help with the failure to upload large files issue #419.
======== README ======== What is GNU MediaGoblin? ======================== * Initially, a place to store all your photos that’s as awesome as, if not more awesome than, existing network services (Flickr, SmugMug, Picasa, etc) * Customizable! * A place for people to collaborate and show off original and derived creations. Free, as in freedom. We’re a GNU project after all. * Later, a place for all sorts of media, such as video, music, etc hosting. * Later, federated with OStatus! Is it ready for me to use? ========================== Yes! But with caveats. The software is usable and there are instances running, but it's still in its early stages. Can I help/hang out/participate/whisper sweet nothings in your ear? =================================================================== Yes! Please join us and hang out! For more information on where we hang out, see `our Join page <http://mediagoblin.org/join/>`_ Where is the documentation? =========================== The beginnings of a site administration manual is located in the ``docs/`` directory in HTML, Texinfo, and source (Restructured Text) forms. It's also available online at http://docs.mediagoblin.org/ in HTML form. Contributor/developer documentation as well as documentation on the project processes and infrastructure is located on `the wiki <http://wiki.mediagoblin.org/>`_.