182 Commits

Author SHA1 Message Date
Boris Bobrov
2b4c339de6 Fix bug 647
Reading a file into memory resulted in depletion of memory. Now files
are read and written lazily, by chunks.
2015-03-02 16:44:13 +03:00
Boris Bobrov
54b4b28f84 Add new hook for two-step media type checking
Before uploaded media files were checked by extension. This led to
situations when a plugin can support file with specific extension but
doesn't due to lack of codecs, for example. Since the plugin reported
that it supports uploaded file type, the upload was being declared
successful, but transcoding failed.

The failures were not easy to debug.

The change adds a new hook that could allow two-step checking of the
content. The result of the hook execution returns a tuple with
media type name, manager and a callable sniffer, that can be used to
perform probably expensive checks of the content.

Also the change adds implementation of the hook for video.
2015-02-16 13:41:09 +03:00
Jessica Tallon
3588522606 Fix #1054 - Add missing attributes to Activity.serialize 2014-12-08 14:46:48 +00:00
Jessica Tallon
5436d980a1 Fix #1024 - Correctly set generator on Activities 2014-12-02 14:51:24 +00:00
Jessica Tallon
bc2c06a10d Fix #1017 - Exception caused by activity being set on incorrect foreign key
The .activity ForeignKey on several models such as MediaEntry expects
	a ActivityIntmediatory ID not an Activity ID however in submit code the
	activity attribute was being set the Activity, simply removing this
	assignment should fix the issue as everything is set correctly at this point.

	I have also moved the creation of the activity above the processing of media
	to avoid race conditions.
2014-10-29 08:47:12 +00:00
Jessica Tallon
63c865790a Only create a slug after a media upload via the API when there is a title 2014-10-22 21:42:57 +01:00
Jessica Tallon
9a1fc423ac Merge branch '905-activities'
Add Activity and Generator models which allow for
    activities to be created. This now works with the
    feed API.
2014-10-07 10:01:38 +01:00
Jessica Tallon
6d36f75f84 Fix all the unit tests and clean up code 2014-10-06 11:48:03 +01:00
Christopher Allan Webber
f6bad0eb26 Merge branch 'master' into merge-python3-port
Has some issues, will iteratively fix!

Conflicts:
	mediagoblin/gmg_commands/__init__.py
	mediagoblin/gmg_commands/deletemedia.py
	mediagoblin/gmg_commands/users.py
	mediagoblin/oauth/views.py
	mediagoblin/plugins/api/views.py
	mediagoblin/tests/test_api.py
	mediagoblin/tests/test_edit.py
	mediagoblin/tests/test_oauth1.py
	mediagoblin/tests/test_util.py
	mediagoblin/tools/mail.py
	mediagoblin/webfinger/views.py
	setup.py
2014-09-16 14:01:43 -05:00
Jessica Tallon
b949201152 Create activity model and add activity creation
This creates the Activity and Genrator models from the Activity
Streams spec and. I then created a migration which retro-actively
create activities for media uploaded and comments created. Through
out the code I've added so automatically activties are created when
a user peforms an action (uploading media, commenting, etc.).
2014-08-22 23:18:01 +01:00
Jessica Tallon
9246a6ba89 Tidy up federation code and add tests to cover more of the APIs 2014-08-18 10:51:32 -05:00
Berker Peksag
cda3055bd6 Fix another tests.
(forgot to commit earlier)
2014-08-07 13:08:42 +03:00
Jessica Tallon
5e5d445890 Fix #927 - Clean up federation code after Elrond's review
- Add json_error and use inplace of json_response where appropriate.
- Add garbage_collection to config spec file.
- Fix bugs in both garbage collection task and test
- Handle /api/whoami when no user logged in and a test for such a case.
- Validate ID is correct and user has comment privilege to comment.
2014-07-30 21:53:52 +01:00
Loïc Le Ninan
0742e11dff Fixes #899 : DeprecationWarning about Required going away in WTForms 3.0. Replaced Required with InputRequired. 2014-06-13 10:23:07 -05:00
Berker Peksag
e49b7e02b2 Use six.text_type instead of unicode().
I will be switch to use ``from __future__ import unicode_literals`` later.
2014-06-02 20:59:28 +03:00
tilly-Q
2daf8ec000 Fixed a small error relating to the default value of media_metadata 2014-05-14 12:34:13 -04:00
tilly-Q
45f426ddee Made it possible to submit media with the metadata provided 2014-04-21 12:11:12 -04:00
Christopher Allan Webber
131b749529 Making the API make use of the new submit tooling.
This commit sponsored by Zakkai Kauffman-Rogoff.  Thanks! :)
2013-11-14 14:04:32 -06:00
Christopher Allan Webber
c802c41af4 Pass in request.urlgen to submit_media so it can update the feed_url stuff
This commit sponsored by Jesus Alberto Rodriguez Chavez.  Thank you!
2013-11-14 13:53:07 -06:00
Christopher Allan Webber
70b2f1ec0c Switching use of submit_media to be keyword arguments in views.py.
This should make things a bit easier for me to fiddle with argument
ordering.

This commit sponsored by Maira Sutton.  Thanks!
2013-11-14 13:41:09 -06:00
Christopher Allan Webber
5d754da741 Breaking out get_upload_file_limits into its own utility.
This commit sponsored by Mikael Nordfeldth.  Thank you!
2013-11-14 11:06:10 -06:00
Christopher Allan Webber
cb7716f3b5 A couple more adjustments for unicode'ification of some arguments.
This commit sponsored by Bruno Girin.  Thank you!
2013-11-14 10:01:41 -06:00
Christopher Allan Webber
301da9caba Generic'ifying the submit code to not rely on the werkzeug FileStorage datastructure
Important, because that only makes sense for wsgi! :)

This commit sponsored by Geoff Lehr.  Thank you!
2013-11-13 19:40:13 -06:00
Christopher Allan Webber
5202924c91 Docstring for submit_media()
This commit sponsored by Benjamin Prager.  Thank you!
2013-11-13 17:25:33 -06:00
Christopher Allan Webber
9e15c674ce Refactoring submission utility to make it more generic! Reusable!
This commit sponsored by Joar Wandborg.  Joar, thanks for the many
things you've done for MediaGoblin!
2013-11-13 10:22:46 -06:00
Christopher Allan Webber
1779a0702f Starting to split out a submit_media utility 2013-11-12 14:03:17 -06:00
Christopher Allan Webber
56c4ad89eb Merge remote-tracking branch 'refs/remotes/tilly-q/OPW-Moderation-Update'
Conflicts:
	mediagoblin/templates/mediagoblin/user_pages/user.html
	mediagoblin/tests/test_auth.py
	mediagoblin/tests/test_submission.py
2013-10-07 15:48:33 -05:00
tilly-Q
045fe0ee9d Merge branch 'master' into OPW-Moderation-Update
Conflicts:
	mediagoblin/db/migrations.py
2013-09-12 18:58:04 -04:00
tilly-Q
8e91df8734 I did some more code-keeping in this commit. I added a lot of documentation, so
that most of my functions do indeed have effective docstrings. I also changed
the decorators so that they imply eachother in a logical way. I also modified
the one decorator get_media_comment_by_id to be more usable with the variable
urls of mediagoblin.user_pages.views:file_a_report. I also noticed a few tests
had broken, so I went through them and fixed them up, finding that mostly there
were problems in my actual writing of the tests. I also did a few other small
tasks such as creating a new User method to check whether or not a User is ban-
-ned.

===============================================================================
    Added in documentation
===============================================================================
--\  mediagoblin/db/models.py
--\  mediagoblin/decorators.py
--\  mediagoblin/moderation/forms.py
--\  mediagoblin/moderation/tools.py
--\  mediagoblin/moderation/views.py
--\  mediagoblin/user_pages/lib.py

===============================================================================
    Rearranged decorators to be more efficient
===============================================================================
--\  mediagoblin/decorators.py
--| Made it so that user_not_banned is encapsulated in require_active_login
--| Made it so that require_active_login is encapsulated in user_has_privilege
--| Changed get_media_comment_by_id into get_optional_media_comment_by_id. It
  | now returns valid code if the MediaComment id is absent. This makes it pos-
  | -sible to use this decorator for the function:
  |         mediagoblin.user_pages.views:file_a_report

--\  mediagoblin/user_pages/views.py
--| Replaced the mediagoblin.user_pages.views:file_a_comment_report with the
  | decorator mentioned above

--\  mediagoblin/user_pages/routing.py

        -----------------------------------------------------------
        |     took out unnecessary @user_not_banned decorators    |
        -----------------------------------------------------------
--\  mediagoblin/submit/views.py
--\  mediagoblin/user_pages/views.py

===============================================================================
    Fixed broken tests
===============================================================================
--\  mediagoblin/tests/test_auth.py
--\  mediagoblin/tests/test_privileges.py
--\  mediagoblin/tests/test_submission.py

===============================================================================
    Fixed broken code
===============================================================================
--\  mediagoblin/tools/response.py

===============================================================================
    Other Tasks
===============================================================================
--\  mediagoblin/db/models.py
--| Added in User.is_banned() method
--\  mediagoblin/decorators.py
--| Utitilized User.is_banned() method in the user_not_banned decorator

--\  mediagoblin/moderation/views.py
--| Made it impossible for an admin to ban themself.
--| Got rid of a vestigial print statement

--\  mediagoblin/templates/mediagoblin/base.html
--| Made it so the top panel does not show up for users that are banned.

--\  mediagoblin/templates/mediagoblin/moderation/user.html
--| Rearranged the javascript slightly

===============================================================================
2013-09-03 16:19:07 -04:00
tilly-Q
1bb367f613 This is a quick commit. I gave admins the ability to ban or unban users
straight from the moderation.users_detail page. I also changed the
UserBan.expiration_date type from DateTime into Date. I also began work on the
Terms of Service, pulled from another website (which will be cited clearly
before I'm done). I added new tests as well for the ban/unbanning. Lastly,
I added a few `user_not_banned` decorators to relevant views, so banned users
cannot access any pages.
2013-08-29 17:31:19 -04:00
Rodney Ewing
0b95003cd4 use media.get_uploader and fix typo 2013-08-26 06:33:30 -07:00
Rodney Ewing
2188925bab javascript limit validation 2013-08-26 06:33:30 -07:00
Rodney Ewing
a80ea74702 fix after rebase 2013-08-26 06:33:30 -07:00
Rodney Ewing
ecb4512822 max file size 2013-08-26 06:33:30 -07:00
Rodney Ewing
001a50a850 fixed tests and defaults 2013-08-26 06:33:29 -07:00
Rodney Ewing
91a52878cf made no upload limit the default 2013-08-26 06:33:29 -07:00
Rodney Ewing
150bee3f8f typos 2013-08-26 06:33:28 -07:00
Rodney Ewing
c3cce7564a added tests 2013-08-26 06:33:28 -07:00
Rodney Ewing
bdd2242155 added user upload limits 2013-08-26 06:33:28 -07:00
Christopher Allan Webber
f6497ce572 Merge remote-tracking branch 'refs/remotes/rodney757/reprocessing'
Conflicts:
	mediagoblin/processing/task.py
	mediagoblin/submit/lib.py
2013-08-21 12:39:38 -05:00
tilly-Q
9e204e49c9 Merge branch 'master' into OPW-Moderation-Update
Conflicts:
	mediagoblin/db/models.py
	mediagoblin/decorators.py
	mediagoblin/routing.py
	mediagoblin/user_pages/views.py
2013-08-20 12:21:13 -04:00
Rodney Ewing
b505952508 -update to latest master
- have mg generate task_id

remove
2013-08-19 15:26:06 -07:00
Sebastian Spaeth
bf2dafd1a0 Tweak Celery Task
- Make sure Exceptions are pickleable (not sure if this was not the
  case but this is the pattern as documented in the celery docs.
- Don't create a task_id in the GMG code, but save the one
  implicitely created by celery.
- Don't create a task-id directory per upload. Just store queued uploads
  in a single directory (this is the most controversial change and might
  need discussion!!!)

Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
2013-08-19 14:42:13 -07:00
Christopher Allan Webber
98d1fa3bed Fixing normal submission of media (well for images anyway) 2013-08-16 15:30:17 -07:00
Christopher Allan Webber
77ea4c9bd1 Updating to the point where we can allllmost run with the new reprocessing code
This commit sponsored by Odin Hørthe Omdal.  Thank you!
2013-08-16 15:30:16 -07:00
Rodney Ewing
9a2c66ca9e added image reprocessing 2013-08-16 15:30:14 -07:00
Rodney Ewing
b26edfb944 Revert "Leave slug empty until we are sure media processing was successful."
This reverts commit f67611fb485b5a84cedc62b73beb1e551e8cb934.

For some reason, generating a slug here throws an integrity error during
a query when there is a duplicate slug.
2013-08-08 15:49:33 -07:00
Rodney Ewing
f67611fb48 Leave slug empty until we are sure media processing was successful.
Patch submitted by LotusEcho
2013-08-07 16:15:27 -07:00
tilly-Q
52a355b275 Merge branch 'ticket-679' into OPW-Moderation-Update
Conflicts:
	mediagoblin/auth/tools.py
	mediagoblin/auth/views.py
	mediagoblin/db/migration_tools.py
	mediagoblin/db/migrations.py
	mediagoblin/db/models.py
	mediagoblin/decorators.py
	mediagoblin/user_pages/views.py
2013-07-29 18:40:19 -04:00
Rodney Ewing
44082b12d8 Patch by Strum. Ticket #451 - Convert all mongokit style .find, .find_one, .one calls over to SQLAlchemy queries 2013-07-11 14:17:50 -07:00