In some cases (notably the mark_entry_failed function) it
is useful to have atomic update functionality on the db. On
mongo this requires special syntax.
So created an atomic_update function for mongo and started
to use it in mark_entry_failed.
So that the SQL backend is more useable, let the MediaEntry
have a faked media_data.
It's extremely fake: The returned dict is always a new one.
So any stored info is even lost!
Change the media_data for video from
entry.media_data["video"] to use entry.media_data directly.
Also start a bare MediaEntry.media_data_init(**kwargs)
method for setting up the media_data and possibly
initialising it with kwargs.
It's good practice to cleanup the SQL session after each
request so that the next request gets a fresh one.
It's an application decision whether one wants a
just-in-case ROLLBACK or COMMIT. There are two ideas behind
it, really. I have decided for ROLLBACK. The idea is "if
you forget to commit your changes yourself, there's
something broken. Maybe you got an exception?".
attachments working with the sql backend.
- SQL Schema for attachment files, ordering attachments by
their name, not by the submission order (as earlier).
- Dot-Notation for attachments, where missing.
- convert existing attachments over from mongo -> sql
Some parts in the code like to use .setdefault(). So make
them happy and provide a minimal version. It ignores the
given default and expects the attribute to already exist.
Other parts use .delete() to delete a complete object. This
version expects the object to live in a session and also
does the final commit.
To allow easier migration to the SQLAlchemy style .id give
the User and MediaEntry mongo classes an alias attribute of
.id that maps to ['_id'].
Use it in the upload process, because this was one of the
last positions with a ['_id'] instead of ._id (due to a bug
in mongokit).
Finally, to make testing of sql a bit easier, create a
bin/gmg command to do the conversion from mongo to sql.
It's currently named "convert_mongo_to_sql".
The most important option is the gmg -cf option to give a
configfile with the appropiate sql_engine definition.
Order the conversion by the "created" attribute. That way
the sql ids are mostly in the order they would have been,
if sql was used earlier.
Makes things nicer to look at in a db dump.
Mostly this means: Havintg a config_spec.ini that has a
local (relative to mediagoblin.ini) sqlite db with the name
"mediagoblin.db".
Also:
- Add to .gitignore
- Add a notice to mediagoblin.ini about the db
fail_metadata used to be a dict in mongo. So a json encoded
field should be okay too.
We could use a pickled field instead, which would be more
flexible.
After a bit of discussion, we decided to drop the
pre-rendered html from the database and render it on
the fly.
In another step, we will use some proper caching method to
cache this stuff.
This commit affects the MediaComment.content_html part.
After a bit of discussion, we decided to drop the
pre-rendered html from the database and render it on
the fly.
In another step, we will use some proper caching method to
cache this stuff.
This commit affects the MediaEntry.description_html part.
After a bit of discussion, we decided to drop the
pre-rendered html from the database and render it on
the fly.
In another step, we will use some proper caching method to
cache this stuff.
This commit affects the User.bio_html part.
Many thanks go to Svavar Kjarrval who has taken a deeper
look at our current sql db design and made a bunch of
suggestions. The suggestions are currently put as TODO
items in the docstrings. This way we can keep track of
them directly where we need it.
